Port forwarding & OpenVPN?

penartur · Feb 4, 2018, 7:45 PM

I have read other topics discussing port forwarding with openvpn, but nothing mentioned there seems to help in my case.

So I have the following setup:

Internet-based client, let's say 1.1.1.1
Internet-based pfsense with OpenVPN server; let's say its public IP is 2.2.2.2. OpenVPN is configured to use /30 networks; let's say OpenVPN address is 192.168.0.5.
Private pfsense with OpenVPN client, OPENVPN_MANUAL interface for OpenVPN connection and two private networks (CLIENTS and SERVICES). Its OpenVPN address is 192.168.0.6, its address in CLIENTS network is 192.168.1.1, its address in SERVICES network is 192.168.2.1.
Private PC in CLIENTS network, 192.168.1.2.
Private server in SERVICES network, 192.168.2.2.
Port forwarding from internet-based pfsense to private pfsense: all traffic coming from the WAN interface to internet-based pfsense on port 10443 is forwarded to 192.168.0.6:10443.
Three identical (mutatis mutandis) port forwardings from private pfsense to the private server; all traffic coming from the OpenVPN interface OR OPENVPN_MANUAL interface OR CLIENTS interface on port 10443 is forwarded to 192.168.2.2:443.
All relevant firewall rules are enabled.
Both pfSenses are at 2.4.2 version; there is not a lot of customization, and most of the settings are set to default values.

What does not work:

Public client can not connect to private server: 1.1.1.1->2.2.2.2:10443->192.168.0.6:10443->192.168.2.2:443 does not work.
Public-based pfsense can not test private pfsense 10443 port (on "test port" page, attempt to test 192.168.0.6:10443 fails): 2.2.2.2->192.168.0.6:10443->192.168.2.2:443 does not work.

What does work:

Private PC (192.168.1.2) can successfully connect to the private server: 192.168.1.2->192.168.1.1:10443->192.168.2.2:443 works.
Internet-based pfsense can successfully test port 443 of the private pfsense (on "test port", attempt to test 192.168.0.6:443 succeeds): 2.2.2.2->192.168.0.6:443 works.
Reconfiguring port forwarding on public-based pfsense to redirect traffic to 192.168.0.6:443 (so that it'll get to private pfsense webConfigurator instead of port forwarding) works fine: 1.1.1.1->2.2.2.2:10443->192.168.0.6:443 works.
If I enable logging for private pfsense rule allowing traffic from openvpn to private server, and attempt to connect to private server from the public client (by going to 2.2.2.2:10443), there are two messages in private pfsense firewall logs: "allow / OPENVPN_MANUAL / NAT (long rule ID) / 1.1.1.1:55586 / 192.168.2.2:443", and another for the source port 55587.
If I attempt to connect to private server from the public client, then on the "states" page, four entries briefly appear: "OPENVPN_MANUAL / tcp / 1.1.1.1:55586 -> 192.168.2.2:443 (192.168.0.6:10443) / CLOSED:SYN_SENT / 2, 0 / 104B, 0B", "SERVICES / tcp / 1.1.1.1:55586 -> 192.168.2.2:443 / SYN_SENT:CLOSED / 2, 0 / 104B, 0B", and the same pair of messages for source port 55587.
If I attempt to connect to private server from the private PC (by going to 192.168.1.1:10443), then on the "states" page, there are three pairs of entries, all three with different source ports, all with non-zero incoming and outgoing packets and bytes, two of these with "TIME_WAIT:TIME_WAIT" and one with "ESTABLISHED:ESTABLISHED". I guess that it is the correct behavior, and "SYN_SENT:CLOSED" is not. Also I have noted that every message in "states" contains the pfsense IP in "original destination", while for connections from public IP, messages for SERVICES network do not contain it.

So I'm out of the ideas to try. It seems that there is some problem on private pfsense for port forwarding from openvpn network to the private network (because connections from public client to private pfsense itself works, as does port forwarding between private networks). Additionally, it seems that the port forwarding is only half broken (as there are some entries on "states" page). What could be the problem?

penartur · Feb 4, 2018, 9:21 PM

So it turned out that in addition to configuring port forwarding, I also had to switch outbound NAT mode to "Hybrid Outbound NAT" on both pfsense instances, and add the corresponding rules there. For example, on private pfsense, in addition to Port Forwarding rule "everything coming to 192.168.0.6:10443 should be forwarded to 192.168.2.2:443", I also had to add an outbound NAT rule "everything destined to 192.168.2.2:443 on SERVICES interface should be translated".

The original problem, it seems, had nothing to do with OpenVPN. Connections from private PC had originally worked because I had a gateways set up on both SERVICES and OPENVPN_MANUAL interfaces, and so there already was an automatic outbound rule "everything coming from CLIENTS subnet to anything on SERVICES interface should be translated", but there was no rules for anything coming from anybody else to SERVICES.

Now, with these added outbound NAT rules, everything works. But I'm sure that I'm doing something wrong; to implement port forwarding certainly should not require to create port forwarding rule AND and outbound NAT rule; the former should be enough. So could anyone please clarify what is the correct way to implement port forwarding with pfSense?

Derelict · Feb 4, 2018, 9:47 PM

When you receive connection INTO an OpenVPN interface from arbitrary addresses and do not accept the default route on that OpenVPN:

You have to assign an interface to that OpenVPN instance

You have to be sure that the incoming traffic DOES NOT MATCH a rule on the OpenVPN TAB and DOES MATCH a rule on the assigned interface tab.

Note that I did not spend an hour deciphering that wall of text there. I just saw "openvpn" and "port forwards"

A diagram would be far, far, better than that.

I drew one just for you in my sig. You can convert your scenario to the addressing scheme there and use it to describe what you are trying to do.

penartur · Feb 4, 2018, 11:00 PM

@Derelict:

When you receive connection INTO an OpenVPN interface from arbitrary addresses and do not accept the default route on that OpenVPN:

You have to assign an interface to that OpenVPN instance

You have to be sure that the incoming traffic DOES NOT MATCH a rule on the OpenVPN TAB and DOES MATCH a rule on the assigned interface tab.

Note that I did not spend an hour deciphering that wall of text there. I just saw "openvpn" and "port forwards"

A diagram would be far, far, better than that.

I drew one just for you in my sig. You can convert your scenario to the addressing scheme there and use it to describe what you are trying to do.

The confusing wall of text is there because I've got confusing results while trying to diagnose the issue.

Since then, my original issue was solved (in a second post), and it seems that it has nothing to do with OpenVPN.
The simplified scheme of my original post (only parts relevant to the second post remain), applied mutatis mutandis to the one from your signature, is that: some PC in WAN is trying to connect to Host B1.
All relevant firewall rules are enabled. The only gateway on pfSense B is on WAN interface. There is a single port forwarding rule on pfSense B, forwarding to 172.25.233.100:443 all requests destined to 172.25.228.9:10443. This port forwarding does not work ("states" are as described in the first post).
Now if I will switch outbound NAT mode to "Hybrid Outbound NAT", and add an outbound NAT rule "everything destined to 172.25.233.100:443 on LAN interface should be translated", port forwarding starts to work, and PCs in WAN connect successfully to 172.25.233.100:443 via 172.25.228.9:10443.
It seems to me that enabling port forwarding should only require adding relevant port forwarding rule, and definitely should not require touching outbound NAT settings. Am I correct? How, then, should I configure port forwarding?

Derelict · Feb 4, 2018, 11:27 PM

You should look at all of the things here:

https://doc.pfsense.org/index.php/Port_Forward_Troubleshooting

The most common reason for what you are seeing is the firewall ON HOST 172.25.233.100 does not allow connections from outside its local subnet. Making the connection appear to be same-subnet to THAT HOST's local firewall (Sourced from 172.25.233.1 via Outbound NAT) means the traffic is allowed.

Another common reason for what you are seeing is the default gateway on host 172.25.233.100 is not pfSense B.

Creating an Outbound NAT rule on LAN means that the reply traffic is same-subnet to 172.25.233.100 (172.25.233.100 talking to 172.25.233.1 is an ARP event, not a routing event) so the routing table does not come into play.

All of this is listed in the referenced link.

penartur · Feb 4, 2018, 11:56 PM

You are right, Host B1 indeed does not have pfSense as its default gateway (problem 3 on that troubleshooting wiki page). Of course I have read the entire page several times before creating this topic; now I cannot believe that I spent 8 hours diagnosing, checking and re-checking things, but somehow missed that obvious point.

I've just checked that Host B1 logs, and it indeed seems to believe that the traffic comes from 172.25.233.1.

It doesn't matter for me, as all the traffic comes from Cloudflare anyway; however, it might be of use to other readers of this topic. So the only potentially remaining question now is this: is it possible to make port forwarding work correctly (with correct incoming IP detection) without having to configure pfSense as the default gateway on server? I guess not.

Derelict · Feb 5, 2018, 12:02 AM

To all other readers: https://doc.pfsense.org/index.php/Port_Forward_Troubleshooting (Check (really check) everything there!!!!!)

No, it is not possible. That host's routing table prevails. If that host happens to have some reply-to magic like pfSense does, then maybe. But that would be a subject for that host's support forum.