NAT to WAN not working when openvpn service is running

Nischi · May 23, 2021, 3:40 PM

Hi

So I'm just a novice here, so please bear with me, I've tried searching these forums for a solution before posting this.

Network is as follows.

ISP --> WAN Pfsense --> LAN and opnvpn LAN clients

On Pfsense I have a openvpn client running. I have some specific clients which are routed out the openvpn, as well as a few NAT/port forwards which are working good for these openvpn clients.

My problem is that my port forward/NAT for LAN clients not using openvpn is not working. The ports are closed when testing through sites like http://ismyportopen.com/

Although I have to mention that UDP seems to be working regardless as Warzone is specifying NAT: Open when playing the game.

If I stop the openvpn service, the NAT for the LAN clients starts to work!

I've tried the automatic fix in the Troubleshooting Asymmetric Routing chapter to no avail (setting Static route filteringBypass firewall rules for traffic on the same interface)

One client I'm trying to NAT is 192.168.1.132 using port 8096.
My port forward looks like this, and it is the first rule.

When the openvpn service is running, I get the the following in my firewall logs:
May 23 14:06:07 LAN Default deny rule IPv4 (1000000103) 192.168.1.132:8096 64.111.122.99:46240 TCP:SA

As far as I've understood that means that nothing in my defined rules list matched, and that the default deny rule set in.

Outbound can me seen below.

Nischi · May 23, 2021, 3:40 PM

@nischi Edit. Added text file for rules.debugrules.debug.txt

viragomann · May 23, 2021, 7:26 PM

@nischi said in NAT to WAN not working when openvpn service is running:

I have some specific clients which are routed out the openvpn

How did you do this?

Which pfSense version?

Post the routing table, please (Diagnostic > Routes).

Nischi · May 23, 2021, 9:11 PM

@viragomann version 2.5.1

See previous image for outbound.

Nischi · May 24, 2021, 8:06 AM

@nischi Should I post something more to help diagnose with?

viragomann · May 24, 2021, 9:15 AM

@nischi
Not clear, why this happens here. There is a known bug on 2.5.1 concerning port forwarding (Port forward works only on interface with default gateway, does not work for alternative wans (CE Only)), however this should not be the case here.

It seems to me like response packets on the forwards are routed out to the VPN gateway. You can check that out by sniffing the traffic on the OpenVPN interface while trying to access from WAN using a port checker to be sure, what's going on.

Is the 192.168.1.132 a member of vpn_clients?

Nischi · May 24, 2021, 11:53 AM

@viragomann the 192.168.1.132 is a client that is connecting to WAN without openvpn. It's for that client(and any other client not on the openvpn) I can't get the port forwarding to work unless I stop the openvpn service.

I found a thread I thought had a similar problem(I think), but I couldn't get the solution to work since it was depricated.
https://forum.netgate.com/topic/128238/nat-stops-working-in-multi-wan-when-primary-wan-goes-down/6
That thread suggested using "Default gateway switching".

Note that the following captures are made with "full" level of detail.
See the text-file 1 1.txt for what was captured on the WAN-interface during the port lookup of 8096 which is NAT/port forwarded to 192.168.1.132

See the text-file 2 2.txt for what was captured on the OpenVPN-client-interface doing the same port 8096 lookup from WAN.

See the text-file 3 3.txt for what was captured on the VPN-interface doing the same port 8096 lookup from WAN.

Nischi · May 24, 2021, 12:22 PM

@viragomann Hey, I just had to try since you mentioned the open issue earlier. I have updated to 2.6.0.a.20210524.0100 DEV, and it's working as expected now... Thank you for taking your time and pushing me toward the right direction!

viragomann · May 24, 2021, 12:29 PM

@nischi said in NAT to WAN not working when openvpn service is running:

See the text-file 1 1.txt for what was captured on the WAN-interface during the port lookup of 8096 which is NAT/port forwarded to 192.168.1.132

WTH! This capture shows that 192.168.1.132 is responding without NAT! The packets go out with the internal IP as source, which cannot be routed.
No clue, where this comes from here.

And if you turn off the vpn this works as expected?
There should not be any different regarding this.

Your setup seems to me like pfSense is virtualized on an host, where the WAN interface is passed through to pfSense, but the LAN is a virtual NIC.
Is it possible that there is something wrong with this setup? Maybe some traffic is bypassed.
Is the WAN exclusively used by pfSense?

Nischi · May 24, 2021, 1:21 PM

@viragomann I can't test it again as I've updated. But yes, the case was that if I turned off the openvpn service it was working as expected for the non-vpn clients.

I'm running pfsense inside a VM on unraid(QEMU), with a physical NIC for WAN, and a virtual NIC for LAN. The WAN is exlusive for pfsense and is isolated.

I was very confused by all of this as I'm sure I had these settings working last year. Must have been something in 2.5 that got solved now by the issue mentioned earlier in 2.6.

viragomann · May 24, 2021, 1:21 PM

@nischi
Last your was 2.4.5. 2.5.0 came out last February.
Maybe a roll-back is an option for you.

https://docs.netgate.com/pfsense/en/latest/releases/versions.html

Nischi · May 24, 2021, 1:50 PM

@viragomann btw how could you discern that I was running virtualized?

viragomann · May 24, 2021, 1:52 PM

@nischi
The routing table shows the network ports: vtnet0, em0.

Nischi · May 24, 2021, 4:38 PM

@viragomann oh right, thanks :)