Firewall drops packets between LAN and OPT1

WhiteHawk

Hello All,

I'd really appreciate help with the following issue:

I have the OpenVPN service set up, clients (e.g. Android phone, cloud VM) can connect and ping hosts on the LAN.

But TCP connections don't work at all from the phone and occasionally work from the cloud VM.

When trying to troubleshoot, I found tons of similar logs for LAN and OPT1 in the firewall log:

The two interfaces involved:

LAN rules:

OPT1 rules:

Gateways:

Things I've tried:

relaxing firewall rules
list itemhttps://docs.netgate.com/pfsense/en/latest/troubleshooting/asymmetric-routing.html#automatic-fix as I noticed that the cloud VM does not see the gateway:

Unknown adapter OpenVPN TAP-Windows6:

Connection-specific DNS Suffix . : home
Description . . . . . . . . . . . : TAP-Windows Adapter V9
Physical Address. . . . . . . . . : 00-FF-EE-73-CD-35
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::e70a:b3d6:3d87:1c26%65(Preferred)
IPv4 Address. . . . . . . . . . . : 192.168.3.2(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Lease Obtained. . . . . . . . . . : 2024. január 27., szombat 18:25:10
Lease Expires . . . . . . . . . . : 2025. január 26., vasárnap 18:25:09
Default Gateway . . . . . . . . . :
DHCP Server . . . . . . . . . . . : 192.168.3.0
DHCPv6 IAID . . . . . . . . . . . : 1090584558
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-2C-77-9D-B2-00-22-48-A3-8B-DA
DNS Servers . . . . . . . . . . . : 192.168.2.1
NetBIOS over Tcpip. . . . . . . . : Enabled

Thanks for any pointers!

johnpoz

@WhiteHawk All of those blocks are out of state blocks.. notice they are all PA (psh flag on an ack) and one is even fin,ack - ie close this session.

there is nothing showing pfsense blocking any sort of traffic that it shouldn't block.. You need to figure out where your state is going, or why you have traffic hitting pfsense after the state has expired..

Just went over in great detail blocks caused by lack of state.. in this thread.

https://forum.netgate.com/topic/185762/need-help-resolving-asymmetric-routing-issue-in-a-network-with-pfsense-and-netgear-managed-switch-gs724tv4

WhiteHawk

Thanks @johnpoz for your reply.

I took the time to go through the linked post.

I don't see any syn flags in the firewall logs, in fact nothing comes up if I filter between 2 impacted hosts (192.168.3.2 -> 192.168.2.2:32400)

Sometimes I can even catch the states:

But packet capture taken on OPT1 clearly shows that packets don't get acked from seq=222 by the VPN client (Android phone with OpenVPN):

So this must be the OpenVPN connection - how to continue from here?

Derelict

@WhiteHawk I posit something is in your path that cannot process packets as soon as they get large. There is some windowing in the capture but based on the retransmissions later, the TCP Sequence never gets past the 222 of the first large packet. Try putting this in the OpenVPN configuration custom options and see what happens...

mssfix 1400;

You could start even smaller like 1300. 1400 is 50Bytes under the default of 1450.

This assumes UDP is the transport for the OpenVPN tunnel.

WhiteHawk

@Derelict said in Firewall drops packets between LAN and OPT1:

mssfix 1400;

Thanks for the suggestion, this seems to be the root cause!
An MTU of 1300 improved OpenVPN connections from my Android phone a lot, it might be flawless now. And yes, UDP is used for the tunnel.

A cloud VM's TCP connections still have hiccups. I'm yet to do further experiments.
The strange thing is that OpenVPN worked with default settings in my previous (similar but not identical) setup.