Fragments over IPsec

[vsc]

While my traffic in question is all UDP, the following discussion focuses on ICMP (ping). That said I have four flows, where one is 4220 byte UDP packets and the other three are <500 byte packets. The large packet arrives on the LAN port fragmented and does not transit the IPsec tunnel.

Using ping with various packet sizes shows that packets of a size equal to 1500 bytes minus headers will get through the tunnel, where the pings that are larger (e.g. 4220 bytes) are dropped. In all cases don't fragment is not set.

I have tried various options in the advanced firewall&NAT page without a positive result (e.g. IP fragment reassemble).

What am I missing?

[viragomann]

@vsc
Did you also trey "Enable MSS clamping on VPN traffic"?

[vsc]

@viragomann

I tried but that didn't work, though MSS or maximum segment size applies to TCP not UDP. My traffic is all UDP.

I tried playing with MTU on the LAN and WAN interfaces but that didn't help...

[mcury]

I tried but that didn't work, though MSS or maximum segment size applies to TCP not UDP. My traffic is all UDP.

I tried playing with MTU on the LAN and WAN interfaces but that didn't help...

Do both sides of the tunnel support VTI? If so, set the MTU on the VTI interfaces directly

[vsc]

@mcury

Hmmm... I'm using an either MacOS or IOS with Apple's mobile device manger (MDM) to create a profile. What I don't see is anything about VTI short of maybe configuring for Cisco Anyconnect like but still there I can't find anywhere to put in a MTU, unlike other clients I have worked with in the past.

Still, I'm confused. Is 99.9% of traffic REST (TCP) and people don't use UDP over a VPN?

[mcury]

@vsc said in Fragments over IPsec:

@mcury

Hmmm... I'm using an either MacOS or IOS with Apple's mobile device manger (MDM) to create a profile. What I don't see is anything about VTI short of maybe configuring for Cisco Anyconnect like but still there I can't find anywhere to put in a MTU, unlike other clients I have worked with in the past.

Still, I'm confused. Is 99.9% of traffic REST (TCP) and people don't use UDP over a VPN?

For that purpose, I would have chosen openVPN, wireguard or tailscale.
As I see it, these solutions work better with client to site.
Not saying that IPsec doesn't, but I don't see a reason to use it when you have more suitable options.

Never had a MTU problem with tailscale.
Wireguard, you can change the MTU on the client configuration with just one line.
OpenVPN, there are ways to push that and other things to the client, including via radius.

[vsc]

@mcury

I understand. I'm going to install Wireguard this morning for a quick test.

While I'm not a huge fan of IPsec I was the lead engineer for an implementation of the client and server sides, having written some of the code when I wasn't educating the team on the protocol much to the dismay at the time of Samsung Research. That was two jobs ago and there is a bunch of that code in the wild. I guess I went with something I knew and trusted.

That said pushing the larger UDP packets down an IPSec tunnel should work, where I reluctantly have to say I think pfSense is broken in this case or I'm missing a configuration option. Hopefully the later.

[mcury]

You could try IPsec VTI client to site, but you won't be able to use static routes.

You would need to set a SNAT from any, to your local network, using your IPsec VTI address (NAT Outbound in pfSense).

Reply-to enabled in the firewall rule, which is the default already.
Then, set the MTU on the IPsec VTI interface accordingly.

The downside of this is of course the NAT, and also that you would have to set IPsec Filter Mode to Filter IPsec VTI and Transport on assigned interfaces, block all tunnel mode traffic
With this setting, IPsec tunnel mode wouldn't work anymore, only VTIs.

More info at:
https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/phase-2-nat.html
https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/advanced.html

Note: It will probably work although I never tried.

If you want to keep using IPsec, I would try the method above, or just try wireguard as you mentioned.

Edit: I would try a 1440 MTU for the tunnel initially.

From a Windows machine, test with:
ping -s 1412 -M do destination_IP
Or, if you want a faster test, you could try:
for /l %i in (1400,1,1441) do @ping -n 1 -w 250 destination_IP -l %i -f

Or, test from pfSense directly:
ping -c2 -D -s 1412 destination_IP

If 1412 works, fine, it it doesn't, try 1420 MTU instead, then 1392 ICMP should pass, if not, keep decreasing it until you find the value.

[vsc]

@mcury

Configured Wireguard this morning and I have it working using an MTU of 1420, which apparently is recommended by Netgate...

Still contend the IPSec implementation is not correct. But what do I know, sigh.

[mcury]

@vsc said in Fragments over IPsec:

Still contend the IPSec implementation is not correct. But what do I know, sigh.

If you found a bug, please report it at redmine.pfsense.org
CE 2.8.0 and pfSense Plus 25.03 are both in Beta versions, if it is a bug, now it is the best time to report it.

[vsc]

@mcury

Copy. Will either write it up on the plane today or when I get to my destination.