Mobile IPSec tunnel fails on big WAN flows (MSS issue)

basicmonkey

Hello all!

I have a site to site IPSec between a pfSense VM (here) and a XG7100 (client office) which works perfectly. Mobile access to my pf VM works great and mobile users can access the Internet successfully with no issues. Mobile VPN users to the XG7100 can access the LAN ranges fine, but when initiating a decent amount of traffic to the WAN the tunnel soon falls over and stops passing traffic.

This felt like a fragmentation issue so I looked at the packet capture for a mobile client on the 7100:

138.38.32.28.8080 > 10.51.9.1.59579: Flags [S.], cksum 0xe493 (correct), seq 2255240545, ack 2201881487, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
12:34:21.971010 (authentic,confidential): SPI 0x04ed7552: (tos 0x0, ttl 50, id 0, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 8a4e (->8b4e)!)
    138.38.32.28.8080 > 10.51.9.1.59578: Flags [S.], cksum 0x937f (correct), seq 2517366989, ack 1801562485, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
12:34:22.096363 (authentic,confidential): SPI 0xc653e058: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 64)
    10.51.9.1.59577 > 216.58.205.46.443: Flags [S], cksum 0x75eb (correct), seq 2256292054, win 65535, options [mss 1240,nop,wscale 5,nop,nop,TS val 785260731 ecr 0,sackOK,eol], length 0
12:34:22.106563 (authentic,confidential): SPI 0x04ed7552: (tos 0x0, ttl 122, id 34300, offset 0, flags [none], proto TCP (6), length 60, bad cksum 123 (->223)!)
    216.58.205.46.443 > 10.51.9.1.59577: Flags [S.], cksum 0x1f46 (correct), seq 1008162808, ack 2256292055, win 65535, options [mss 1430,sackOK,TS val 2207935115 ecr 785253726,nop,wscale 8], length 0

This shows an attempt at Speedtest.net, but MSS is showing as 1460 which would clearly fragment over the IPSec. MSS clamping for IPSec traffic is set to 1392 in the advanced options so looks like this is being ignored for WAN traffic.

The strange thing is, the exact same config is on a pfSense VM at my location (MSS clamping, mobile IPSec algorithms and hashes etc) and it works absolutely fine.

pfSense on XG-7100 is 2.5.0.a.20201003.0650
pfSense on my VM is 2.5.0.a.20200922.0650

Thanks in advance!

Edit:

I've checked status.php and I can see:

table <vpn_networks> { 10.51.9.0/24
...
scrub from any to <vpn_networks> max-mss 1392
scrub from <vpn_networks> to any max-mss 1392

So all looks good there! Scrub is enabled too...

Edit 2:

I'm actually seeing much the same in a packet capture on my working pf VM but nothing falls over.

Any ideas? I'm lost.

basicmonkey

Shameless bump...

Any ideas very much welcome. It's odd that the same config works fine elsewhere. It's not the encryption engine as I can do 300Mbit between sites LAN to LAN. It's only when WAN is involved.

Thanks,

James