IPSEC NAT-T, MTU and Fragmentation: ping size 1394 works, 1395 doesn't
-
I have three pfsense systems.
A: is configured to set up an IPSEC tunnel to C with NAT-T enabled (172.16.0.10)
B: is configured as a home ADSL router with NAT (1.1.1.1) (we also tried another make of router)
C: is the corporate firewall (2.2.2.2)
We cannot ping with a packet size of 1395 bytes. We can with 1394. The packets reach the intermediate router B, but when the packet is too big it just doesn't reach C.
Obviously it's a fragmentation issue. But we have turned off scrubbing on all three devices. We also tried a different ISP, just in case.
A has an MTU of 1492 to match ADSL so as to prevent B from having to chop the data up into different sized chunks.
Ping size 1394
Coming into pfsense B from A
20:08:34.920032 IP 172.16.0.10.4500 > 2.2.2.2.4500: UDP-encap: ESP(spi=0x0ee930d1,seq=0x23e), length 1460
20:08:34.931861 IP 2.2.2.2.4500 > 172.16.0.10.4500: UDP-encap: ESP(spi=0x0150c13a,seq=0x8d), length 1460
20:08:35.930249 IP 172.16.0.10.4500 > 2.2.2.2.4500: UDP-encap: ESP(spi=0x0ee930d1,seq=0x23f), length 1460
20:08:35.941808 IP 2.2.2.2.4500 > 172.16.0.10.4500: UDP-encap: ESP(spi=0x0150c13a,seq=0x8e), length 1460Sent down pppoe
20:08:34.920125 IP 1.1.1.1.25048 > 2.2.2.2.4500: UDP-encap: ESP(spi=0x0ee930d1,seq=0x23e), length 1460
20:08:34.931805 IP 2.2.2.2.4500 > 1.1.1.1.25048: UDP-encap: ESP(spi=0x0150c13a,seq=0x8d), length 1460
20:08:35.930357 IP 1.1.1.1.25048 > 2.2.2.2.4500: UDP-encap: ESP(spi=0x0ee930d1,seq=0x23f), length 1460
20:08:35.941746 IP 2.2.2.2.4500 > 1.1.1.1.25048: UDP-encap: ESP(spi=0x0150c13a,seq=0x8e), length 1460Arriving at other end
20:08:34.899085 IP 1.1.1.1.25048 > 2.2.2.2.4500: UDP-encap: ESP(spi=0x0ee930d1,seq=0x23e), length 1460
20:08:34.899740 IP 2.2.2.2.4500 > 1.1.1.1.25048: UDP-encap: ESP(spi=0x0150c13a,seq=0x8d), length 1460
20:08:35.909094 IP 1.1.1.1.25048 > 2.2.2.2.4500: UDP-encap: ESP(spi=0x0ee930d1,seq=0x23f), length 1460
20:08:35.909711 IP 2.2.2.2.4500 > 1.1.1.1.25048: UDP-encap: ESP(spi=0x0150c13a,seq=0x8e), length 1460Ping size 1395
Coming into pfsense (tcpdump -n -i vr3)
20:14:32.541040 IP 172.16.0.10.4500 > 2.2.2.2.4500: UDP-encap: ESP(spi=0x0ee930d1,seq=0x3c2), length 1464
20:14:32.541285 IP 172.16.0.10 > 2.2.2.2: ip-proto-17
20:14:33.551225 IP 172.16.0.10.4500 > 2.2.2.2.4500: UDP-encap: ESP(spi=0x0ee930d1,seq=0x3c3), length 1464
20:14:33.551417 IP 172.16.0.10 > 2.2.2.2: ip-proto-17Sent down pppoe (tcpdump -n -i pppoe1)
20:14:32.541222 IP 172.16.0.10.4500 > 2.2.2.2.4500: UDP-encap: ESP(spi=0x0ee930d1,seq=0x3c2), length 1464
20:14:32.541339 IP 172.16.0.10 > 2.2.2.2: ip-proto-17
20:14:33.551360 IP 172.16.0.10.4500 > 2.2.2.2.4500: UDP-encap: ESP(spi=0x0ee930d1,seq=0x3c3), length 1464
20:14:33.551470 IP 172.16.0.10 > 2.2.2.2: ip-proto-17Does not arrive at another end
-
From the looks of that, whatever is doing the NAT is dropping fragmented traffic. Where you see it leaving one end, and not showing up at the other end, something in between is stopping it from passing.