Strange issue with IPv4 packet fragmentation
-
I've been experimenting with MTU path discovery, MSS sees etc. (only from a measurement perspective, not changing any config values anywhere).
I came across these test sites published by CLoudflare:
http://icmpcheckv6.popcount.org
If I go to the first site (IPv4) I get:
ICMP path MTU packet delivery
✓ All good! ICMP path MTU message was successfully delivered to you.In this test your browser uploads a sizeable payload to our server, while your long TCP segments are being dropped. Our server will generate an ICMP message asking you to reduce the MTU to a laughingly small value.
ICMP is critical to a well functioning Internet connections, yet networks are often misconfigured. Broken ICMP (being an ICMP blackhole) can lead to serious issues:
Slow DNS lookups.
Glitches and lag when playing games using UDP.
TCP connections failing occasionally for no apparent reason.IP fragmented packet delivery
✗ The request timed out. Looks like IP fragments failed to be delivered to you.In this test we send a large payload from the server, to your browser. Each TCP segment sent by the server will be fragmented into multiple IP packets.
A failure of this test means:
Some UDP based protocols (like games, NTP) might occasionally fail.
Certain DNS queries may time out, slowing down your internet experience.
Your ISP, firewall or CPE is not supporting basic Internet Protocol features.Whereas if I go to the second one (IPv6) I get:
ICMP path MTU packet delivery
✓ All good! ICMP path MTU message was successfully delivered to you.
...
IP fragmented packet delivery
✓ All good! IP fragments were successfully delivered to your host.
...This seems to suggest I might have some issue in regard to fragmented IPv4 packets. However I am not aware of any issues and everything seems to be working as it should (as far as I can tell!).
Is the 'error' from the IPv4 test bogus? Or...? Are there even any pfSense settings that are relevant in this regard?
All thoughts appreciated...
-
If Path MTU discovery is working then you shouldn't see fragmented traffic for most things. Almost everything. Also if your actual path MTU is a standard size you might not see any issues even for things that don't try to discover it, like UDP streams. But you should be able to pass fragmented packets.
Try running very large pings to something external like:[23.09.1-RELEASE][admin@fw1.stevew.lan]/root: ping -s 1600 1.1.1.1 PING 1.1.1.1 (1.1.1.1): 1600 data bytes 1608 bytes from 1.1.1.1: icmp_seq=0 ttl=57 time=7.185 ms 1608 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=7.021 ms 1608 bytes from 1.1.1.1: icmp_seq=2 ttl=57 time=6.790 ms ^C --- 1.1.1.1 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 6.790/6.999/7.185/0.162 ms
Since those are bigger than the link MTU (1500) they are fragmented. A packet capture shows that:
15:48:15.520278 IP 192.168.1.5 > 1.1.1.1: ICMP echo request, id 20946, seq 0, length 1480 15:48:15.520288 IP 192.168.1.5 > 1.1.1.1: ip-proto-1 15:48:15.527148 IP 1.1.1.1 > 192.168.1.5: ICMP echo reply, id 20946, seq 0, length 1480 15:48:15.527162 IP 1.1.1.1 > 192.168.1.5: ip-proto-1 15:48:16.521571 IP 192.168.1.5 > 1.1.1.1: ICMP echo request, id 20946, seq 1, length 1480 15:48:16.521576 IP 192.168.1.5 > 1.1.1.1: ip-proto-1 15:48:16.528868 IP 1.1.1.1 > 192.168.1.5: ICMP echo reply, id 20946, seq 1, length 1480 15:48:16.528881 IP 1.1.1.1 > 192.168.1.5: ip-proto-1 15:48:17.522566 IP 192.168.1.5 > 1.1.1.1: ICMP echo request, id 20946, seq 2, length 1480 15:48:17.522572 IP 192.168.1.5 > 1.1.1.1: ip-proto-1 15:48:17.529618 IP 1.1.1.1 > 192.168.1.5: ICMP echo reply, id 20946, seq 2, length 1480 15:48:17.529628 IP 1.1.1.1 > 192.168.1.5: ip-proto-1
Steve
-
Any chance you're blocking ICMP on IPv4?
Here are my ICMP rules on the WAN:
As you can see, I'm not blocking anything. Some people think blocking ICMP is a security feature, but not really and it will really kill things on IPv6. At most, you might want to block pings.
-
@JKnott I have the same rules and ICMP traffic (v4 and v6) flows freely. The more I think about it the more I think that the Cloudflare test may be somewhat flawed (which would be unusual for them).
-
I get the same for both IPv4 and 6…
ICMP path MTU packet delivery
✓ All good! ICMP path MTU message was successfully delivered to you.And
IP fragmented packet delivery
✗ The request timed out. Looks like IP fragments failed to be delivered to you. -
@Squuiid UPDATE: Since changing my ISP several weeks ago (no change to pfSense configuration) both tests now work for me. So whatever may original problem with IPv4 was, it seems it was probably something to do with my ISP's network...