UDP fragmented packet loss / IPv6 / VoIP / pfSense version inconsistency
-
One thing I see, the fragment is 1492 and the original invite is 98 bytes, which is a total of 1590. What's the MTU of the device that's sending that?
Regardless, the mandatory function on IPv6 is to send a too big message. Fragmenting along the route is prohibited. So, this fragment has to be coming from the source. Even with IPv4, a router can only fragment a packet that's passing through it. It cannot create a fragment on the input. That has to be done upstream.
-
Not sure if there is a basic misunderstanding here (which might be mine :)). The packet capture is on the WAN port so the Invite packet is arriving fragmented from SipGate. The issue then is that instead of processing the packet and sending it to the LAN side it's being dropped [so the Invite is never delivered and the phone doesn't 'ring']. I can ask SipGate tech support what the MTU is on their server and wouldn't be surprised to learn its 1500. That's perhaps why a 1590 byte Invite is being fragmented.
Why does my 2.3.5 i386 instance of pfSense process the packet? Surely the mandatory function on IPv6 is to send a too big message is precisely the point any my 2.4.2 pfSense box isn't. That's rather disappointing at the very least…
-
^^^^
That too big message is part of Path MTU Discovery, which is mandatory on IPv6 and often used on IPv4. The main reason for PMTUD is to reduce the processing load on routers and also to reduce the buffering requirements at the destination. Perhaps a chat with someone at SipGate can help. I don't know why pfSense 2.4.2 isn't passing what should be a valid packet. Incidentally, I just experimented with oversize pings to pfSense and get the same sort of fragment as shown in your capture. So, it would appear that fragment is generated at the source. According to your capture, pfSense is sending the too big message, as expected.Did you do a capture on the LAN side? Does it show anything of that Invite?
-
One thing I've notice. In the System > Advanced > System Tunables page there's a value net.inet.udp.maxdgram. On my system, it's 57344. I wonder if that or some other value is causing pfSense to not pass larger datagrams (IP supports up to 64K bytes in a datagram). This is unrelated to the MTU size.
-
JKnott - thanks for responding to me on this issue - I appreciate the help. :)
I understand the purpose of the 'too big' message but get the impression that the packets that trigger it being sent are dropped pending a re-transmission with a smaller size. (The retransmission doesn't happen - I see a lot of speculation that ICMP packets are often filtered out for security reasons).
My contention is the 'too big' message is being sent in error - the result of a failure to process the fragments correctly perhaps. I just ran a capture on the LAN port: with an outbound call. All the SIP and RTP packets are seen as expected. However the incoming Invite packets never make it to the LAN side.
Since 2.3.5 manages to process the fragmented Invite packets (arriving from SipGate on the WAN port) and do not trigger ICMPV6 too big messages, I can assume they are processed correctly (confirmed by the phone actually ringing).
I think System > Advanced > System Tunables value net.inet.udp.maxdgram has nothing to do with this problem as it's the same for both the working (2.3.5 i386) and non working (2.4.2 AMD64) version.
Is there any way to check the IP parameters at the FreeBSD level? Sometimes the bug can be between pfSense and the O/S so the parameters for BSD are being set up incorrectly. If that's not the case then this might be a bug in BSD 11.1-RELEASE-p4. Were it a glitch in the hardware I can't see how it ever worked (which it did pre 2.4.0 AMD64).
-
FWIW, I just tried an experiment. First off, I tried pinging, with both IPv4 & IPv6, with oversize packets to a computer on the local LAN. Both worked fine. I then connected one computer to another interface on my firewall and tried again. This time IPv4 worked, but IPv6 didn't. I can see the fragmented IPv6 pings leaving one computer, but not arriving on the other. So, it appears pfSense/FreeBSD is not passing packets that have been fragmented by the source, but does pass IPv4.
Looks like it's time to file a bug report, as it should be passing both IPv4 and IPv6 fragments.
-
JKnott - thanks for confirming the problem. Are you going to file a bug report or do you expect me to? I'm happy not to…. ;D
-
I'll let you have the honours. :)
-
#8165
:)
-
Looks like https://redmine.pfsense.org/issues/8165 is closed to early. We still see problems with IPv6 fragments, in our case with local created ones which simply disappear. Depending on certificates and keysizes used Strongswan will use "oversized" UDP packets in the IKEv2 connection etsablishment. If the remote side does not support IKEv2 Fragmentation (Windows older than Version 10 /1803) the packet is never leaving the pfsense box if IPv6 is used. A Capture done at the WAN Interface show that this packet is simply missing and therefore the handshake never completes. This is still the case on latest 2.4.4-RELEASE-p1.