Local created oversized IPv6 UDP packets get dropped by pfsense

JKnott

@sigi said in Local created oversized IPv6 UDP packets get dropped by pfsense:

@lst_hoe said in Local created oversized IPv6 UDP packets get dropped by pfsense:

Such packets are than dropped and never appear in a WAN port capture. To my knowledge pfsense should create IPv6 UDP fragments as it is the source of the packets?

No, pfsense should send "Packet Size Too Big" and the client should fragment to appropriate size. No device in is allowed to fragment in transit. This is the big difference between IPv4 and IPv6. (AFAIK)

I believe he said pfSense was the source. If so, it should create fragments as necessary. However, if it's just forwarding packets from elsewhere, then it must not fragment. Also, that difference between IPv4 and IPv6 is disappearing with the use of the IPv4 do not fragment flag. Linux uses it on all IPv4 traffic. Windows does on TCP, but apparently not UDP or ICMP.

johnpoz

So if pfsense is the source and its not fragmenting - what is it doing... Putting data larger than the MTU on the wire? Is it not adding the correct frag header?

How exactly did you test that pfsense is not doing what its suppose to be doing?

sigi

@jknott said in Local created oversized IPv6 UDP packets get dropped by pfsense:

I believe he said pfSense was the source.

Not sure about this. Because for me in a VPN always the client is initiating the connection. Therefore the VPN Concentrator should never answer with packets/fragments exceeding size of the existing back-channel. But if MS XP is taking part, there can anything happen ;-)

JKnott

@sigi said in Local created oversized IPv6 UDP packets get dropped by pfsense:

Not sure about this. Because for me in a VPN always the client is initiating the connection.

There are 2 packets here, the one coming from the source and the one created by OpenVPN, which contains the source data. Which one are we talking about? This is where Wireshark or Packet Capture come in handy. Until we know which, it's hard to tell where the problem is.

lst_hoe

The use case is IPSEC VPN with pfsense (Strongswan) as endpoint and Windows clients. The problem is at the key exchange with IKEv2 which is UDP by design. The client initiate the connection and the endpoint need to authenticate itself. This is done in our case with certificates (EAP-TLS). With this the response the pfsense is supposed to send is around 2000 Bytes which will be send as one UDP datagram if the clients/server does not support IKEv2 Fragmentation which is only available with Windows 10 1803 and later. So Strongswan create a UDP datagram with more bytes than the MTU is set and therefore it needs to be put on the wire as IPv6 UDP Fragments. This does not happen, a dump on the WAN interface, where Strongswan is bind to shows that this special packets are simply missing. The IPv4 case does work by the way, in this case two UDP fragmnets go out to the client and the key exchange is done.

sigi

According to my experience with BIND i would experiment with following SSW settings:
charon.plugins.kernel-netlink.mss 0 MSS to set on installed routes, 0 to disable.
charon.plugins.kernel-netlink.mtu 0 MTU to set on installed routes, 0 to disable.

sigi

@jknott said in Local created oversized IPv6 UDP packets get dropped by pfsense:

There are 2 packets here, the one coming from the source and the one created by OpenVPN

Agree. But in my opinion all packets from OpenVPN are answers inside the session initated by the client. There is normally no session initiated from the server side. MTU/MSS happens per session (AFAIK)

lst_hoe

To my knowledge MSS is TCP only, not UDP. The MTU will not help if you can't set it higher than the biggest datagram which must be send in one piece. For the IKE handshake the application must send the authentication in one IP packet if IKE fragmentation at application layer is not supported on both ends. So if you are using EAP-TLS with more secure keysizes you get UDP fragments on the wire. As said it is working as expected with IPv4, only with IPv6 pfsense does not send UDP fragments over the wire.

lst_hoe

@johnpoz said in Local created oversized IPv6 UDP packets get dropped by pfsense:

So if pfsense is the source and its not fragmenting - what is it doing... Putting data larger than the MTU on the wire? Is it not adding the correct frag header?

How exactly did you test that pfsense is not doing what its suppose to be doing?

According to the capture done at the WAN interface the packet simply does not happen. The only trace is in the VPN/Strongswan log and the knowledge that the packet must exist to finish the IKE key exchange and as said the expected behaviour and capture in case of IPv4.

sigi

netstat -s -p ip6 give some insights to fragment handling.. Anything strange there?

lst_hoe

It looks like FreeBSD is able to create IPv6 fragments (45 fragments created), but i have no idea where they are going to in case of the WAN interface.

netstat -s -p ip6
ip6:
402474348 total packets received
0 with size smaller than minimum
0 with data size < data length
0 with bad options
0 with incorrect version number
0 fragments received
0 fragments dropped (dup or out of space)
0 fragments dropped after timeout
0 fragments that exceeded limit
0 packets reassembled ok
614833 packets for this host
401766863 packets forwarded
2 packets not forwardable
45 redirects sent
746064 packets sent from this host
0 packets sent with fabricated ip header
0 output packets dropped due to no bufs, etc.
0 output packets discarded due to no route
0 output datagrams fragmented
45 fragments created
0 datagrams that can't be fragmented
0 packets that violated scope rules
0 multicast packets which we don't join