Changing AdvLinkMTU when using NPt

Napsterbater

For various reasons I've opted to use NPt between my LAN and WAN. Because of this the MTU set in radvd.conf is wrong as it seems to be following the LAN side MTU when not using interface tracking.
This becomes an issue when using, for example, a GIF tunnel to HE as MTU has to be lowered to 1280.

Ummm NPt has nothing to do with MTU, it is not the reason the "MTU is wrong" (its not). An Ethernet LAN has a MTU of 1500, thus will be advertised as such. It is NOT suposed to advertise that WAN MTU to the LAN.

Nothing is wrong with it advertising 1500 MTU, that is as designed/intended.

JKnott

An Ethernet LAN has a MTU of 1500, thus will be advertised as such.

Actually, I was experimenting with 9K byte jumbo frames the other day. Even had the pfSense DHCP server configured with option 26 to do that. The only thing that has a limit of 1500 bytes is 802.3 Ethernet. IP uses Ethernet II, which does not have a size limit, though older hardware might be limited to 1500. Gigabit gear generally supports jumbo frames.

Napsterbater

@JKnott:

An Ethernet LAN has a MTU of 1500, thus will be advertised as such.

Actually, I was experimenting with 9K byte jumbo frames the other day. Even had the pfSense DHCP server configured with option 26 to do that. The only thing that has a limit of 1500 bytes is 802.3 Ethernet. IP uses Ethernet II, which does not have a size limit, though older hardware might be limited to 1500. Gigabit gear generally supports jumbo frames.

godd point.. I should have mentioned that in general an MTU of 1500 is used/standard. But you are right, its not the max.

But my point of it not being related to NPt, no more having to be the same as or even related to the WAN MTU stands.

FuN_KeY

I am using 6RD for ipv6 tunneling, and I am also required to lower the AdvLinkMTU to 1280 in order to get ipv6 working.

I would also greatly benefit from a proper way to define this value (either in the webGUI or some config file). I do not see how auto MTU detection could be of any help in this scenario, my clients blindly follow what is advertised by radvd.

Napsterbater

@FuN_KeY:

I am using 6RD for ipv6 tunneling, and I am also required to lower the AdvLinkMTU to 1280 in order to get ipv6 working.

I would also greatly benefit from a proper way to define this value (either in the webGUI or some config file). I do not see how auto MTU detection could be of any help in this scenario, my clients blindly follow what is advertised by radvd.

There is no reason to set LAN MTU less then 1500, even if your WAN has a different/smaller MTU.

And everything I see says pfSense uses/forces 1280 MTU for 6rd WAN, which means when pfSense gets a 1500 packet, it will send a ICMPv6 Type 2 "Packet to Big" to the client, the client should then reduce its MTU for that connection to 1280 on its own, that is how IPv6 was designed to work and how it does work, until people start blocking ICMPv6 without knowing what they are doing.

Which means you are probably either breaking/blocking ICMPv6/PMTUD, or you have broken clients.

JKnott

And everything I see says pfSense uses/forces 1280 MTU for 6rd WAN, which means when pfSense gets a 1500 packet, it will send a ICMPv6 Type 2 "Packet to Big" to the client, the client should then reduce its MTU for that connection to 1280 on its own, that is how IPv6 was designed to work and how it does work, until people start blocking ICMPv6 without knowing what they are doing.

Also, when the connection fails, even though the MTU was properly negotiated, then it's assumed someone is blocking ICMP and so TCP will adjust the size automagically, so that the connection will work.

FuN_KeY

First of all, I did not hardcode any MTU on any interface

I did some more investigation, and on the pfsense level, the MTU negotiation is fine:

wan_stf: flags=4041 <up,running,link2>metric 0 mtu 1280
inet6 2a02:xxxx:xxx:xxx:: prefixlen 32
nd6 options=1 <performnud>v4net xxx.x.xx.xx/32 -> tv4br xxx.xxx.xxx.xxx
groups: stf

The problem is that radvd is either taking 1500 or the MTU of the LAN interface (which end up being 1500 in my case). I did some wireshark and saw no packet too bit.

How is it meant to be working? Should pfsense advertise the correct MTU? Should the client be able adjust on packet too big? Who would issue the packet too big (pfsense, 6RD GW, …) ?</performnud></up,running,link2>

JKnott

PfSense should be advertising the MTU of the local link only, not any other interface. So, even if your tunnel is only 1280, the local link is 1500. IPv6 will then use Path MTU Discovery to set the MTU for any traffic passing through that tunnel. So, if you look at packets going through the tunnel, you should see an MTU of 1280. You can capture the packets with Packet Capture, but will have to export to Wireshark to see the MTU. If you capture everything, you can see PMTUD in action.

Napsterbater

@FuN_KeY:

First of all, I did not hardcode any MTU on any interface

I didnt say you did, I said pfSense does on 6RD interfaces.

@FuN_KeY:

The problem is that radvd is either taking 1500 or the MTU of the LAN interface (which end up being 1500 in my case). I did some wireshark and saw no packet too bit.

That is not a problem, that is exactly what it is supposed to do.

@FuN_KeY:

How is it meant to be working? Should pfsense advertise the correct MTU? Should the client be able adjust on packet too big? Who would issue the packet too big (pfsense, 6RD GW, …) ?

pfSense is advertising the correct MTU for the Ethernet LAN, 1500, again that is what it is supposed to do.

When pfSense receives a packet that need to be forwarded though the 6RD interface with an MTU of 1280 that is larger then that it will send a ICMNPv6 Type 2 (Packet to big) message with the MTU that should be used, the client will then resend its packets to that destination with the new MTU. If you block All ICMP or those messages the connection will fail if any packets are over the MTU of the link.

Also really 6rd does not have an automatic MTU of 1280, pfSense just sets it that way for some reason. 6rd MTU can be upto:
(Your WAN MTU) - 20 = (6RD MTU)

Also NPt has NOTHING to do with MTU at all.

JKnott

^^^^
Where's this 6rd coming from? I thought the OP was talking about he.net, which uses 6in4 over a configured tunnel. 6rd is a method used by some ISPs to provide IPv6, using the ISPs IPv4 addresses. While both methods use a tunnel for IPv6, the set up is quite different.

https://en.wikipedia.org/wiki/IPv6_rapid_deployment

Napsterbater

@JKnott:

^^^^
Where's this 6rd coming from? I thought the OP was talking about he.net, which uses 6in4 over a configured tunnel. 6rd is a method used by some ISPs to provide IPv6, using the ISPs IPv4 addresses. While both methods use a tunnel for IPv6, the set up is quite different.

https://en.wikipedia.org/wiki/IPv6_rapid_deployment

Op was using a GIF INterface and I assume he.net tunnel.

FuN_KeY is using 6rd.

JKnott

AC3200 is acting as my main gateway, and I want to use it as DHCP server for local and VPN clients.

If they were using 6rd, there'd be no need for he.net. Either method creates an IPv6 tunnel, but you wouldn't use both. So, it's either 6rd or he.net. Take your pick.

FuN_KeY

I was able to capture the packet too big on wireshark. Everything looks good, except for my Windows 10 client that appear to ignore this value.

JKnott

@FuN_KeY:

I was able to capture the packet too big on wireshark. Everything looks good, except for my Windows 10 client that appear to ignore this value.

So, it continues to send 1500 byte packets, despite the too big message? I certainly never had a problem running Windows on IPv6, back when I used a tunnel.

Napsterbater

@JKnott:

@FuN_KeY:

I was able to capture the packet too big on wireshark. Everything looks good, except for my Windows 10 client that appear to ignore this value.

So, it continues to send 1500 byte packets, despite the too big message? I certainly never had a problem running Windows on IPv6, back when I used a tunnel.

Agreed,, I have never had an issues with PMTUD on Winows since XP..

Napsterbater

@FuN_KeY:

I was able to capture the packet too big on wireshark. Everything looks good, except for my Windows 10 client that appear to ignore this value.

Any 3rd party firewall?/security software? Have you made any canges to the windows firewall.

FuN_KeY

Nope, vanilla windows 10 (tested on the host and in a VM with a fresh windows install)

I did attach 2 captures. In the first one, one can see the packet too big. And in the second you can see some errors beyond my basic understanding of wireshark.

I did filter the capture over traffic towards a web site (www.swisscom.ch) + icmpv6. Sadly the website I am having problem with uses SSL, so the capture is not that clear.

If I edit the services.inc to let radvd advertise a MTU of 1280 (or even 1480 - despite the 6RD being configured to use 1280) everything works fine.

![wireshark 1.PNG](/public/imported_attachments/1/wireshark 1.PNG)
![wireshark 1.PNG_thumb](/public/imported_attachments/1/wireshark 1.PNG_thumb)
![wireshark 2.PNG](/public/imported_attachments/1/wireshark 2.PNG)
![wireshark 2.PNG_thumb](/public/imported_attachments/1/wireshark 2.PNG_thumb)

JKnott

I haven't seen those errors before either, however it appears something might be corrupting the Ethernet frames. There's the malformed packet error, which means there was a problem somewhere causing bit errors in the frame. That might also be the cause of the segment errors. There's not enough info shown to know where the problem is coming from. Do other computers have the same problem? If only one has the problem, I'd suspect something like a defective NIC. The 1480 MTU shows PMTUD is working. What other equipment is there between the Windows computer and pfSense? Again those malformed packet, frame check sequence incorrect errors make me suspect hardware.

Napsterbater

I just thought of something to. Is that the only site you have an issue with when you let it advertised a 1500 MTU. Because I noticed something from that site when I ran a certain test to it. I'll link and show it in a minute when I get a chance

JKnott

@Napsterbater:

I just thought of something to. Is that the only site you have an issue with when you let it advertised a 1500 MTU. Because I noticed something from that site when I ran a certain test to it. I'll link and show it in a minute when I get a chance

The site shouldn't cause Ethernet frame errors, as he appears to be getting.