IPv6 PPPoE MSS incorrect

viktor_g

Please create a bugreport:
https://docs.netgate.com/pfsense/en/latest/development/bug-reports.html

JKnott

@viktor_g said in IPv6 PPPoE MSS incorrect:

2.5.0-RC sets MTU of the interface and doesn't change the MSS value:

My understanding is the MTU sets the maximum size for packets created by pfsense, whereas MSS limits the size of packets passing through it. If the previous MTU is too big to pass through pfsense, either the packet will be fragmented or a too big ICMP message will be sent back to the source. The MSS will tell the source to not send such a big packet.

viktor_g

@msmith100 have you configured MSS manually on the interfaces page?

viktor_g

@bm118 said in IPv6 PPPoE MSS incorrect:

Looking at the logs and traces I have, it appears that PFsense is incorrectly applying the IPv4 calculations to IPv6 packets; 1500 MTU - 8 PPPoE - 20 TCP - 20 IPv4 = 1452 MSS. Therefore IPv6 packets have the IPv4 40-byte subtraction instead of the required IPv6 60-byte subtraction when calculating MSS and will, therefore, use the 1440 MSS as offered by the client as it is lower.

seems to be true:
https://github.com/pfsense/pfsense/blob/11fd7da72502c991b1f1c0e886ea212235f4a505/src/etc/inc/filter.inc#L640

msmith100

@viktor_g Yes I have set MSS manually, to 1472.

A screenshot of my settings is attached. I'll also create a bug report shortly.

viktor_g

@msmith100 The PPPoE interfaces already have a TCP MSS Fix option: https://docs.netgate.com/pfsense/en/latest/interfaces/ppp.html#advanced-ppp-options
Could you try to remove MSS settings from the interface and check again?

msmith100

@viktor_g I tried removing the MSS setting, and leaving TCP MSS fix on - i.e. not checking the disable box for it in the GUI. I can confirm that my issues accessing websites such as yahoo.com re-appear.

If I disable IPv6 on my client computer, accessing the same websites works just fine.

msmith100

@viktor_g @BM118 Bug report created as requested: https://redmine.pfsense.org/issues/11409

BM118

@viktor_g this is 100% the problem. It’s applying 40 bytes no matter what from my quick check. If that value was updated to 60 bytes it would alleviate needing to set the MSS to 20bytes lower than MTU.

However this is a workaround that would prevent people having to apply their own workaround, and the ideal fix would be to have different MSS applied based on IPv4 or IPv6. Mind you, I don’t know how how other vendors do it, or if they simply apply the standard 40 byte, and then change that to 60byte the moment IPv6 is enabled on the interface which achieves the same as the above.

For others reading this, as a general rule, don’t ever disable TCP MSS adjust settings, in this age of the internet you will have problems somewhere at some point, even if you don’t see it straight away. Also keep in mind that adjusting MSS only affects TCP and not UDP, which is even more reliant on a correct MTU than TCP is.

BM118

@msmith100 said in IPv6 PPPoE MSS incorrect:

@viktor_g Thanks for looking into this.
As I understand it, by default pfsense does not block any essential IPv6-related ICMP.

Regarding point 1, in a sense yes it does. IPv6 requires certain ICMP messages to be sent back to the source of a connection, which is an inbound new connection from a firewalls perspective. Specifically “Packet Too Big” messages amongst a couple of others that are critical to IPv6 working properly. Firewalls deliberately will block these packets unless you allow them inbound on your Outside/WAN interface.

IPv6 does not allow packets to be fragmented by intermediate routers like IPv4 does, and therefore the source will never know that the packet needs to be smaller to get through hop C when going from A to D if you don’t allow that packet inbound.
In IPv4 land, router C could have fragmented it and the packet would get there but this causes scalability and efficiency issues in the internet in general.

When using IPv6-test.com, it has a test to see if your are accepting ICMP messages or not, and even has a screen where it sends a constant ping against your IPv6 and IPv4 address which is very useful to see if ICMPv6 messages are making it all the way to your system.

Hint: if they aren’t succeeding, check your local system firewall as well as the PFSense rules.

msmith100

@bm118 Looking through the raw rules in pfsense, I see this:

pass quick inet6 proto ipv6-icmp all icmp6-type unreach keep state
pass quick inet6 proto ipv6-icmp all icmp6-type toobig keep state
pass quick inet6 proto ipv6-icmp all icmp6-type neighbrsol keep state
pass quick inet6 proto ipv6-icmp all icmp6-type neighbradv keep state

These are not anything I created, and I believe part of the default pfsense rules. If I understand them correctly, they mean that by default pfsense will pass along "packet too big" messages without issue. I also allow pings with one of my own rules, and I pass the IPv6-test.com tests no problem. Doesn't help me on other sites though, as I think some sites like yahoo.com don't do PMTUD properly. Even for sites that do, the delay while it figures out the proper size is very noticeable and undesirable.

viktor_g

@bm118

Could you test this patch: 135.diff

You need to install System Patches pkg:
https://docs.netgate.com/pfsense/en/latest/development/system-patches.html

msmith100

@viktor_g Applied on 2.4.5 without issue and confirmed working on all my problem websites with 1492 entered as MSS into WAN configuration page.

[2.4.5-RELEASE][root@pfSense]/root: pfctl -sr | grep mss
scrub on pppoe0 inet all max-mss 1452 fragment reassemble
scrub on pppoe0 inet6 all max-mss 1432 fragment reassemble

Thank you very much!

EDIT: May also want to update the description on the interfaces page:
If a value is entered in this field, then MSS clamping for TCP connections to the value entered above minus 40 (TCP/IP header size) will be in effect.

dem

Perhaps slightly off topic, but my IPv6 is via 6rd over PPPoE and when I was having PMTUD issues what I did was patch the radvd config to advertise a lower MTU.

viktor_g

@dem please try the updated patch with 6rd/6to4 support: 135 (1).diff

dem

@viktor_g Thanks, but I'm unable test with my production system right now.

BM118

@msmith100 said in IPv6 PPPoE MSS incorrect:

@bm118 Looking through the raw rules in pfsense, I see this:
pass quick inet6 proto ipv6-icmp all icmp6-type unreach keep state
pass quick inet6 proto ipv6-icmp all icmp6-type toobig keep state
pass quick inet6 proto ipv6-icmp all icmp6-type neighbrsol keep state
pass quick inet6 proto ipv6-icmp all icmp6-type neighbradv keep state
These are not anything I created, and I believe part of the default pfsense rules. If I understand them correctly, they mean that by default pfsense will pass along "packet too big" messages without issue. I also allow pings with one of my own rules, and I pass the IPv6-test.com tests no problem. Doesn't help me on other sites though, as I think some sites like yahoo.com don't do PMTUD properly. Even for sites that do, the delay while it figures out the proper size is very noticeable and undesirable.

Thanks for this, I actually wasn’t aware that PFSense had any rules for ICMPv6 built-in. I wouldn’t expect to see NeighbourSolicit and NeighbourAdvertise through a WAN interface though, however to the interface may be another thing. I believe there are some types missing from that list that are commonly accepted to be essential.

I would recommend having a look through this article if you are so inclined as it details a bit about the types of messages that must be allowed, generally should be allowed and ones that don’t matter.

https://tools.ietf.org/html/rfc4890#page-14

JKnott

@bm118 said in IPv6 PPPoE MSS incorrect:

I wouldn’t expect to see NeighbourSolicit and NeighbourAdvertise through a WAN interface though

I certainly see them here. IPv6 can't work without them.

viktor_g

@bm118 see https://redmine.pfsense.org/issues/9167

BM118

@jknott said in IPv6 PPPoE MSS incorrect:

@bm118 said in IPv6 PPPoE MSS incorrect:

I wouldn’t expect to see NeighbourSolicit and NeighbourAdvertise through a WAN interface though

I certainly see them here. IPv6 can't work without them.

I might be going on a tangent here, and while I 100% agree you need NS and NA for IPv6 to work, I still don't believe that the packets need to be allowed to transit through the WAN interface or any L3 interface for that matter. Should the packets be allowed to the interface of the Firewall, and to the Multicast address block, sure, but I don't see why they need to be allowed through them.

In the grand scheme of things, a routed NS/NA will likely be dropped anyway.

NS and NA messages are purely for local link management, and as such if you have an L2/Bridge setup then it makes sense to allow them through the interface.