Adjust MSS calculation to account for VLANs

virtual-frog

Help us understand your set up
The VLAN2 i assume is your 192.168.20.0/24 network

It's VLAN tag 20, but yes that's the right network.

The non tagged VLAN that you have blurred out but not completely has a public IP of 162.197.52.166. So is this LAN not behind a switch?

Oops, missed the one in the detail pane
Those packet captures are on the WAN side of my firewall, so you're seeing the NAT address. My VLANs are on top of the LAN network, and it all gets NATed to one WAN connection on my firewall.

Do you have a drawing of how this is designed?

No graphic but it's a pretty standard setup I think.
ISP modem -> pfSense device -> switch -> PC
Where the firewall/switch connection is a VLAN trunk and the switch is managed with different VLAN config for each port.

Edit
I do have Snort intrusion detection running on the WAN interface but I have tried disabling it with the same results.
I stripped down as much as possible on my ISP's router so it is not running its packet filter or anything.

virtual-frog

@RobbieTT you're not listening.

The problem is that ICMP messages are arriving on the WAN side of my firewall and not making it to the VLAN side to return to my PC. So my PC just keeps sending packets not knowing someone is unhappy about the MTU.

VLANs are related somehow because I don't get these fragmentation messages without them.

RobbieTT

@virtual-frog

I have no problem knocking on your door with full-sized packets and getting the same in return:

2023-06-14 at 20.20.59.png

Did you try without the switch?

️

virtual-frog

@RobbieTT Right, and I can reach the internet just fine through the switch, on an untagged port. If the switch was a problem, wouldn't the VLAN packet capture still show the ICMP frag messages leaving the firewall? I haven't observed any issues between my PC and firewall.

If nothing else comes up I can try taking out the switch this evening.

michmoor

@virtual-frog

10.0.1.0/24 - LAN1
192.168.20.0/24 - LAN2

For devices on LAN2 network which are having issues

Dont believe this to be the case but are they configured for jumbo frames?
Move devices from LAN2 to LAN1. Are they still having the issue or does it go away?

What is the MTU set on pfSense for each interface.

virtual-frog

@michmoor

I have not configured jumbo frames on any devices, although it seems the switches are configured by default to handle 1518 (the minimum jumbo size).

In your example is LAN1 the parent interface or another VLAN? pfSense MTU and MSS are at defaults now, i.e. 1500/1460.

JKnott

@michmoor said in Adjust MSS calculation to account for VLANs:

I assume VLAN2 on the switch, what is the MTU of the switch ports for devices connected to VLAN2?

Do switches even have an MTU setting? That's a layer 3 concept. Switches have a maximum buffer size, but that's a lot larger than you'll likely find in an MTU. I have a switch with a 16KB buffer size. It will pass any frame up to that size.

JKnott

@virtual-frog said in Adjust MSS calculation to account for VLANs:

there is a jumbo frame option that is set to 1518

That's barely frame expansion, to allow things like VLAN tags. Jumbo frames are several KB, with 9000 common.

virtual-frog

@JKnott said in Adjust MSS calculation to account for VLANs:

Do switches even have an MTU setting? That's a layer 3 concept.

Well, an L3 switch might.. mine is not so it just has the jumbo frame size

virtual-frog

@JKnott said in Adjust MSS calculation to account for VLANs:

@virtual-frog said in Adjust MSS calculation to account for VLANs:

there is a jumbo frame option that is set to 1518

That's barely frame expansion, to allow things like VLAN tags. Jumbo frames are several KB, with 9000 common.

It's the same setting; I can adjust it much higher but haven't set up jumbo frame on my devices yet. The default is 1518

johnpoz

@virtual-frog said in Adjust MSS calculation to account for VLANs:

haven't set up jumbo frame on my devices yet.

no point to it either.. You think you having issues now - wait til you try and do that when you have devices that don't support jumbo or have different sizes, etc..

if you had a san network running say 10ge and you had storage devices and say server all on this san network ok then it might make sense - but there is really zero point to trying to do it on a normal gig network.. To be honest I would turn that feature off on your switch..

Not sure where you got the idea that those icmp packets saying to frag would or should go to your end device - how would it??

You are sending those.. Not ddg..

virtual-frog

@johnpoz
D'oh. My bad. But that makes even less sense. Why does WAN send the frag message for only packets originating on the VLAN?

johnpoz

@virtual-frog you have something all f'd up on that vlan would be my guess. So the mtu on the interface is 1500? What about your clients, are they wired or is wireless involved..

No idea about that tp link when it comes to vlans - they don't seem to quite get it ;) A few years back they had an issue where they would not allow you to remove vlan 1 from any ports.. So if you wanted to run say vlan 10 on that port, you were also untagged in vlan 1.. Took them like 2 years to correct it..

Why does WAN send the frag message for only packets originating on the VLAN?

Its not it sending it because what ddg is sending back.. A full packet seems to be too big for your vlan?? Even though the client sent a mss of 1460.. you can see that from the syn you sent.

virtual-frog

@johnpoz

Yes, all MTUs are the default 1500.
Mix of wired and wireless, same effect on both.
TP Link seems to have matured somewhat (still not perfect) so I don't think there's some huge flaw in the switch.
It's strange that this only happens with one site, too.
I may reset and reinstall everything.

johnpoz

@virtual-frog I am not having any issues with that site, via tagged vlans.. Than again I am not running on tplink ;)

Did you try just turning off jumbo support?

RobbieTT

@virtual-frog
Are you not going to try removing the switch from the equation by going direct to the router for fault finding purposes?

Ruling these things in or out takes seconds of self-help.

️

RobbieTT

@johnpoz said in Adjust MSS calculation to account for VLANs:

@virtual-frog you have something all f'd up on that vlan would be my guess.

I'd probably not use that phrasing but I agree that this looks to be a local network issue.

️

RobbieTT

@JKnott said in Adjust MSS calculation to account for VLANs:

Do switches even have an MTU setting? That's a layer 3 concept.

I think you just outed yourself as a fellow greybeard. For sure that was once the case and still referenced in some documentation but many (and disparate*) changes undermined that. Indeed, we were probably looking at switches with awe back then, as we nursed our hubs along.

Our distinction now is that in the context of Layer 2 switches the larger MTU is referenced with specific regard to jumbo frames and these are solely at L2.

At Layer 3 (network layer) the larger MTU is used in the context of jumbo packets. This is before we drift-off to sleep with oversized L3 MTU jumbo packets being jumbograms.

A few things don't matter and most things don't matter at all.

️

*some would argue that the word 'haphazard' should be inserted here

JKnott

@johnpoz said in Adjust MSS calculation to account for VLANs:

Did you try just turning off jumbo support?

What would that do? It only affects the size of frame the switch will pass. It has nothing to do with MTU.

JKnott

@RobbieTT said in Adjust MSS calculation to account for VLANs:

At Layer 3 (network layer) the larger MTU is used in the context of jumbo packets

I am aware of that. However, did the OP mention anything about a L3 switch? Are there any consumer level L3 switches. I have a switch that can pass 16K jumbo frames and, IIRC, there's no setting for that. It's just there and will pass any frames up to that size. Of course, with 1500 MTU, it won't see many that big.