Adjust MSS calculation to account for VLANs

[michmoor]

@virtual-frog said in Adjust MSS calculation to account for VLANs:

The problem is 1) why do I only get fragmentation messages on VLAN? 2) why are they stopped by the firewall?

Reply

I assume VLAN2 on the switch, what is the MTU of the switch ports for devices connected to VLAN2?

EDIT:
Help us understand your set up
The VLAN2 i assume is your 192.168.20.0/24 network
The non tagged VLAN that you have blurred out but not completely has a public IP of 162.197.52.166. So is this LAN not behind a switch?
Do you have a drawing of how this is designed?

[virtual-frog]

@michmoor
I'm on VLAN 20. Not sure about MTU settings for this switch (using TP Link Omada SDN so it's a little weird) - there is a jumbo frame option that is set to 1518 so I assume it can at least handle the standard 1500.

I can ping both google and my firewall at 1472. If I turn off the MTU checking flag I can ping my firewall at much larger packet sizes, they will just get fragmented. Note that Google is one of the sites that I don't have issues with.

The connection between my PC and firewall works fine, no fragmentation issues or dropped packets.

[RobbieTT]

@virtual-frog said in Adjust MSS calculation to account for VLANs:

2. why are they stopped by the firewall?

As described earlier, VLANs are stripped away by the router. They do not leave your network. This is not a firewall issue. You need to look internal.

A successful exchange with duckduckgo:

Zero issues with MSS or MTU with full 1500 (1514) byte packets being exchanged with DDG.

️

[virtual-frog]

@michmoor

Help us understand your set up
The VLAN2 i assume is your 192.168.20.0/24 network

It's VLAN tag 20, but yes that's the right network.

The non tagged VLAN that you have blurred out but not completely has a public IP of 162.197.52.166. So is this LAN not behind a switch?

Oops, missed the one in the detail pane
Those packet captures are on the WAN side of my firewall, so you're seeing the NAT address. My VLANs are on top of the LAN network, and it all gets NATed to one WAN connection on my firewall.

Do you have a drawing of how this is designed?

No graphic but it's a pretty standard setup I think.
ISP modem -> pfSense device -> switch -> PC
Where the firewall/switch connection is a VLAN trunk and the switch is managed with different VLAN config for each port.

Edit
I do have Snort intrusion detection running on the WAN interface but I have tried disabling it with the same results.
I stripped down as much as possible on my ISP's router so it is not running its packet filter or anything.

[virtual-frog]

@RobbieTT you're not listening.

The problem is that ICMP messages are arriving on the WAN side of my firewall and not making it to the VLAN side to return to my PC. So my PC just keeps sending packets not knowing someone is unhappy about the MTU.

VLANs are related somehow because I don't get these fragmentation messages without them.

[RobbieTT]

@virtual-frog

I have no problem knocking on your door with full-sized packets and getting the same in return:

Did you try without the switch?

️

[virtual-frog]

@RobbieTT Right, and I can reach the internet just fine through the switch, on an untagged port. If the switch was a problem, wouldn't the VLAN packet capture still show the ICMP frag messages leaving the firewall? I haven't observed any issues between my PC and firewall.

If nothing else comes up I can try taking out the switch this evening.

[michmoor]

@virtual-frog

10.0.1.0/24 - LAN1
192.168.20.0/24 - LAN2

For devices on LAN2 network which are having issues

1. Dont believe this to be the case but are they configured for jumbo frames?
2. Move devices from LAN2 to LAN1. Are they still having the issue or does it go away?

What is the MTU set on pfSense for each interface.

[virtual-frog]

@michmoor

I have not configured jumbo frames on any devices, although it seems the switches are configured by default to handle 1518 (the minimum jumbo size).

In your example is LAN1 the parent interface or another VLAN? pfSense MTU and MSS are at defaults now, i.e. 1500/1460.

[JKnott]

@michmoor said in Adjust MSS calculation to account for VLANs:

I assume VLAN2 on the switch, what is the MTU of the switch ports for devices connected to VLAN2?

Do switches even have an MTU setting? That's a layer 3 concept. Switches have a maximum buffer size, but that's a lot larger than you'll likely find in an MTU. I have a switch with a 16KB buffer size. It will pass any frame up to that size.

[JKnott]

@virtual-frog said in Adjust MSS calculation to account for VLANs:

there is a jumbo frame option that is set to 1518

That's barely frame expansion, to allow things like VLAN tags. Jumbo frames are several KB, with 9000 common.

[virtual-frog]

@JKnott said in Adjust MSS calculation to account for VLANs:

Do switches even have an MTU setting? That's a layer 3 concept.

Well, an L3 switch might.. mine is not so it just has the jumbo frame size

[virtual-frog]

@JKnott said in Adjust MSS calculation to account for VLANs:

@virtual-frog said in Adjust MSS calculation to account for VLANs:

there is a jumbo frame option that is set to 1518

That's barely frame expansion, to allow things like VLAN tags. Jumbo frames are several KB, with 9000 common.

It's the same setting; I can adjust it much higher but haven't set up jumbo frame on my devices yet. The default is 1518

[johnpoz]

@virtual-frog said in Adjust MSS calculation to account for VLANs:

haven't set up jumbo frame on my devices yet.

no point to it either.. You think you having issues now - wait til you try and do that when you have devices that don't support jumbo or have different sizes, etc..

if you had a san network running say 10ge and you had storage devices and say server all on this san network ok then it might make sense - but there is really zero point to trying to do it on a normal gig network.. To be honest I would turn that feature off on your switch..

Not sure where you got the idea that those icmp packets saying to frag would or should go to your end device - how would it??

You are sending those.. Not ddg..

[virtual-frog]

@johnpoz
D'oh. My bad. But that makes even less sense. Why does WAN send the frag message for only packets originating on the VLAN?

[johnpoz]

@virtual-frog you have something all f'd up on that vlan would be my guess. So the mtu on the interface is 1500? What about your clients, are they wired or is wireless involved..

No idea about that tp link when it comes to vlans - they don't seem to quite get it ;) A few years back they had an issue where they would not allow you to remove vlan 1 from any ports.. So if you wanted to run say vlan 10 on that port, you were also untagged in vlan 1.. Took them like 2 years to correct it..

Why does WAN send the frag message for only packets originating on the VLAN?

Its not it sending it because what ddg is sending back.. A full packet seems to be too big for your vlan?? Even though the client sent a mss of 1460.. you can see that from the syn you sent.

[virtual-frog]

@johnpoz

Yes, all MTUs are the default 1500.
Mix of wired and wireless, same effect on both.
TP Link seems to have matured somewhat (still not perfect) so I don't think there's some huge flaw in the switch.
It's strange that this only happens with one site, too.
I may reset and reinstall everything.

[johnpoz]

@virtual-frog I am not having any issues with that site, via tagged vlans.. Than again I am not running on tplink ;)

Did you try just turning off jumbo support?

[RobbieTT]

@virtual-frog
Are you not going to try removing the switch from the equation by going direct to the router for fault finding purposes?

Ruling these things in or out takes seconds of self-help.

️

[RobbieTT]

@johnpoz said in Adjust MSS calculation to account for VLANs:

@virtual-frog you have something all f'd up on that vlan would be my guess.

I'd probably not use that phrasing but I agree that this looks to be a local network issue.

️