Odd MTU / fragmented packet issue on web GUI and haproxy
-
@johnpoz thank you for that explanation. I have another question. Since my physical interface and vlan interface in pfsense are set to MTU 1500 (default) how are the packets allowed to be 1522 in size? Does the MTU size not count FSC, checksum, etc?
-
@shewless the mtu is the data size.. there is also the headers.. 42-1500
You will notice not all of your frames shown in wireshark show 1518, some will be smaller. And sometimes they will show larger even still when your doing windows scaling..
But that not exactly what went on the wire - that is just wireshark putting it together for you, etc.
As to cosmetic issues with switches - the tplink ones that suppose to do vlans, but when you did to vlans their counters showed all the tagged packets as "errors" ;)
-
@johnpoz thx. Consider the matter resolved!
-
@shewless you mind sharing the specific make and model of the switch what firmware on it, etc. I'm just curious if what your seeing is common info, fixed with a firmware/patch, etc.
-
@johnpoz absolutely. It's a S3900-24T4S-R (which seems to operate differently than the S3900-24T4S).
I could not find any newer firmware available for this switch nor could I find much documentation regarding this specific issue.
From what I have read different switches consider MTU differently (Ethernet framesize vs IP MTU, etc)
Fiberstore Co., Limited Internetwork Operating System Software S3900-24T4S-R Series Software, Version 2.2.0E Build 88393, RELEASE SOFTWARE Copyright (c) 2021 by FS.COM All Rights ReservedI'd be happy to submit a bug report for this issue if I could understand it more :)
-
@shewless do you happen to have an example of the error?
What is odd, is here it shows that 1522 is normal size, and anything larger would be jumbo
https://img-en.fs.com/file/user_manual/s3900-series-configuration-guide.pdf
2.21 Jumbo frames
2.21.1 Introduction
Jumbo frames are Ethernet frames with a frame length greater than 1522 bytes.Could you show status of one of the interfaces your seeing the errors on?
show interfaces status ethernet
-
@johnpoz yeah. The switch seems to be a slightly newer model with a bit different syntax:
switch#show interface tg0/25 TGigaEthernet0/25 is up, line protocol is up protocolstatus upTimes 1, downTimes 0, last transition 2000-1-1 0:0:21 Ifindex is 189, unique port number is 49 Hardware is 10Giga-FX, address is 649d.9928.4a3d (bia 649d.9928.4a3d) MTU 1550 bytes, BW 10000000 kbit, DLY 10 usec Encapsulation ARPA Full-duplex, 10000Mb/s, Flow-Control Off 5 minutes input rate 778815 bits/sec, 117 packets/sec 5 minutes output rate 119461 bits/sec, 88 packets/sec Real time input rate 0%, 327494 bits/sec, 100 packets/sec Real time output rate 0%, 145816 bits/sec, 95 packets/sec Received 10549650 packets, 9541356381 bytes 2110 broadcasts, 12068 multicasts, 10535472 ucasts 0 discard, 0 error, 0 PAUSE 0 align, 0 FCS, 0 symbol 0 jabber, 318071 oversize, 0 undersize 0 carriersense, 0 collision, 0 fragment 0 L3 packets, 0 discards, 0 Header errors Transmitted 7473138 packets, 1683821836 bytes 313229 broadcasts, 112992 multicasts, 7046917 ucasts 0 discard, 0 error, 0 PAUSE 0 sqettest, 0 deferred, 277106 oversize 0 single, 0 multiple, 0 excessive, 0 late 0 L3 forwardsI tried it on a lesser used port and verify that as soon as I access the web interface the oversize counter goes up.
Here is the config:
interface TGigaEthernet0/25 switchport mode trunk switchport pvid 100 -
@shewless that is not showing the status.
show interfaces status ethernet tg0/25
You have the mtu on that port set to 1550..
-
@johnpoz that command doesn't work on my switch unless I'm in the wrong mode or something? My switch is the "newer" model: The https://www.fs.com/products/134655.html?attribute=8032&id=289447
switch#show interfaces status ethernet tg0/25 show interfaces status ethernet tg0/25 ^ Too many parametersswitch#show interface ? GigaEthernet -- GigaEthernet interface TGigaEthernet -- Ten GigaEthernet interface Vlan -- VLAN interface Null -- Null interface brief -- brief information of the interface range -- show interface range ifindex -- show interface based on ifindex | -- Output modifiers <cr>switch#show ethernet ? cfm -- Configure Connection Fault Management protocol(CFM) oam -- Operations, Administration and MaintenanceThe MTU was set globally. When I look at the manual I only see a way to do it globally for this switch (the -R version):
switch#show system mtu System MTU size is 1550 bytesWhat am I missing?
-
@shewless I would set it back to 1500, 1550 mtu is not standard by any means.
did you look over that link to the manual - isn't that your switch?
-
@johnpoz I can do that (set the MTU to 1500). I think when it was at 1500 I was seeing both "error" counter increases and "oversize" packet increases... I will change it now and see if that is the case.
I totally have been pouring over the manual. As I mentioned the -R version seems to have a different CLI unfortunately. -
@johnpoz yeah as soon as I set the system mtu to 1500 I see both "error" and "oversize" increasing.. though I haven't noticed any functionality problems.. I'd love to get to the bottom of this.
TGigaEthernet0/25 is up, line protocol is up protocolstatus upTimes 1, downTimes 0, last transition 2000-1-1 0:0:21 Ifindex is 189, unique port number is 49 Hardware is 10Giga-FX, address is 649d.9928.4a3d (bia 649d.9928.4a3d) MTU 1500 bytes, BW 10000000 kbit, DLY 10 usec Encapsulation ARPA Full-duplex, 10000Mb/s, Flow-Control Off 5 minutes input rate 202703 bits/sec, 71 packets/sec 5 minutes output rate 105389 bits/sec, 69 packets/sec Real time input rate 0%, 133720 bits/sec, 64 packets/sec Real time output rate 0%, 99284 bits/sec, 67 packets/sec Received 14056231 packets, 14473843206 bytes 2951 broadcasts, 12725 multicasts, 14040555 ucasts 0 discard, 478 error, 0 PAUSE 0 align, 0 FCS, 0 symbol 0 jabber, 340230 oversize, 0 undersize 0 carriersense, 0 collision, 0 fragment 0 L3 packets, 0 discards, 0 Header errors Transmitted 8803287 packets, 1840177991 bytes 334995 broadcasts, 121700 multicasts, 8346592 ucasts 0 discard, 0 error, 0 PAUSE 0 sqettest, 0 deferred, 287250 oversize 0 single, 0 multiple, 0 excessive, 0 late 0 L3 forwards -
@johnpoz I submitted a ticket to the switch vendor. I'll update here if I find out anything useful.
-
@shewless great - that damn curiosity cat of mine is is always meowing.. Yes please let us know what comes of that.
-
@johnpoz update:
I verified that on my ubuntu client that this ping works:
ping -c 10 -M do -s 1472 192.168.120.1
But it results in the oversize counter increasing. wireshark shows a packet size of 1514 for both request and reply.The size of 1468 is required to avoid oversize counter increasing
ping -c 10 -M do -s 1468 192.168.120.1
This results in a wireshark packet size of 1510.Likely a cosmetic problem as the packets all seam to go where they are supposed to go... a support case is opened for the switch vendor.
-
@shewless yeah that is odd! for sure.. 1522 should be max size, even says so in the doc I linked too. I could see anything over 1522 triggering the oversize counter, if you didn't have jumbo enabled..
-
@johnpoz Update:
Switch vendor believes it's okay to have to update the system MTU to a higher value (in order to remove the error counter) and to live with the "oversize" counter increases.
I have since set the system MTU to 9000 (which is the L2 MTU). Any IP Interface I create can have an L3 MTU applied to it of 1500.Thanks for the help.
-
@shewless said in Odd MTU / fragmented packet issue on web GUI and haproxy:
it's okay to have to update the system MTU to a higher value (in order to remove the error counter)
Yeah that is not the correct solution.. Mismatched mtu is never a good thing..
-
@johnpoz I agree that this is not the correct solution and I have let the switch vendor know. Maybe if other potential customers provide feedback they'll make changes.
In my case I don't think the MTU settings will be harmful. I've already tested that my endpoint devices all have "Layer3" MTU of 1500 and having the switch system MTU set to a higher value doesn't impact the traffic being sent. -
@shewless Personally I would just live with the cosmetic errors being listed most likely vs setting a clearly mismatched mtu.
As long as its not actually dropping the traffic, etc.
