pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link)
-
@JKnott
I run 1508 MTU on my WAN, 1500 MTU on my maintenance LAN and 9k set on my main LAN and VLAN at 10 GbE.In this mixed environment I have 2 main switches, a couple of tertiary switches, 5 APs, numerous servers, clients and sundry networked items. Everything purrs along as it should.
️ -
@RobbieTT said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
9k set on my main LAN and VLAN at 10 GbE.
Have you tried Wifi on it? I'm curious if it will handle the larger frames. I believe you'll need to run 802.11ac or later to get the aggregation. I can't do that here, as my notebook computer is 12 years old and only runs 802.11n on 2.4 GHz.
PfSense running on Qotom mini PC
i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
UniFi AC-Lite access pointI haven't lost my mind. It's around here...somewhere...
-
But upstream on a WAN where you have no idea what is in the path I would always use 1500 unless whoever does control that hop has explicitly told you to use something larger. Yes, path MTU discovery should catch it but when it doesn't it sucks!
-
@JKnott said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
Have you tried Wifi on it? I'm curious if it will handle the larger frames. I believe you'll need to run 802.11ac or later to get the aggregation. I can't do that here, as my notebook computer is 12 years old and only runs 802.11n on 2.4 GHz.
The 10 GbE / 9k MTU has access points on it but wifi is not ethernet - encapsulated packets, additional headers, inherently simplex, pause frames, beacons, air time et al - so nothing native is carried across unadulterated on the RF segment of a link.
-
@stephenw10 said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
But upstream on a WAN where you have no idea what is in the path I would always use 1500 unless whoever does control that hop has explicitly told you to use something larger. Yes, path MTU discovery should catch it but when it doesn't it sucks!
Does anyone even try to run jumbo frames over a WAN - in almost all scenarios you would not make it past the first hop?
-
@RobbieTT said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
so nothing native is carried across unadulterated on the RF segment of a link.
What happens if you just connect your own AP to the LAN side?
PfSense running on Qotom mini PC
i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
UniFi AC-Lite access pointI haven't lost my mind. It's around here...somewhere...
-
@RobbieTT said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
Does anyone even try to run jumbo frames over a WAN - in almost all scenarios you would not make it past the first hop?
If the other side of the router is just 1500, then you'd experience fragmentation or ICMP too big messages. Either way, it should still work. This is part of the function of every router.
-
@JKnott said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
What happens if you just connect your own AP to the LAN side?
I do have my own APs on the LAN (& a VLAN) but wifi is still wifi. Even protocols can change in function over a wifi segment - for example, UDP packets are subject to ACKs over a wifi segment in a similar fashion to that of TCP over a wired segment.
️ -
You seem to be missing my point. Can you ping various sizes beyond 1500, to see if they're successfully passed, without fragmentation or PMTUD? You may need to run Wireshark to see what's happening. I just want to know if WiFi is usable with jumbo frames. WiFi, without aggregation, has a larger MTU than 1500. I think it's around 2300 or so. What happens if you try to send a large file through WiFi?
I know the operation of WiFi is different than Ethernet, but it is effectively a bridge with all the dirty details hidden.
-
@JKnott
If you are looking for the book figure* then the MTU you are seeking is 2312 bytes, so yes, it can easily be seen as jumbo frames as it is above the normal 1500 (well 1514) byte frame. But those 2312 bytes are a bit of a misnomer and most wifi boffins would not recognise it.It is more of an air-time space for transmission to take place, inclusive of all the additional overheads. Some may even argue that 2312 MTU should be cut in half due to the simplex nature of wifi; but as we are really talking about an airtime opportunity I think that is poor analogy.
WiFi is just RF and RF is fun!
️*Often quoted as such but not actually the case in modern wifi but who cares
-
@RobbieTT said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
WiFi is just RF and RF is fun!
I know. I just borrowed this book from the library. It's the first physical book I've read in years, as they didn't have it in an ebook.
I have also done some work with cell phone sites and short haul microwave and got my amateur radio licence almost 52 years ago.
-
@JKnott said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
I have also done some work with cell phone sites and short haul microwave and got my amateur radio licence almost 52 years ago.
5G seemed to go with simplicity and defined requirements; albeit they had to be steered in that direction in the early days. I don't think you could pay them now to do some of the stuff they originally had in mind. It's all about stability as that = printing money.
My background was more on the exploiting, spoofing, denying, intercepting or generally being on the aggressive side of RF. I did also have to sit on the UK board that allocates and manages the RF spectrum. That was the 'not fun' part of RF.
Regarding your thoughts on testing beyond 1500 MTU on a wifi segment. It's harder than you may think with commodity hardware as most have barriers in place to stop you from doing so. Many wifi interfaces deny the opportunity to fettle the interface at all. Some look like you can tweak them but under the hood nothing changes, either due to the OS or the driver/firmware/hardware itself.
There are exceptions of course - macOS lets you control and pass MTU changes to the wifi interface but in more recent version this became limited to a range of 1280 to 1436 MTU. The point to note here is that it will not allow anywhere near the minimum or maximum MTU unless 'auto' is in place (which caps the interface at 1514 MTU). Ironically the only commodity hardware that used to use a wifi MTU above 1514 was also Apple in the guise of AirPort Time Capsules and Express units.
Meanwhile we are seeing a drift to a new-normal MTU of 1280 (1294), so even the 1500 (1514) MTU will start to look relatively capacious.
️ -
@RobbieTT said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
Meanwhile we are seeing a drift to a new-normal MTU of 1280 (1294),
Where is that? It's definitely a step in the wrong direction.
-
@JKnott said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
Where is that? It's definitely a step in the wrong direction.
We are partially there already due to Google's interpretation of TLS 1.3, HTTPS/3 and QUIC. YouTube for example:
The joys of GAFAM.
️ -
@RobbieTT said in pfSense as router: question about MTU to be set on a 10Gbps interface (WAN interface to 10Gbps ISP link):
We are partially there already due to Google's interpretation of TLS 1.3, HTTPS/3 and QUIC. YouTube for example:
That's not the same as layer 2 technology. VoIP uses even smaller packets. I don't know enough about QUIC etc. to comment.
-
@JKnott
L2 spends a lot of time moving formerly-L3 traffic around.️ -
L2 is always moving L3 around, as it doesn't do anything by itself. L3 is encapsulated in L2. Fire up Wireshark and see what's happening.
-
@JKnott
The levels in the OSI model are just abstract constructs, they are not protocols or encapsulation methods. They are somewhat dog-eared in the modern era and are more like marker poles for our thinking or short-hand for describing stuff in loose handfuls.
️ -
OK. Here's a Wireshark capture of me starting a SSH session:
You see that line that begins Ethernet II? That's layer 2
Then, a bit further down is Internet Protocol Version 6. That's layer 3
Below that, is Transmission Control Protocol. That's layer 4.
It's beyond this, starting with SSH Protocol that we differ from OSI, with layers 5-7, incorporated into the application layer.So, we have SSH encapsulated in TCP, encapsulated in IPv6, encapsulated in an Ethernet frame. That is the protocol stack we use these days.
BTW, I first learned about TCP/IP through a local college in spring of 1995. I then had a Novell 3.x CNA course and got certified, followed by CNE 4 I then learned more when I was working at IBM, where I did 3rd level support, with the added bonus of token ring and SNA. I have several books on TCP/IP and Ethernet, which I have read cover to cover. And I've had a lot more experience over the years and frequently use Wireshark or the Packet Capture in pfSense, to examine network traffic.
-
We may have strayed from the topic here.
