Is pfSense handling jumbo frames correct !??

Here's my understanding of it.

When it says MTU, it would be usually either 1500 or 9000. When it says Frame Size, it would be MTU size + 18, so 1518 or 9018. However, some NIC drivers let you select 9014 instead of 9018, but that means really the same. Also, some NIC drivers try to make it user-friendly and show 9K instead. Again, it means the same.

pfSense says MTU, so it would be 1500 or 9000. If MTU is set on a parent interface (NIC), all child interfaces inherit that setting by default. That can be changed per child interface to a lower value, but I think it takes effect after pfSense is restarted.

In general, MTU can be set up on a L2 or L3 interface. For L2, it would be a port/NIC. For L3, it would be a SVI/VLAN interface.

I don't think there is a thing like a LAG MTU.

louis2

@kjk54

I have the same feeling, but its stays confusing !

Realtek one choose 9014 => 9014
Mellanox .... default is 1514 => 9014
Netgear one size 16349 => 16349
TP-link ^jumbo 1518 – 9216^ => 9018
zyxel (most switches) Jumbo frame is enabled by default, with support for frame sizes of up to 9K.
TrueNas free format however the text is suggesting 1500 or 9000 => 9000
windows depending om the used interface
etc (if not clear => 9018)

That does not take away that will work or is peace of cake or will instantly work.
Show my testing in the past days .....

On 1-GB vlan's and vlans which do not require high performance and the WAN, I will set the MTU to 1500
on the others 9000

Problem is that I still have no idea how to set the MTU on my pfSense physical interfaces.
(appart from the WAN-interface all other interfaces shown are vlans not the underlaying physical interfaces)

@louis2 said in Is pfSense handling jumbo frames correct !??:

Problem is that I still have no idea how to set the MTU on my pfSense physical interfaces.

In pfSense WebUI, go to Interfaces>Interface Assignments and select a parent interface, say LAN. You can enter MTU for it in the General Configuration section. The interface configuration will also show you the corresponding NIC. If you want to see how it really plays, SSH into pfSense shell and enter ifconfig | grep mtu.

I have jumbo frames enabled on all my switches, CISCO and Netgear, but I do not actually have any practical use-case for it. Although I have two NAS servers, they are of a general kind. At one point, I thought about setting up a TrueNAS server for my ESXi host but opted out for a hardware RAID instead. That was the time I was actually thinking about using jumbo frames.

JKnott

@louis2 said in Is pfSense handling jumbo frames correct !??:

If the famous "1500" and "9000" values are intended as netto values than 1518 and 9018 sounds logical .(+ ethernet header = 14 and crc = 4)

No, they don't include header & CRC. Frame expansion started in the late 90s, to allow for things like VLAN tags, etc.. Later on came jumbo frames. As I have never worked with LAGG, I can't say for certain, but I wouldn't be surprised if it's MTU is dependent on the underlying interface, as is the case with VLANs. So, you set the interface to 9K or whatever and the LAGG follows suit. Give it a try and see what happens. The ifconfig command shows the MTU.

louis2

@JKnott

To take away some misunderstanding, the situation is as follows:

The PPOE-interface is on em0. I I think that one should use the normal 1500 MTU. So nothing to do there
Then there is a lag using two 1G-Intel NIC's. That Lagg is the trunk for all 1G-VLAN's
(I probably leave the MTU to 1500 there)
And last but not least there is a X520-dual-10G intel card in favor of the 10G-lagg. The trunk for all 10G-VLAN's
(I will change the MTU to 9000 for this LAGG)

So all interfaces are VLAN's. There is no ^physical LAN^.

I have no idea how / where to set the MTU's as related to the four NIC's carrying the two LAGG's

JKnott

@louis2 said in Is pfSense handling jumbo frames correct !??:

So all interfaces are VLAN's. There is no ^physical LAN^.

When you create a VLAN, you have to specify a parent interface. That's where you set the MTU.
With LAGG, you also have to specify the parent interface.

I use VLAN3 for my guest WiFi. It's parent interface is LAN on igb1. In the configuration for LAN, I can set the MTU and so also for VLAN3.

BTW, here's some history on Ethernet frames:

The Ethernet

Extended Ethernet Frame Size Support

louis2

@JKnott

I does not work that way

The problem is that if you assign an physical interface to a lagg, it is ofcourse not longer available as an interface.
So there is no way to reach that setting.

There are only two very dirty way I could try:

method-1 (should not work)

assign an interface to the physical nic,
change the MTU value
delete the interface and ^hope^ that the MTU value stays in the DB (it sohould not!!!!)
assign the physical nic to the lagg

method-2 (could perhaps work)

save the config
look for the involved interface with an editor
change the MTU
store the file
load the file

Both methods are not OK of course

skogs

With nice fluke gear I've been able to not only measure very obvious hardware cpu load reductions with jumbo, but also vastly improved speed. 10G jumbo/9k is indeed noticeably faster than 10G normal MTU.

LAGG on the other hand, almost never gives the theoretical improvements expected. I'm guessing manual assignment will be your answer if it isn't hiding on interface lagg settings like it was a few years back.
I'm curious if you still need to tweak kernel settings afterward.

JKnott

@louis2 said in Is pfSense handling jumbo frames correct !??:

method-1 (should not work)

Have you tried it?

Here's what the pfSense manual says:

"Link aggregation is handled by lagg(4) type interfaces (LAGG) on pfSense software. LAGG combines multiple physical interfaces together as one logical interface. There are several ways this can work, either for gaining extra bandwidth, redundancy, or some combination of the two."

As I mentioned before, you create the LAGG from physical interfaces. I expect it would inherit whatever MTU the physical interfaces had.

It's simple enough to check this. Change the MTU and then create the LAGG. You can then use ifconfig to check the MTU.

This is so simple to try, arguing about it is wasting our time.

louis2

@JKnott

I had a look inside the configfile. There you see this

<laggs>
	<lagg>
		<members>igb0,igb1</members>
		<descr><![CDATA[LAGG TO 1G MAIN SW (GS1920)]]></descr>
		<laggif>lagg0</laggif>
		<proto>lacp</proto>
		<lacptimeout>slow</lacptimeout>
		<lagghash>l2,l3,l4</lagghash>
	</lagg>
	<lagg>
		<members>ix0,ix1</members>
		<descr><![CDATA[LAGG to 10G MAIN Switch (SX3008F)]]></descr>
		<laggif>lagg1</laggif>
		<proto>lacp</proto>
		<lacptimeout>slow</lacptimeout>
		<lagghash>l2,l3,l4</lagghash>
	</lagg>
</laggs>

AND

	<opt17>
		<descr><![CDATA[Emerg_Mngt]]></descr>
		<if>igb2</if>
		<spoofmac></spoofmac>
		<enable></enable>
		<ipaddr>192.168.9.1</ipaddr>
		<subnet>24</subnet>
		<mtu>9000</mtu>
	</opt17>

However there is no config block as show above for igb2 in favor of
igb0 / igb1 / ix0 / ix1

Neither is there such a config set for em0 .

The only situation where I see an ^<op117> like control blok, is in case of a "Physical LAN"

So adding not yet existing control block types, feels very hazzy

I think I will open a ticket. Lets see how the developers react ...