Jumbo frames and MTU
I'm troubleshooting a Jumbo Frame issue on my LAN and something came up when investigation but i'm not sure if its actually an issue.
Basically, in pfsense, the LAN interface if set to use 9000 bytes frames and the ifconfig command returns this exact information for the corresponding interface (em0).
However, the netstat -rnW command returns me a MTU of 1500 for anything related to the em0 (thus LAN) interface.
Is this normal? What does it mean exactly?
if I run "route change 192.168.0.0/24 -mtu 9000" I lose connectivity and VM crashes at some point…
That's a FreeBSD 8.3 bug. It's possible to manually work around via 'route change' hacked into a shellcmd. It's fixed in the base OS in 2.2.
I'm sorry to report, that this issue is indeed not solved in 2.2.
[2.2-RC][admin@vpn201a]/root: ifconfig bge2
bge2: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 9000
options=c009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,vlan_hwtso,linkstate>ether a0:d3:c1:02:5c:da
inet XXX.XX.XX.XX netmask 0xffffffe0 broadcast XXX.XX.XX.XX
inet XXX.XX.XX.XX netmask 0xffffffe0 broadcast XXX.XX.XX.XX vhid 27
media: Ethernet autoselect (1000baseT <full-duplex>)
carp: MASTER vhid 27 advbase 1 advskew 0
[2.2-RC][admin@vpn201a]/root: route get XXX.XX.XX.X
route to: XXX.XX.XX.X
flags: <up,done,pinned>recvpipe sendpipe ssthresh rtt,msec mtu weight expire
0 0 0 0 1500 1 0
[2.2-RC][admin@vpn201a]/root: uname -a
FreeBSD vpn201a 10.1-RELEASE-p2 FreeBSD 10.1-RELEASE-p2 #0 d1c9830(releng/10.1)-dirty: Thu Dec 18 23:14:17 CST 2014 root@pfsense-22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10 amd64
Output anonymized (sorry, cannot post real IPs)</up,done,pinned></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,vlan_hwtso,linkstate></up,broadcast,running,promisc,simplex,multicast>
It is definitely solved in general in 2.2. Add a static route to a gateway reachable on an interface with jumbo frames enabled at 9000, and you get:
# route -n get 220.127.116.11 route to: 18.104.22.168 destination: 22.214.171.124 mask: 255.255.255.255 gateway: 192.168.125.254 fib: 0 interface: lagg0_vlan1001 flags: <up,gateway,done,static>recvpipe sendpipe ssthresh rtt,msec mtu weight expire 0 0 0 0 9000 1 0</up,gateway,done,static>
We don't have bge NICs in our lab systems so not sure about those specifically. There was some additional weirdness in jumbo frames in bge in 8.x that may still be an issue.
I've had no chance to test with a static route yet, but I can confirm that this doesn't work with the default route on 2.2-RC, even if the interface for the default route is set to 9000 MTU.
The workaround with "route change x.x.x.x/x -mtu 9000" command in shellcmd for the default route works just fine.
Still on bge NICs? Just ran through a couple tests there, one on igb and one on vmx in ESX and both update the default route's MTU as well.
default 172.27.44.1 UGS 121 9000 vmx0