Jumbo Frames (Larger MTU)



  • Hello all gurus,

    I wish to split my LAN segment and enable Jumbo Frames of 9000 on the new segment.
    I can enable it on my NAS and 'gamer' PC but under pfSense I get an error depicted in the attached picture.

    1.2-RELEASE






  • Well I tweaked interfaces_opt.php so that it would let me enter an mtu of 9000 without error then connected my PC (also set for 9000 mtu) to it.

    But alas it didn't work, I get oversized frame errors in the system log when I try to send packets of varying sizes above 1500.

    Its says these are kernel related discards! Is this a limitation of the 6.2 kernel or the kernel mode driver for the NIC?

    Cheers

    UPDATE: Yet more trawling of the net I found this,

    http://www.nabble.com/Call-for-testers%3A-re(4)-and-RTL8168C-RTL8168CP-RTL8111C-RTL8111CP-td17705059.html

    Looks like some bloke has written or tweaked the freebsd 7 drivers with the intention of better supporting jumbo frames. He is asking for testers for his patch, I wouldn't even know where to start! Is it beyond my means to upgrade to 1.2.1 (freebsd 7) and install this driver?




  • You should not use jumbo frames with your configuration, it increase latency badly, its only to be used for high speed networks (1Gbps+) and when you don't need a low latency. Also smaller MTUs permit more pipelining of the packets.



  • Forgive me but I am running (1Gbps+) and I am well aware of the benefits of running Jumbo!

    Light reading!
    http://sd.wareonearth.com/~phil/jumbo.html
    My switch supports it;
    http://www.3com.com/products/en_US/detail.jsp?pathtype=purchase&tab=features&sku=3CGSU08

    My Nas supports it;

    And my Quad core PC supports it;
    http://www.asus.com/products.aspx?modelmenu=2&model=1872&l1=3&l2=11&l3=572&l4=0
    Marvell 88E8056® PCIe Gigabit LAN controller featuring AI!

    I chuck around large .VOB files and HDTV streaming so feel I could benefit from the larger throughput and better CPU utilisation due to there being less encapsulation and de-encapsulation of frames.

    I could defiantly benefit from these results;

    But I'm having a little trouble getting my firewall to support it, even though I'm confident the hardware is capable.

    A Jetway J&F4K 1.2Ghz + 1Gb Ram
    http://www.jetwaycomputer.com/spec/J7F4-Series.pdf

    My understanding was that 1500 was chosen due to the quality of the lines and equipment at the time and if an error occures only 1,518-bytes would need to be re-transmitted. Now days with most networks being switched and the quality of the 'media' there is really no reason to stick at 1500 because of the quality of the kit even a home user has now days it is less likely (fewer collisions and mall formed packets) that a 9000 byte frame will need dumping and re-transiting.

    Regards



  • You may want to read the man page for re(4):

    http://www.freebsd.org/cgi/man.cgi?query=re&apropos=0&sektion=0&manpath=FreeBSD+6.2-RELEASE&format=html

    Where it says The MTU is limited to 7422, since the chip cannot transmit larger frames.

    So, no 9K frames for you ;)



  • The RealTek 8169, 8169S and 8110S chips appear to only be capable of transmitting jumbo frames up to 7.5K in size.

    Cheers for that.

    7.5K is still quite acceptable, why do you think I had trouble with a frame of even 5K? Looking at my error it says 1514 is the absolute max but your post sugests it can go higher and is not a limitation of FreeBSD 6.2.

    Cheers



  • Let us know if that works for ya.  I am interested on seeing if you successfully got jumbo frames working.  Your pic for you network setup looks very nice.  I like the 20mb connection >< and all the nice pics.  I have a gig linksys 16 port, but all that is on one interface and does not go through the firewall for that.



  • Well this weekend I had some spare time to mess around while th misses was out. Ironically it turned out to be easier to change the MTU under pfSense than my other devices.

    I had first tweaked interfaces_opt.php so that it would let me enter an mtu above 1500 without error. but on going to the console and typing # ifconfig re0 it still showed 1500?

    so I tried setting it from the console by typing # ifconfig re0 mtu 7422 and this worked a treat. so I then went to my windows box equipped with two different 1Gb NICS to select the same on that, but bugger me they where both different.
    The Marvell Yukon only had the option of 4088 bytes and 9014 bytes and the Realtek had 3KB -> 7KB MTU.
    As the Realtek was the very same chipset as my router I stuck with that and amused that the 7KB must equate to 7422 as I already know (thanks to Cry Havok) that that is the chips upper limit, this seemed to work fine!

    My problem now lies with my NAS that has the options of 4074, 7418 and 9000. I can select 7418 and manually knock my firewall down to 7418 but my windows PC doesn't have the option to enter manual input so I assume it would then be dishing out over sized packets that the NAS and router would just drop.

    My problem turned out to be one of conformation because in practice no matter what size packets I sent (ping's with a large size) they still replied because I assume they where jut being chopped up into smaller chunks.

    This left me with the following questions I wonder if any gurus can answer for me?

    1. How to make ifconfig mtu settings persistent across reboots?
    2. How to accurately test that a packet of a set size is being sent and received without any fragmentation (even with both ends set to 7422 I can still send a ping of 9000 bytes.)
    3. How to accurately test the potential performance increase once jumbo frames are in operation.
    4. How to show / set a windows mtu not using the driver applet.

    One thing I have learnt is that it is quite safe to mess with mtu values and not get disconnected. Even with both ends mismatched things like DHCP and ssl still work fine as the packets are so small your not going to reach the 7k packet where it may or may not be discarded.

    Also there doesn't seem to be much standardisation under 9k. On some boxes the 7K setting will be 7422 some will be 7418 and some just 7K - It does seem a bit hit and miss!












  • You can make it stick by putting the ifconfig commands in shellcmd tags (google it). The boxes you see on interfaces is for MSS clamping, not MTU. There's an open ticket to clarify that and add true MTU support for 1.3.



  • @cmb:

    You can make it stick by putting the ifconfig commands in shellcmd tags (google it). The boxes you see on interfaces is for MSS clamping, not MTU. There's an open ticket to clarify that and add true MTU support for 1.3.

    Ahhhhh…. thats why it doesn't show up on an ifconfig command!! Cheers - Saved me posting a question :-)



  • FYI - save a google search:

    Just add your command between two <shellcmd>command*</shellcmd> tags in the section of your .xml file like so.

    <system><optimization>normal</optimization>
    <hostname>router</hostname>
    <domain>cheesyboofs.co.uk</domain>
    <username>admin</username>
    <password>BLANK</password>
    <timezone>Europe/London</timezone>
    <time-update-interval><timeservers>0.uk.pool.ntp.org 1.uk.pool.ntp.org 2.uk.pool.ntp.org 3.uk.pool.ntp.org</timeservers>
    <webgui><protocol>http</protocol>
    <certificate><private-key></private-key></certificate></webgui>
    <disablenatreflection>yes</disablenatreflection>
    <ssh><authorizedkeys></authorizedkeys></ssh>
    <enablesshd>yes</enablesshd>
    <maximumstates><shapertype><dnsserver>208.67.222.222</dnsserver>
    <dnsserver>208.67.220.220</dnsserver>
    <shellcmd>ifconfig re0 mtu 7422</shellcmd></shapertype></maximumstates></time-update-interval></system>

    Now on to item 2.  ;)



  • Right, the result of yet more testing.

    To answer my own question 3. "testing the performance of a network drive" you can use a program called  PerformanceTest v6.1 which enables you to test the performance of network drives and compare them to previously saved baseline for comparison.

    Download
    Credits

    To answer my own question 4. To set the MTU under Vista you can use a command prompt command called netsh

    Credits

    netsh interface ipv4 show subinterfaces to show current MTU
    netsh interface ipv4 set subinterface "Local Area Connection" mtu=7418 store=persistent to set a new MTU of 7418

    Cool - Now I have my router set to 7418, my Vista PC and I know my NAS has a drop down that supports it.

    So I set the NAS and rebooted and hay presto I can ping the NAS, SSH to the NAS but can't bring up the web interface or browse the windows shares held on it.

    So knock my PC back to 1500 (netsh interface ipv4 set subinterface "Local Area Connection" mtu=1500 store=persistent) and it all instantly springs into life. So I do an # ifconfig eth0 on my NAS to make sure it is set for 7418 and sure enough it says MTU is 7420 (two more than it web interface)

    With all three devices set to 7420 they can ping each other with a fat packet (ping -l 7420 under windows ping -s 7420 under linux/freebsd) except that the windows PC can not ping the nas with a fat packet only with one of 1500 but the NAS can ping the PC, so I think that the NAS is lying and not capable of running with JUMBO frames even though it says it should so I will be giving the Qnap guy's an ear bending on their forum.

    Anyway, in summary Jumbo frames are supported and quite easy to implement and when I get my NAS playing ball I will be able to confirm the increase in speed.

    Cheers and hope this has given some people food for thought.


Log in to reply