Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Lagg - Make Jumbo Frames Persistent?

    Scheduled Pinned Locked Moved General pfSense Questions
    20 Posts 3 Posters 9.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • W
      wallabybob
      last edited by

      @allpoints:

      But the ifconfig command required to change a NIC's MTU (#ifconfig nicname mtu 9022) is lost on shutdown, and can't be applied to a running lagg.

      I wasn't discussing an MTU value configured by you issuing a shell ifconfig command, I was discussing an MTU value configured on the lagg member interfaces (and possibly the lagg interface itself) through the web GUI and saved (by clicking on the Save button) in the configuration database.

      1 Reply Last reply Reply Quote 0
      • A
        allpoints
        last edited by

        @wallabybob:

        @allpoints:

        But the ifconfig command required to change a NIC's MTU (#ifconfig nicname mtu 9022) is lost on shutdown, and can't be applied to a running lagg.

        I wasn't discussing an MTU value configured by you issuing a shell ifconfig command, I was discussing an MTU value configured on the lagg member interfaces (and possibly the lagg interface itself) through the web GUI and saved (by clicking on the Save button) in the configuration database.

        I was discussing why that approach does not work. If it were as simple as typing 9014 into Interfaces>LAN>MTU field in the GUI, or editing config.xml, I wouldn't need to ask the hivemind for suggestions.

        I've tried <shellcmd>and <earlyshellcmd>tags around the appropriate ifconfig commands in config.xml, but the lagg is configured by the time those commands are read, so they fail.
        Config.xml seems to merely assign interfaces after they are configured by the driver modules one can't modify in pfSense userland.

        The only way to configure a lagg MTU is via the individual member NIC MTUs.
        The only way to configure a NIC's MTU in pfSense is to change it with ifconfig after it is plumbed with MTU 1500 by the driver module, which is not configurable in pfSense userland.

        tl;dr: Web config doesn't work with a lagg. Ifconfig doesn't work with a booted lagg or survive a reboot. Config.xml is as powerless over a lagg as ifconfig.</earlyshellcmd></shellcmd>

        1 Reply Last reply Reply Quote 0
        • W
          wallabybob
          last edited by

          Have you tried it?

          1 Reply Last reply Reply Quote 0
          • A
            allpoints
            last edited by

            @wallabybob:

            Have you tried it?

            Yes, I have tried it.  Interestingly enough, this is precisely how I know it does not work.

            ;D

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              The actual problem here is that you have to define the MTU on the member interfaces of the LAGG and there is no way to do that via the GUI because the member interfaces are not assigned so there are no config options for them. Is that it?

              Steve

              1 Reply Last reply Reply Quote 0
              • A
                allpoints
                last edited by

                @stephenw10:

                The actual problem here is that you have to define the MTU on the member interfaces of the LAGG and there is no way to do that via the GUI because the member interfaces are not assigned so there are no config options for them. Is that it?

                Steve

                Somewhat…
                I can set a MTU value in Interfaces>LAN. This works fine for a single NIC say, em0.
                I can find the undesignated NICs in Interfaces>Assign>LAGG, Cntrl+clik to designate the multiple interfaces - say em0 and em1 - select a protocol, and build a LAGG, and assign it to LAN.

                BUT:
                Unless em0 and em1 have been set to a nondefault MTU beforehand, the LAGG will have an MTU of only 1500, regardless of what is entered into the MTU field in Interfaces>LAN>MTU.
                In case of reboot, pfSense loses any MTU setting made to LAGG members beforehand, then proceeds to configure the LAGG with members set to default MTU, which results in a LAGG set to default MTU. No more Jumbo Frames.

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Excactly.
                  You can only set an MTU value in the GUI to, say, em0 once it has been assigned but you can only add unassigned interfaces to the LAGG.
                  If you do it manually in the right order it works because behind the scenes it's just doing ifconfig and unassigning the interface does not reset the MTU. However it does remove it from the config file so after a reboot it doesn't come back up.
                  Is it possible to use a manual hack on the config file to assign attributes to LAGG member interfaces?
                  Edit: Doesn't look like it is but I'm open to suggestions.

                  This seems a bit like overkill but you could probably use the record/playback feature in the php shell to generate a script that configures the MTU of each member and then reconfigures them as LAGG. Have it playback at boot. I've never tried that.  ;)

                  Steve

                  1 Reply Last reply Reply Quote 0
                  • A
                    allpoints
                    last edited by

                    @stephenw10:

                    Excactly.
                    You can only set an MTU value in the GUI to, say, em0 once it has been assigned but you can only add unassigned interfaces to the LAGG.
                    If you do it manually in the right order it works because behind the scenes it's just doing ifconfig and unassigning the interface does not reset the MTU. However it does remove it from the config file so after a reboot it doesn't come back up.
                    Is it possible to use a manual hack on the config file to assign attributes to LAGG member interfaces?
                    Edit: Doesn't look like it is but I'm open to suggestions.

                    This seems a bit like overkill but you could probably use the record/playback feature in the php shell to generate a script that configures the MTU of each member and then reconfigures them as LAGG. Have it playback at boot. I've never tried that.  ;)

                    Steve

                    Thanks for taking an interest, and thanks for the PHP shell suggestion. I'll let you know how it works. Hopefully, it won't hinge on just when in the boot process the playback script needs to run…

                    ;D

                    1 Reply Last reply Reply Quote 0
                    • A
                      allpoints
                      last edited by

                      PhP shell playback just modifies config.xml after the LAGG is built.
                      As such, it's no more effective than anything else done in config.xml, because it's read too late in the boot to change anything about a LAGG.

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        But you can create a script that removes the LAGG, assigns the interfaces noamally, sets the MTU and then removes them and replaces the LAGG.
                        Yes, a horrendous hack!  ::)

                        Steve

                        1 Reply Last reply Reply Quote 0
                        • A
                          allpoints
                          last edited by

                          @stephenw10:

                          …
                          Yes, a horrendous hack!  ::)

                          Steve

                          A Parable:

                          The '69 Chevelle Malibu SS with the 396ci High Output engine had a thing with spark plugs. That big block was shoehorned into the engine compartment so tight that it was difficult, if not impossible, for a grown man to get his hand into a place where he could pull the plug wire and put a deep socket on the plug.
                          When these cars inevitably found their way into the service bays of dealer techs and other experienced professionals, the first thing that happened was the pro loosened all the motor mounts, jacked up the front of the engine, took a metal bar and a 5-lb sledgehammer and dented in the car's firewall to allow a human being to apply a wrench to number seven and eight @#$% spark plugs.

                          FF to present:

                          Is there anyone reading this who wouldn't want a '69 Chevelle SS with a 396 and a slightly dimpled firewall?

                          The End

                          1 Reply Last reply Reply Quote 0
                          • A
                            allpoints
                            last edited by

                            @stephenw10:

                            But you can create a script that removes the LAGG, assigns the interfaces noamally, sets the MTU and then removes them and replaces the LAGG.
                            Yes, a horrendous hack!  ::)

                            Steve

                            "We had to burn LAGG0 in order to save it."

                            Easiest way:
                            1. Backup config.xml from Webconfigurator to desktop.
                            2. Open config in text editor, "Save As"
                            3. Add appropriate ifconfig commands inside <earlyshellcmd></earlyshellcmd>tags placed directly above the existing tag in the .xml. The existing LAGG must be destroyed first, then an appropriate MTU can be applied to the former/future members, then a new LAGG0 is configured manually, ie

                            <earlyshellcmd>ifconfig lagg0 destroy</earlyshellcmd>
                            <earlyshellcmd>ifconfig em0 mtu 9022</earlyshellcmd>
                            <earlyshellcmd>ifconfig em1 mtu 9022</earlyshellcmd>
                            <earlyshellcmd>ifconfig lagg0 laggproto roundrobin laggport em0 laggport em1 \ 172.16.1.1 netmask 255.255.255.0</earlyshellcmd>
                            
                            

                            4. Save newly edited .xml.
                            5. In Webconfigurator>Backup/Restore, restore with newly edited config.xml you just saved.
                            6. pfSense reboots
                            7. Get a shell and check your work:

                            ifconfig lagg0
                            lagg0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 9022
                            ...</up,broadcast,running,simplex,multicast>
                            
                            ping -s 8900 -D 172.16.1.112
                            PING 172.16.1.112 (172.16.1.112): 8900 data bytes
                            8908 bytes from 172.16.1.112: icmp_seq=0 ttl=128 time=0.957 ms
                            8908 bytes from 172.16.1.112: icmp_seq=1 ttl=128 time=0.814 ms
                            8908 bytes from 172.16.1.112: icmp_seq=2 ttl=128 time=0.848 ms
                            ^C
                            --- 172.16.1.112 ping statistics ---
                            3 packets transmitted, 3 packets received, 0.0% packet loss
                            round-trip min/avg/max/stddev = 0.814/0.873/0.957/0.061 ms
                            

                            8. Sit back and bask in the higher transport speeds and lower resource utilization of your jumbo aggregation.  ;D

                            Thanks stephenw10! Your suggestion to destroy the LAGG first was the key.

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Nice!  :)
                              pfSense still shows the correct interfaces in the dash etc?
                              Not pretty but if it works…

                              It should be possible to do this in the webgui IMHO. There was a change to the way bridge interfaces are handled that would seem similar to this problem. Previously the bridge itself could not be assigned as an interface. It seems to me quite likely that if you need to create a lagg to increase bandwidth you might also want to use jumbo frames. Something for the future perhaps.

                              Steve

                              1 Reply Last reply Reply Quote 0
                              • A
                                allpoints
                                last edited by

                                @stephenw10:

                                Nice!  :)
                                pfSense still shows the correct interfaces in the dash etc?

                                Yep.  :)

                                Not pretty but if it works…

                                Indeed…

                                It should be possible to do this in the webgui IMHO. There was a change to the way bridge interfaces are handled that would seem similar to this problem. Previously the bridge itself could not be assigned as an interface. It seems to me quite likely that if you need to create a lagg to increase bandwidth you might also want to use jumbo frames. Something for the future perhaps.

                                Steve

                                Tried to edit in GUI, but it hung on "Save", and wouldn't refresh the page. Probably a browser thing on my end.
                                I don't see why it wouldn't work to edit /cf/conf/config.xml in the GUI.

                                Also, this may well work with the preferred <shellcmd>tags rather than <earlyshellcmd>. I tried "Early" first, and it worked, so I quit winners.  :D

                                I think the "glitch" is systemic with ifconfig(8), and originates in ifconfig not having the capacity to pass a flag for persistence across boots. The FreeBSD ifconfig(8) man never even considers it, but it's common enough knowledge so that we pretty much all know we have to make an entry in rc.conf, either manually or via sysctrl, by the time most of us develop the courage to pop the hood on pfSense.

                                But pfSense is hardened, and so it can't work like that.
                                This is the definition of an "ugly" hack in that it has to destroy and recreate something after the fact, but what the hell, it works.

                                </earlyshellcmd></shellcmd>

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post
                                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.