Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Failover LAGG of LACP LAGGs (Nested LAGG)

    Scheduled Pinned Locked Moved General pfSense Questions
    16 Posts 9 Posters 2.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      cpfarhood
      last edited by

      I'd like to create 2 LACP LAGGs (each to separate switches) then build a failover LAGG on top of that (and VLAN tags on top of that).  Is this something that can be accomplished?

      1 Reply Last reply Reply Quote 0
      • J
        jasonlitka
        last edited by

        You can't put a lagg into another lagg.

        Why would you even want to do this?

        I can break anything.

        1 Reply Last reply Reply Quote 0
        • jimpJ
          jimp Rebel Alliance Developer Netgate
          last edited by

          Get switches that support proper stacking and then just use one lagg with four ports across two chassis.

          If your switches don't support stacking, then you're out of luck for proper cross-switch failover.

          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

          Need help fast? Netgate Global Support!

          Do not Chat/PM for help!

          1 Reply Last reply Reply Quote 0
          • C
            cpfarhood
            last edited by

            @Jason:

            You can't put a lagg into another lagg.

            Why would you even want to do this?

            It's quite simple really, I'm attempting to attach to 2 separate physical switches for failover and want more than a single GB of throughput, I do this often with NetApp equipment.  Up until Cisco Nexus stacking across chassis (Virtual Port Channel) is not supported.

            1 Reply Last reply Reply Quote 0
            • C
              cpfarhood
              last edited by

              @jimp:

              Get switches that support proper stacking and then just use one lagg with four ports across two chassis.

              If your switches don't support stacking, then you're out of luck for proper cross-switch failover.

              Any chance of this being an option in the future? any idea if it's a FreeBSD or pfSense specific limitation?

              1 Reply Last reply Reply Quote 0
              • jimpJ
                jimp Rebel Alliance Developer Netgate
                last edited by

                A "lagg of laggs" I'm not sure will ever come to be.

                If you use only the "failover" mode, it might work, though it's largely unverified.

                LACP is known/proven to work in that situation, provided your switches are stackable.

                Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                Need help fast? Netgate Global Support!

                Do not Chat/PM for help!

                1 Reply Last reply Reply Quote 1
                • C
                  cmb
                  last edited by

                  @cpfarhood:

                  @jimp:

                  Get switches that support proper stacking and then just use one lagg with four ports across two chassis.

                  If your switches don't support stacking, then you're out of luck for proper cross-switch failover.

                  Any chance of this being an option in the future? any idea if it's a FreeBSD or pfSense specific limitation?

                  Neither. It's a limitation of your switches in that case. One that's impossible to work around without your switches being involved. Ethernet bonding in general can't be nested.

                  1 Reply Last reply Reply Quote 0
                  • C
                    cthomas
                    last edited by

                    I'll point out that even Netgear ProSafe switches in a stack can support a LAG distributed across two or more switches in the stack.

                    1 Reply Last reply Reply Quote 0
                    • P
                      ph0x
                      last edited by

                      Any changes on that topic?

                      "Why would you even want to do this?" Because I have a 10 GbE switch (with an lacp bond) and a 1 GbE switch (with an lacp bond) which should work as backup if the fast one fails. This has been working reliably under Linux, which is why I was surprised to find out that FreeBSD doesn't seem to support it.

                      And yes, I know, that this is an old thread, but I signed up exactly for this issue. ;-)

                      M stephenw10S 2 Replies Last reply Reply Quote 0
                      • M
                        michmoor LAYER 8 Rebel Alliance @ph0x
                        last edited by

                        @ph0x I have never seen this setup used in an enterprise. Typically there are switches that support stacking OR using MC-LAG. Just an odd design thats being requested.

                        Firewall: NetGate,Palo Alto-VM,Juniper SRX
                        Routing: Juniper, Arista, Cisco
                        Switching: Juniper, Arista, Cisco
                        Wireless: Unifi, Aruba IAP
                        JNCIP,CCNP Enterprise

                        P 1 Reply Last reply Reply Quote 1
                        • P
                          ph0x @michmoor
                          last edited by

                          Well, not everyone runs pfSense in an enterprise environment, right?

                          M 1 Reply Last reply Reply Quote 0
                          • M
                            michmoor LAYER 8 Rebel Alliance @ph0x
                            last edited by

                            @ph0x of course but not everyone needs nested LAGGs when there are proper solutions out there.

                            Firewall: NetGate,Palo Alto-VM,Juniper SRX
                            Routing: Juniper, Arista, Cisco
                            Switching: Juniper, Arista, Cisco
                            Wireless: Unifi, Aruba IAP
                            JNCIP,CCNP Enterprise

                            P 1 Reply Last reply Reply Quote 0
                            • P
                              ph0x @michmoor
                              last edited by

                              Buying a redundant switch only for failover if there already is a failover switch is obviously not deemed a proper solution by everone. There are (or have been) at least two persons in this forum that work it like that, and in different environments (NetApp and Proxmox).

                              Sad, but that's how it is.

                              Dobby_D 1 Reply Last reply Reply Quote 0
                              • Dobby_D
                                Dobby_ @ph0x
                                last edited by

                                @ph0x

                                Any changes on that topic?

                                I personally think that someone was mixing up some different points or was not knowing it better to ask or
                                point. If I read it again and again and again, I will be
                                seeing three different things got merged together.
                                But perhaps I was not really able to understand it
                                right, so please be patient and excuse it if I am
                                wrong with it.

                                LACP LAGG (dynamic LAGG)
                                You use it to setup a fail save link or as a combined link
                                to gain more throughput or better a higher throughput.
                                It is also called bond (Linux term) and transporting vlans
                                it is called a trunk (Cisco term).

                                We use it here in some cases and different modes in
                                the entire company network. We set up;

                                • LAGGs from servers to one switch or more switches
                                  Gaining the throughput and redundancy
                                • LAGGs from switches to switches
                                  Throughput and redundancy
                                • LAGGs from ringed switch stacks to other switches
                                  (ToR or Core Switches) as LAGG or MLAG
                                  Throughput and redundancy
                                • LAGGs from switch stacks members to servers (OSPF)
                                  Throughput gain, short patch and freeing the Core from load on top, if it is to much traffic

                                We are also using VLANs and this for the management, WiFi APs, IP cameras, VOIP phones, printers, servers,.........
                                We set up a VLAN1 for the admin and the management
                                and then the other vlans for all the devices named above.

                                If all is ready we lay over all vlans, but not the vlan1, another VLAN to separate it from the VLAN1, and
                                this not here and there, we do it even.

                                From the router or firewall to the core we use VRRP and from the core to the distribution and access layer we use OSPFv2/3. All is running fine here.

                                If we talk about network redundancy in normal we talk about money till 25.000 € and if it goes about HA (high
                                availability) it starts at the amount of 25.000 €.

                                Using Netgear or Cisco is here not the question in my eyes
                                and also not if Netgear is serving that point. It is more in wich manner the question was asked. You can link up LAGGs and you may be able to set up vlans over
                                another vlan with switches, and also in that manner
                                that the router or firewall must be not involved.

                                .....I was surprised to find out that FreeBSD doesn't seem to support it.

                                And on the other side OpenBSD and FreeBSD are able to serve a load balancer over CARP (active/active) where the other operating systems will be not able to do.

                                And yes, I know, that this is an old thread, but I signed
                                up exactly for this issue. ;-)

                                • Use another linux based router after the firewall
                                • Use it only on the switch site (from switch to switch)

                                May be not the best solutions but working. Perhaps their brand new TNSR is able to realize it?

                                #~. @Dobby

                                Turris Omnia - 4 Ports - 2 GB RAM / TurrisOS 7 Release (Btrfs)
                                PC Engines APU4D4 - 4 Ports - 4 GB RAM / pfSense CE 2.7.2 Release (ZFS)
                                PC Engines APU6B4 - 4 Ports - 4 GB RAM / pfSense+ (Plus) 24.03_1 Release (ZFS)

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator @ph0x
                                  last edited by stephenw10

                                  @ph0x said in Failover LAGG of LACP LAGGs (Nested LAGG):

                                  Any changes on that topic?

                                  No. The lagg(4) driver does not support adding existing laggs as lagg members:

                                  [23.01-RELEASE][root@4100.stevew.lan]/root: ifconfig lagg10 
                                  lagg10: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
                                  	options=800000<>
                                  	ether 00:00:00:00:00:00
                                  	laggproto failover lagghash l2,l3,l4
                                  	groups: lagg
                                  	media: Ethernet autoselect
                                  	status: no carrier
                                  	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                                  [23.01-RELEASE][root@4100.stevew.lan]/root: ifconfig lagg10 laggport lagg0
                                  ifconfig: lagg10 lagg0: SIOCSLAGGPORT: Invalid argument
                                  

                                  How do you have other devices connected to both switches? A single link failover lagg on each?

                                  You might be able to do something by bridging the two laggs and using STP. Hard to really recommend that though!

                                  P 1 Reply Last reply Reply Quote 0
                                  • P
                                    ph0x @stephenw10
                                    last edited by ph0x

                                    @stephenw10 Yeah, I also noticed the error messages while trying to establish the bond on the command line.

                                    All my other devices are Linux based and there it is absolutely not problem to have two LACP bonds in another active-backup bond. This has been working reliably for years. I've been tinkering with OpenWRT in the recent hours, and there it's also possible.

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.