Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Failover LAGG of LACP LAGGs (Nested LAGG)

    Scheduled Pinned Locked Moved General pfSense Questions
    16 Posts 9 Posters 2.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jimpJ
      jimp Rebel Alliance Developer Netgate
      last edited by

      A "lagg of laggs" I'm not sure will ever come to be.

      If you use only the "failover" mode, it might work, though it's largely unverified.

      LACP is known/proven to work in that situation, provided your switches are stackable.

      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

      Need help fast? Netgate Global Support!

      Do not Chat/PM for help!

      1 Reply Last reply Reply Quote 1
      • C
        cmb
        last edited by

        @cpfarhood:

        @jimp:

        Get switches that support proper stacking and then just use one lagg with four ports across two chassis.

        If your switches don't support stacking, then you're out of luck for proper cross-switch failover.

        Any chance of this being an option in the future? any idea if it's a FreeBSD or pfSense specific limitation?

        Neither. It's a limitation of your switches in that case. One that's impossible to work around without your switches being involved. Ethernet bonding in general can't be nested.

        1 Reply Last reply Reply Quote 0
        • C
          cthomas
          last edited by

          I'll point out that even Netgear ProSafe switches in a stack can support a LAG distributed across two or more switches in the stack.

          1 Reply Last reply Reply Quote 0
          • P
            ph0x
            last edited by

            Any changes on that topic?

            "Why would you even want to do this?" Because I have a 10 GbE switch (with an lacp bond) and a 1 GbE switch (with an lacp bond) which should work as backup if the fast one fails. This has been working reliably under Linux, which is why I was surprised to find out that FreeBSD doesn't seem to support it.

            And yes, I know, that this is an old thread, but I signed up exactly for this issue. ;-)

            M stephenw10S 2 Replies Last reply Reply Quote 0
            • M
              michmoor LAYER 8 Rebel Alliance @ph0x
              last edited by

              @ph0x I have never seen this setup used in an enterprise. Typically there are switches that support stacking OR using MC-LAG. Just an odd design thats being requested.

              Firewall: NetGate,Palo Alto-VM,Juniper SRX
              Routing: Juniper, Arista, Cisco
              Switching: Juniper, Arista, Cisco
              Wireless: Unifi, Aruba IAP
              JNCIP,CCNP Enterprise

              P 1 Reply Last reply Reply Quote 1
              • P
                ph0x @michmoor
                last edited by

                Well, not everyone runs pfSense in an enterprise environment, right?

                M 1 Reply Last reply Reply Quote 0
                • M
                  michmoor LAYER 8 Rebel Alliance @ph0x
                  last edited by

                  @ph0x of course but not everyone needs nested LAGGs when there are proper solutions out there.

                  Firewall: NetGate,Palo Alto-VM,Juniper SRX
                  Routing: Juniper, Arista, Cisco
                  Switching: Juniper, Arista, Cisco
                  Wireless: Unifi, Aruba IAP
                  JNCIP,CCNP Enterprise

                  P 1 Reply Last reply Reply Quote 0
                  • P
                    ph0x @michmoor
                    last edited by

                    Buying a redundant switch only for failover if there already is a failover switch is obviously not deemed a proper solution by everone. There are (or have been) at least two persons in this forum that work it like that, and in different environments (NetApp and Proxmox).

                    Sad, but that's how it is.

                    Dobby_D 1 Reply Last reply Reply Quote 0
                    • Dobby_D
                      Dobby_ @ph0x
                      last edited by

                      @ph0x

                      Any changes on that topic?

                      I personally think that someone was mixing up some different points or was not knowing it better to ask or
                      point. If I read it again and again and again, I will be
                      seeing three different things got merged together.
                      But perhaps I was not really able to understand it
                      right, so please be patient and excuse it if I am
                      wrong with it.

                      LACP LAGG (dynamic LAGG)
                      You use it to setup a fail save link or as a combined link
                      to gain more throughput or better a higher throughput.
                      It is also called bond (Linux term) and transporting vlans
                      it is called a trunk (Cisco term).

                      We use it here in some cases and different modes in
                      the entire company network. We set up;

                      • LAGGs from servers to one switch or more switches
                        Gaining the throughput and redundancy
                      • LAGGs from switches to switches
                        Throughput and redundancy
                      • LAGGs from ringed switch stacks to other switches
                        (ToR or Core Switches) as LAGG or MLAG
                        Throughput and redundancy
                      • LAGGs from switch stacks members to servers (OSPF)
                        Throughput gain, short patch and freeing the Core from load on top, if it is to much traffic

                      We are also using VLANs and this for the management, WiFi APs, IP cameras, VOIP phones, printers, servers,.........
                      We set up a VLAN1 for the admin and the management
                      and then the other vlans for all the devices named above.

                      If all is ready we lay over all vlans, but not the vlan1, another VLAN to separate it from the VLAN1, and
                      this not here and there, we do it even.

                      From the router or firewall to the core we use VRRP and from the core to the distribution and access layer we use OSPFv2/3. All is running fine here.

                      If we talk about network redundancy in normal we talk about money till 25.000 € and if it goes about HA (high
                      availability) it starts at the amount of 25.000 €.

                      Using Netgear or Cisco is here not the question in my eyes
                      and also not if Netgear is serving that point. It is more in wich manner the question was asked. You can link up LAGGs and you may be able to set up vlans over
                      another vlan with switches, and also in that manner
                      that the router or firewall must be not involved.

                      .....I was surprised to find out that FreeBSD doesn't seem to support it.

                      And on the other side OpenBSD and FreeBSD are able to serve a load balancer over CARP (active/active) where the other operating systems will be not able to do.

                      And yes, I know, that this is an old thread, but I signed
                      up exactly for this issue. ;-)

                      • Use another linux based router after the firewall
                      • Use it only on the switch site (from switch to switch)

                      May be not the best solutions but working. Perhaps their brand new TNSR is able to realize it?

                      #~. @Dobby

                      Turris Omnia - 4 Ports - 2 GB RAM / TurrisOS 7 Release (Btrfs)
                      PC Engines APU4D4 - 4 Ports - 4 GB RAM / pfSense CE 2.7.2 Release (ZFS)
                      PC Engines APU6B4 - 4 Ports - 4 GB RAM / pfSense+ (Plus) 24.03_1 Release (ZFS)

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator @ph0x
                        last edited by stephenw10

                        @ph0x said in Failover LAGG of LACP LAGGs (Nested LAGG):

                        Any changes on that topic?

                        No. The lagg(4) driver does not support adding existing laggs as lagg members:

                        [23.01-RELEASE][root@4100.stevew.lan]/root: ifconfig lagg10 
                        lagg10: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
                        	options=800000<>
                        	ether 00:00:00:00:00:00
                        	laggproto failover lagghash l2,l3,l4
                        	groups: lagg
                        	media: Ethernet autoselect
                        	status: no carrier
                        	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                        [23.01-RELEASE][root@4100.stevew.lan]/root: ifconfig lagg10 laggport lagg0
                        ifconfig: lagg10 lagg0: SIOCSLAGGPORT: Invalid argument
                        

                        How do you have other devices connected to both switches? A single link failover lagg on each?

                        You might be able to do something by bridging the two laggs and using STP. Hard to really recommend that though!

                        P 1 Reply Last reply Reply Quote 0
                        • P
                          ph0x @stephenw10
                          last edited by ph0x

                          @stephenw10 Yeah, I also noticed the error messages while trying to establish the bond on the command line.

                          All my other devices are Linux based and there it is absolutely not problem to have two LACP bonds in another active-backup bond. This has been working reliably for years. I've been tinkering with OpenWRT in the recent hours, and there it's also possible.

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.