Failover LAGG of LACP LAGGs (Nested LAGG)

jimp

Get switches that support proper stacking and then just use one lagg with four ports across two chassis.

If your switches don't support stacking, then you're out of luck for proper cross-switch failover.

cpfarhood

You can't put a lagg into another lagg.

Why would you even want to do this?

It's quite simple really, I'm attempting to attach to 2 separate physical switches for failover and want more than a single GB of throughput, I do this often with NetApp equipment. Up until Cisco Nexus stacking across chassis (Virtual Port Channel) is not supported.

cpfarhood

@jimp:

Get switches that support proper stacking and then just use one lagg with four ports across two chassis.

If your switches don't support stacking, then you're out of luck for proper cross-switch failover.

Any chance of this being an option in the future? any idea if it's a FreeBSD or pfSense specific limitation?

jimp

A "lagg of laggs" I'm not sure will ever come to be.

If you use only the "failover" mode, it might work, though it's largely unverified.

LACP is known/proven to work in that situation, provided your switches are stackable.

cmb

@cpfarhood:

@jimp:

Get switches that support proper stacking and then just use one lagg with four ports across two chassis.

If your switches don't support stacking, then you're out of luck for proper cross-switch failover.

Any chance of this being an option in the future? any idea if it's a FreeBSD or pfSense specific limitation?

Neither. It's a limitation of your switches in that case. One that's impossible to work around without your switches being involved. Ethernet bonding in general can't be nested.

cthomas

I'll point out that even Netgear ProSafe switches in a stack can support a LAG distributed across two or more switches in the stack.

ph0x

Any changes on that topic?

"Why would you even want to do this?" Because I have a 10 GbE switch (with an lacp bond) and a 1 GbE switch (with an lacp bond) which should work as backup if the fast one fails. This has been working reliably under Linux, which is why I was surprised to find out that FreeBSD doesn't seem to support it.

And yes, I know, that this is an old thread, but I signed up exactly for this issue. ;-)

michmoor

@ph0x I have never seen this setup used in an enterprise. Typically there are switches that support stacking OR using MC-LAG. Just an odd design thats being requested.

ph0x

Well, not everyone runs pfSense in an enterprise environment, right?

michmoor

@ph0x of course but not everyone needs nested LAGGs when there are proper solutions out there.

ph0x

Buying a redundant switch only for failover if there already is a failover switch is obviously not deemed a proper solution by everone. There are (or have been) at least two persons in this forum that work it like that, and in different environments (NetApp and Proxmox).

Sad, but that's how it is.

Dobby_

@ph0x

Any changes on that topic?

I personally think that someone was mixing up some different points or was not knowing it better to ask or
point. If I read it again and again and again, I will be
seeing three different things got merged together.
But perhaps I was not really able to understand it
right, so please be patient and excuse it if I am
wrong with it.

LACP LAGG (dynamic LAGG)
You use it to setup a fail save link or as a combined link
to gain more throughput or better a higher throughput.
It is also called bond (Linux term) and transporting vlans
it is called a trunk (Cisco term).

We use it here in some cases and different modes in
the entire company network. We set up;

LAGGs from servers to one switch or more switches
Gaining the throughput and redundancy
LAGGs from switches to switches
Throughput and redundancy
LAGGs from ringed switch stacks to other switches
(ToR or Core Switches) as LAGG or MLAG
Throughput and redundancy
LAGGs from switch stacks members to servers (OSPF)
Throughput gain, short patch and freeing the Core from load on top, if it is to much traffic

We are also using VLANs and this for the management, WiFi APs, IP cameras, VOIP phones, printers, servers,.........
We set up a VLAN1 for the admin and the management
and then the other vlans for all the devices named above.

If all is ready we lay over all vlans, but not the vlan1, another VLAN to separate it from the VLAN1, and
this not here and there, we do it even.

From the router or firewall to the core we use VRRP and from the core to the distribution and access layer we use OSPFv2/3. All is running fine here.

If we talk about network redundancy in normal we talk about money till 25.000 € and if it goes about HA (high
availability) it starts at the amount of 25.000 €.

Using Netgear or Cisco is here not the question in my eyes
and also not if Netgear is serving that point. It is more in wich manner the question was asked. You can link up LAGGs and you may be able to set up vlans over
another vlan with switches, and also in that manner
that the router or firewall must be not involved.

.....I was surprised to find out that FreeBSD doesn't seem to support it.

And on the other side OpenBSD and FreeBSD are able to serve a load balancer over CARP (active/active) where the other operating systems will be not able to do.

And yes, I know, that this is an old thread, but I signed
up exactly for this issue. ;-)

Use another linux based router after the firewall
Use it only on the switch site (from switch to switch)

May be not the best solutions but working. Perhaps their brand new TNSR is able to realize it?

stephenw10

@ph0x said in Failover LAGG of LACP LAGGs (Nested LAGG):

Any changes on that topic?

No. The lagg(4) driver does not support adding existing laggs as lagg members:

[23.01-RELEASE][root@4100.stevew.lan]/root: ifconfig lagg10 
lagg10: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=800000<>
	ether 00:00:00:00:00:00
	laggproto failover lagghash l2,l3,l4
	groups: lagg
	media: Ethernet autoselect
	status: no carrier
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
[23.01-RELEASE][root@4100.stevew.lan]/root: ifconfig lagg10 laggport lagg0
ifconfig: lagg10 lagg0: SIOCSLAGGPORT: Invalid argument

How do you have other devices connected to both switches? A single link failover lagg on each?

You might be able to do something by bridging the two laggs and using STP. Hard to really recommend that though!

ph0x

@stephenw10 Yeah, I also noticed the error messages while trying to establish the bond on the command line.

All my other devices are Linux based and there it is absolutely not problem to have two LACP bonds in another active-backup bond. This has been working reliably for years. I've been tinkering with OpenWRT in the recent hours, and there it's also possible.