LAGG and LACP = firewall inactive

phedaikin

Im assuming its plugged into a switch that supports LACP?

Hehe, yes

Maybe is should explain that our firewall 2 had the same "error"
It is an HA setup.
Nearly everything works fine without LAGG (and yes our switches are LACP/Bonding capable)
but we Need the bandwidth.

Greetings
Phedaikin

Sorry for typos, damn you autocorrect ..

kejianshi

You laugh… But you would be surprised...

phedaikin

@kejianshi:

You laugh… But you would be surprised...

Just a little smile, it was a valid question.

razzfazz

@phedaikin:

we`ve figured out that after setting up a LAGG(4 interfaces) with LACP - the firewall is not working anymore,
Rules are not matching and in "Firewall Log" we dont see the Interface from which the traffic comes from.
It is only see "lagg0".

So do I understand correctly that you see traffic coming from lagg0 rather than from the individual aggregated interfaces? If so, that sounds exactly like what you'd expect with link aggregation.

phedaikin

Nope, there are 4 VLANs on this LAGG Interface.

LAGG0 Hardware Interaces: em0 to em4
Interface assignments:

CLIENTS VLAN 1 on lagg0
SYNC VLAN 5 on lagg0
MANAGEMENT VLAN 2 on lagg0
NODES VLAN 3 on lagg0
GUEST VLAN 4 on lagg0
OLD VLAN 99 on lagg0
DLS on bce1
BNT on bce0

We dont see the Interface Assignments in our Firewalllog.
there are only "LAGG0".

Maybe pfsense is not capable with handling multipe VLAN-Interface-assignements on LAGG?
LAGG-type Failover is workin fine, LAGG-type LACP works as writen in first post

regards
phedaikin

Klaws

I cannot keep up with the counting…?

@phedaikin:

Nope, there are 4 VLANs on this LAGG Interface.

LAGG0 Hardware Interaces: em0 to em4
Interface assignments:

CLIENTS VLAN 1 on lagg0

SYNC VLAN 5 on lagg0

MANAGEMENT VLAN 2 on lagg0

NODES VLAN 3 on lagg0

GUEST VLAN 4 on lagg0

OLD VLAN 99 on lagg0

em0 to em4 looks like five physical NICs, not four.

And I count six VLANs on lagg0.

Whatever. What exactly is happening? Is traffic from the switch actually arriving untagged at lagg0?

Can you ping clients from pfSense via the lagg0 interface, or via the VLAN interfaces?

I've heard a rumor that LAGG may not work correctly on some Netgear switch model. Well, I'm not too confident that this rumor is true, but it might be worth checking.

phedaikin

erm 5 interfaces yes, sorry :)

~~hmm there is an Netgear Switch … maybe ... i`ll check that.~~
No it is an HP switch. Netgear hardware is not present anymore sind 2 years …

regarding the ping - i`ll check that too just to make sure.
I think this is not an LAGG problem, this is only happen if we do LACP and LAGG together, and only in pfSense 2.1 RCx

Klaws

Hm. It's been quite some time that I did something with LAGG, and that was on 2.0.x.

I suspect a bug in FreeBSD 8.3. There have been quite a few issues with LACP in FreeBSD 8.3, and I have no idea haw many have been ironed out. For example: http://blog.multiplay.co.uk/2013/03/freebsd-lacp-balancing/

Another bug from the 8.3 pre-release phase was that bondig four or more em* interfaces would fail. Up to three em* interfaces would work as expected. Don't know if this has been ironed out as well…

Of course, if it worked under 2.0.x, and it is mission critical, I'd say "ditch 2.1" for the moment (and wait for 2.2, which will be based on FreeBSD10, IIRC, which may or may not have these issues). Unless, of cousre, you're already set on course to IPv6...

Another option might be to try "load balancing" instead of LACP. This assumes that the bonded links are stable, as there's no failover support there.

eri--

Before going into assumptions.
Can you show how your setup is configured?

phedaikin

We've switched back to 2.0.3 but traxanos should have the config.xml. I'll ask him to post the config.

razzfazz

@Klaws:

Another option might be to try "load balancing" instead of LACP. This assumes that the bonded links are stable, as there's no failover support there.

The "loadbalance" setting does provide failover support, but only as long as a failure causes the link to go down (as opposed to just causing packet loss). So for example, LACP would be able to detect the case where there's a media converter between the NIC and the switch and the connection on the far side of that converter fails (but the link between NIC and media converter stays up), while loadbalance would not.

phedaikin

@ermal

we cant post our config for this issue, after switching back to pfSense 2.0.3 we`ve deleted the old config, but we are trying to reproduce this at our testcluster.

greetings
phedaikin

cthomas

I setup 2-port LACP LAGs with multiple VLANs on a redundant pair of pfSense firewalls (2.1 RC0 8/12) using a pair of stacked Netgear GS7xxTS series switches yesterday. I even setup a crossed configuration where the LAGs from the firewalls were spanned across both switches, i.e., fw1 lan1>sw1p1, lan2 > sw2p1, and fw2 lan1>sw1p2, lan2>sw2p2.

The crossed configuration was cool, except that I had a handful of of connections that were not LAG'd and therefore if switch 1 failed, the firewalls would have failed over anyhow, so it didn't really make sense to cross them. I ended up with a 2-port lag from fw1 to sw1, and fw2 to sw2.