LAGG and LACP = firewall inactive



  • Hello

    we`ve figured out that after setting up a LAGG(4 interfaces) with LACP -  the firewall is not working anymore,
    Rules are not matching and in "Firewall Log" we dont see the Interface from which the traffic comes from.
    It is only see "lagg0".

    pfSense 2.1 - current snapshot

    Maybe some of you can confirm that.

    Greetings
    phedaikin



  • Im assuming its plugged into a switch that supports LACP?



  • @kejianshi:

    Im assuming its plugged into a switch that supports LACP?

    Hehe, yes

    Maybe is should explain that our firewall 2 had the same "error"
    It is an HA setup.
    Nearly everything works fine without LAGG (and yes our switches are LACP/Bonding capable)
    but we Need the bandwidth.

    Greetings
    Phedaikin

    Sorry for typos, damn you autocorrect ..



  • You laugh…  But you would be surprised...



  • @kejianshi:

    You laugh…  But you would be surprised...

    Just a little smile, it was a valid question.



  • @phedaikin:

    we`ve figured out that after setting up a LAGG(4 interfaces) with LACP -  the firewall is not working anymore,
    Rules are not matching and in "Firewall Log" we dont see the Interface from which the traffic comes from.
    It is only see "lagg0".

    So do I understand correctly that you see traffic coming from lagg0 rather than from the individual aggregated interfaces? If so, that sounds exactly like what you'd expect with link aggregation.



  • Nope, there are 4 VLANs on this LAGG Interface.

    LAGG0 Hardware Interaces: em0 to em4
    Interface assignments:

    • CLIENTS VLAN 1 on lagg0
    • SYNC VLAN 5 on lagg0
    • MANAGEMENT VLAN 2 on lagg0
    • NODES VLAN 3 on lagg0
    • GUEST VLAN 4 on lagg0
    • OLD VLAN 99 on lagg0
    • DLS on bce1
    • BNT on bce0

    We dont see the Interface Assignments in our Firewalllog.
    there are only "LAGG0".

    Maybe pfsense is not capable with handling multipe VLAN-Interface-assignements on LAGG?
    LAGG-type Failover is workin fine, LAGG-type LACP works as writen in first post

    regards
    phedaikin



  • I cannot keep up with the counting…?

    @phedaikin:

    Nope, there are 4 VLANs on this LAGG Interface.

    LAGG0 Hardware Interaces: em0 to em4
    Interface assignments:

    • CLIENTS VLAN 1 on lagg0
    • SYNC VLAN 5 on lagg0
    • MANAGEMENT VLAN 2 on lagg0
    • NODES VLAN 3 on lagg0
    • GUEST VLAN 4 on lagg0
    • OLD VLAN 99 on lagg0

    em0 to em4 looks like five physical NICs, not four.

    And I count six VLANs on lagg0.

    Whatever. What exactly is happening? Is traffic from the switch actually arriving untagged at lagg0?

    Can you ping clients from pfSense via the lagg0 interface, or via the VLAN interfaces?

    I've heard a rumor that LAGG may not work correctly on some Netgear switch model. Well, I'm not too confident that this rumor is true, but it might be worth checking.



  • erm 5 interfaces yes, sorry :)

    hmm there is an Netgear Switch … maybe ... i`ll check that.
    No it is an HP switch. Netgear hardware is not present anymore sind 2 years …

    regarding the ping - i`ll check that too just to make sure.
    I think this is not an LAGG problem, this is only happen if we do LACP and LAGG together, and only in pfSense 2.1 RCx



  • Hm. It's been quite some time that I did something with LAGG, and that was on 2.0.x.

    I suspect a bug in FreeBSD 8.3. There have been quite a few issues with LACP in FreeBSD 8.3, and I have no idea haw many have been ironed out. For example: http://blog.multiplay.co.uk/2013/03/freebsd-lacp-balancing/

    Another bug from the 8.3 pre-release phase was that bondig four or more em* interfaces would fail. Up to three em* interfaces would work as expected. Don't know if this has been ironed out as well…

    Of course, if it worked under 2.0.x, and it is mission critical, I'd say "ditch 2.1" for the moment (and wait for 2.2, which will be based on FreeBSD10, IIRC, which may or may not have these issues). Unless, of cousre, you're already set on course to IPv6...

    Another option might be to try "load balancing" instead of LACP. This assumes that the bonded links are stable, as there's no failover support there.



  • Before going into assumptions.
    Can you show how your setup is configured?



  • We've switched back to 2.0.3 but traxanos should have the config.xml. I'll ask him to post the config.



  • @Klaws:

    Another option might be to try "load balancing" instead of LACP. This assumes that the bonded links are stable, as there's no failover support there.

    The "loadbalance" setting does provide failover support, but only as long as a failure causes the link to go down (as opposed to just causing packet loss). So for example, LACP would be able to detect the case where there's a media converter between the NIC and the switch and the connection on the far side of that converter fails (but the link between NIC and media converter stays up), while loadbalance would not.



  • @ermal

    we cant post our config for this issue, after switching back to pfSense 2.0.3 we`ve deleted the old config, but we are trying to reproduce this at our testcluster.

    greetings
    phedaikin



  • I setup 2-port LACP LAGs with multiple VLANs on a redundant pair of pfSense firewalls (2.1 RC0 8/12) using a pair of stacked Netgear GS7xxTS series switches yesterday.  I even setup a crossed configuration where the LAGs from the firewalls were spanned across both switches, i.e., fw1 lan1>sw1p1, lan2 > sw2p1, and fw2 lan1>sw1p2, lan2>sw2p2.

    The crossed configuration was cool, except that I had a handful of of connections that were not LAG'd and therefore if switch 1 failed, the firewalls would have failed over anyhow, so it didn't really make sense to cross them.  I ended up with a 2-port lag from fw1 to sw1, and fw2 to sw2.


Log in to reply