Unable to get VLAN working with LAGG in pfSesne 2.2.2



  • After dealing with LAGG issues during 2.2 snapshot, I now face complete failure of LAGG and VLANs. I cannot get VLAN to work with a Cisco LACP compatible switch. Whenever I enable trunk on the LAGG port to allow mutiple VLANs, all traffic fail. There are no errors in the log. All I get is failed packets because none of the hosts are reachable.

    Here is what I tried:
    1. pfSense LACP+VLAN (vlanhwtag must be disabled on the em interfaces!!) to Trunk LACP (active) on switch - SUCCESS
    2. pfSense LACP to Access LACP (passive) on switch  - FAILED
    3. pfSense LACP to Access LACP (active) on switch  - SUCCESS
    3. pfSense ROUNDROBIN to Access Etherchannel on switch - SUCCESS
    4. pfSense ROUNDROBIN+VLAN to Trunk Etherchannel on switch - FAILED (I did not try this will vlanhwtag disabled)
    5. pfSense em0 to Access Port - SUCCESS

    What I did not try was pfSense em0 to Trunk Port on switch.

    System info:
    pfSense x86_64 with dual port Intel NIC on LAN (em0, em1) connected to cisco 2970g.

    Would someone with a successful VLAN with LAGG configuration share their setup?

    My symptoms are exactly as describe here. Isn't this fix in FreeBSD10?



  • After reading through the issue on the freebsd forum (linked above), I was able to get vlan working on LACP lagg interface. To get this working with Intel em interfaces and Cisco switch:
    1. vlanhwtag MUST be disabled on the em interfaces. '#' is ther interface number. (How do i add this the pfSense config???)

    ifconfig em# -vlanhwtag
    

    2. Cisco switch MUST have LACP in active mode.

    Even with LACP enabled on the switch pfSense still does not aggregate the NICs and the Port-channel shows as "Ag-Not-Inuse", which led to much of my frustration. If I had to guess, it seem the NICs are added in failover mode as traffic only flows on one NIC at a time. Pulling the cable activates the other NIC.

    Port-channel1   (Primary aggregator)
    
    Age of the Port-channel   = 0d:00h:17m:34s
    Logical slot/port   = 2/1          Number of ports = 0
    HotStandBy port = null 
    Port state          = Port-channel Ag-Not-Inuse 
    Protocol            =   LACP
    Port security       = Disabled
    

    Also of note, the switch and pfSense takes some time to negotiate link between the ports and NICs, over of 2 mins in my case.

    So one more question, how do I add the below line to pfSense's startup config?

    ifconfig_em0="-vlanhwtag up"
    


  • Hmm.. so I was thinking about trying to LAGG the LAN and OPT1 ports of my pfSense SG-4860 into my NetGear ProSafe switch as well as an OPT2/OPT3 LAGG for a DMZed Synology NAS.

    If I read right, are you saying the pfSense LAGG can't do active/active bandwidth aggregation, only active/passive?



  • @tux_dude:

    1. vlanhwtag MUST be disabled on the em interfaces. '#' is ther interface number. (How do i add this the pfSense config???)

    This isn't true for any Intel NICs in any recent version I've seen. My home network's run LACP active/active with VLANs for many years on em NICs and more recently igb. Maybe you have ones that need a firmware update to fix issues with the NICs as they shipped, which comes up from time to time, though unusual.

    @tux_dude:

    2. Cisco switch MUST have LACP in active mode.

    This was probably your only problem initially. Though it sounds like you still have a problem with your switch, given the Ag-Not-Inuse and the fact it takes 2 minutes. LACP is instantaneous where the switch replies correctly. What does the output of 'ifconfig' from the firewall, and 'show ether detail' on the switch show?



  • @deinspanjer:

    If I read right, are you saying the pfSense LAGG can't do active/active bandwidth aggregation, only active/passive?

    No. As long as your switch supports active LACP, it'll do active/active. Or you can make it do passive LACP instead. He seems to have a problem somewhere that results in his being active/passive, but as long as you have a properly functioning and configured switch, active/active is no problem.



  • Haven't touch anything since I got it working. About the upgrade to 2.2.3 so here's the requested info.

    @cmb:

    What does the output of 'ifconfig' from the firewall, and 'show ether detail' on the switch show?

    ifconfig

    
    em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=4208b <rxcsum,txcsum,vlan_mtu,vlan_hwcsum,wol_magic,vlan_hwtso>ether 
            nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
    em1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=4008b <rxcsum,txcsum,vlan_mtu,vlan_hwcsum,vlan_hwtso>ether 
            nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
    bge0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
            options=8009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate>ether 
            inet6 %bge0 prefixlen 64 scopeid 0x3 
            inet  netmask 0xfffffe00 broadcast 255.255.255.255 
            nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect (1000baseT <full-duplex,master>)
            status: active
    pflog0: flags=100 <promisc>metric 0 mtu 33144
    pfsync0: flags=0<> metric 0 mtu 1500
            syncpeer: 224.0.0.240 maxupd: 128 defer: on
            syncok: 1
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
            options=600003 <rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6>inet 127.0.0.1 netmask 0xff000000 
            inet6 ::1 prefixlen 128 
            inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6 
            nd6 options=21 <performnud,auto_linklocal>enc0: flags=0<> metric 0 mtu 1536
            nd6 options=21 <performnud,auto_linklocal>lagg0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=4008b <rxcsum,txcsum,vlan_mtu,vlan_hwcsum,vlan_hwtso>ether 
            inet  netmask 0xffffff00 broadcast  
            inet6 fe80::1:1%lagg0 prefixlen 64 duplicated scopeid 0x8 
            inet6  prefixlen 64 duplicated 
            nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect
            status: active
            laggproto lacp lagghash l2,l3,l4
            laggport: em1 flags=1c <active,collecting,distributing>laggport: em0 flags=1c <active,collecting,distributing>lagg0_vlan5: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=3 <rxcsum,txcsum>ether 
            inet6 %lagg0_vlan5 prefixlen 64 scopeid 0x9 
            inet netmask 0xffffff00 broadcast  
            nd6 options=21 <performnud,auto_linklocal>media: Ethernet autoselect
            status: active
            vlan: 5 vlanpcp: 0 parent interface: lagg0
    ovpns1: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
            options=80000 <linkstate>inet6 %ovpns1 prefixlen 64 scopeid 0xa 
            inet  -->  netmask 0xffffffff 
            nd6 options=21 <performnud,auto_linklocal>Opened by PID 88701
    ovpns2: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
            options=80000 <linkstate>inet6 %ovpns2 prefixlen 64 scopeid 0xb 
            inet  -->  netmask 0xffffffff 
            nd6 options=21 <performnud,auto_linklocal>Opened by PID 92106
    wan_stf: flags=4001 <up,link2>metric 0 mtu 1280
            inet6  prefixlen 32 
            nd6 options=1 <performnud>v4net  -> tv4br</performnud></up,link2></performnud,auto_linklocal></linkstate></up,pointopoint,running,multicast></performnud,auto_linklocal></linkstate></up,pointopoint,running,multicast></performnud,auto_linklocal></rxcsum,txcsum></up,broadcast,running,simplex,multicast></active,collecting,distributing></active,collecting,distributing></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwcsum,vlan_hwtso></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></performnud,auto_linklocal></rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6></up,loopback,running,multicast></promisc></full-duplex,master></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,linkstate></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwcsum,vlan_hwtso></up,broadcast,running,simplex,multicast></full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwcsum,wol_magic,vlan_hwtso></up,broadcast,running,simplex,multicast> 
    

    show ether detail

    
    Channel-group listing: 
                    ----------------------
    
    Group: 1 
    ----------
    Group state = L2 
    Ports: 2   Maxports = 16
    Port-channels: 1 Max Port-channels = 16
    Protocol:   LACP
    Minimum Links: 0
                    Ports in the group:
                    -------------------
    Port: Gi0/23
    ------------
    
    Port state    = Up Sngl-port-Bndl Mstr Not-in-Bndl 
    Channel group = 1           Mode = Active          Gcchange = -
    Port-channel  = null        GC   =   -             Pseudo port-channel = Po1
    Port index    = 0           Load = 0x00            Protocol =   LACP
    
    Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.
            A - Device is in active mode.        P - Device is in passive mode.
    
    Local information:
                                LACP port     Admin     Oper    Port        Port
    Port      Flags   State     Priority      Key       Key     Number      State
    Gi0/23    SA      indep     32768         0x1       0x1     0x118       0x7D  
    
    Age of the port in the current state: 48d:11h:41m:13s
    
    Port: Gi0/24
    ------------
    
    Port state    = Up Sngl-port-Bndl Mstr Not-in-Bndl 
    Channel group = 1           Mode = Active          Gcchange = -
    Port-channel  = null        GC   =   -             Pseudo port-channel = Po1
    Port index    = 0           Load = 0x00            Protocol =   LACP
    
    Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.
            A - Device is in active mode.        P - Device is in passive mode.
    
    Local information:
                                LACP port     Admin     Oper    Port        Port
    Port      Flags   State     Priority      Key       Key     Number      State
    Gi0/24    SA      indep     32768         0x1       0x1     0x119       0x7D  
    
    Age of the port in the current state: 48d:11h:41m:14s
    
                    Port-channels in the group: 
                    ---------------------------
    
    Port-channel: Po1    (Primary Aggregator)
    
    ------------
    
    Age of the Port-channel   = 48d:11h:52m:35s
    Logical slot/port   = 2/1          Number of ports = 0
    HotStandBy port = null 
    Port state          = Port-channel Ag-Not-Inuse 
    Protocol            =   LACP
    Port security       = Disabled
    
    


  • @tux_dude
    Did you solve it out now? Did you brought up the LAGs and the VLANs straight working smooth?


Log in to reply