Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    LACP not working

    Scheduled Pinned Locked Moved General pfSense Questions
    113 Posts 2 Posters 24.7k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Seems more like you have to omit the priority in the switch/port to have it match what looks like the default value pfSense is sending. But it's not something I've had to do before.

      Steve

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        In fact there is no way to set a priority in pfSense:

        https://www.freebsd.org/cgi/man.cgi?lagg(4)#BUGS

             There is no way to	configure LACP administrative variables, including
     system and	port priorities.  The current implementation always performs
     active-mode LACP and uses 0x8000 as system	and port priorities.

        So that must be in the switch config somewhere. It looks like it's set to 0 in the output you have posted but there must be somewhere else it's pulling in that value.

        Steve

        C 1 Reply Last reply Reply Quote 0
        • C
          cyberbot @stephenw10
          last edited by

          @stephenw10 said in LACP not working:

          In fact there is no way to set a priority in pfSense:

          https://www.freebsd.org/cgi/man.cgi?lagg(4)#BUGS

               There is no way to	configure LACP administrative variables, including
     system and	port priorities.  The current implementation always performs
     active-mode LACP and uses 0x8000 as system	and port priorities.

          So that must be in the switch config somewhere. It looks like it's set to 0 in the output you have posted but there must be somewhere else it's pulling in that value.

          Steve

          Hi Steve,
          today we had a call with pfsense engineer, however the issue appear the switch still.

          the pfsense is sending LACP and binding the ports nicely.
          but the switch is still blocking the ports and we cannot see any log neither on the switch or the firewall.
          the issue still remains unresolved unfortunately.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Yes, it looks like a switch issue to me too.

            Have you been able to test a lagg to a single switch?

            Check the full switch config from both switches. It must be pulling in the port priority from somewhere.

            Steve

            C 1 Reply Last reply Reply Quote 0
            • C
              cyberbot @stephenw10
              last edited by

              @stephenw10 said in LACP not working:

              Yes, it looks like a switch issue to me too.

              Have you been able to test a lagg to a single switch?

              Check the full switch config from both switches. It must be pulling in the port priority from somewhere.

              Steve

              Hi Steve,
              the priority we were able to fix it, with specifying the priority and the error of the flapping were gone, but the switch was still blocking the interfaces LAG

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                No longer logging the mismatch in the switch?

                C 1 Reply Last reply Reply Quote 0
                • C
                  cyberbot @stephenw10
                  last edited by

                  @stephenw10 said in LACP not working:

                  No longer logging the mismatch in the switch?

                  no, not on the switch and no flapping on the firewall,
                  but when we do show lag it still shows blocked LAG.

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Can we see the current output from both sides?

                    C 1 Reply Last reply Reply Quote 0
                    • C
                      cyberbot @stephenw10
                      last edited by

                      @stephenw10

                      yes of course only there is no log now, nothing happening at all.

                      1 Reply Last reply Reply Quote 0
                      • C
                        cyberbot
                        last edited by

                        when I do show log on the switch nothing really happens about the LACP or errors neither on the pfsense

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by stephenw10

                          Ok but can we see ifconfig -vvv lagg0 from pfSense and show lag from the switch. The complete output from each.

                          Steve

                          C 1 Reply Last reply Reply Quote 0
                          • C
                            cyberbot @stephenw10
                            last edited by cyberbot

                            @stephenw10 said in LACP not working:

                            Ok but can we see ifconfig -vvv lagg0 from pfSense and show lag from the switch. The complete output from each.

                            Steve

                            Hi Steve,
                            see below, the engineer still confirm the switch is the cause.

                            lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=800008<VLAN_MTU>
        ether e8:39:35:11:fa:ab
        inet6 fe80::ea39:35ff:fe11:faab%lagg0 prefixlen 64 scopeid 0xb
        inet 192.168.73.1 netmask 0xffffff00 broadcast 192.168.73.255
        laggproto lacp lagghash l2,l3,l4
        lagg options:
                flags=90<LACP_STRICT>
                flowid_shift: 16
        lagg statistics:
                active ports: 2
                flapping: 64
        lag id: [(8000,E8-39-35-11-FA-AB,016B,0000,0000),
                 (0001,60-9C-9F-4B-80-8C,4E21,0000,0000)]
        laggport: em2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3f<ACTIVITY,TIMEOUT,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
                [(8000,E8-39-35-11-FA-AB,016B,8000,0003),
                 (0001,60-9C-9F-4B-80-8C,4E21,0001,0002)]
        laggport: em3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3f<ACTIVITY,TIMEOUT,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
                [(8000,E8-39-35-11-FA-AB,016B,8000,0004),
                 (0001,60-9C-9F-4B-80-8C,4E21,0001,0102)]
        groups: lagg
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

                            === LAG "LAN" ID 1 (dynamic Deployed) ===
LAG Configuration:
   Ports:         e 1/1/2 e 2/1/2
   Port Count:    2
   Primary Port:  1/1/2
   Trunk Type:    hash-based
   LACP Key:      20001
Deployment: HW Trunk ID 1
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/1/2      Up      Blocked Full 1G    1     Yes N/A  0   609c.9f4b.105d  LAN1
2/1/2      Up      Blocked Full 1G    1     Yes N/A  0   609c.9f4b.105d LAN2

Port       [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/1/2           1        1   20001   Yes   S   Agg  Syn  Col  Dis  Def  No   Err
2/1/2           1        1   20001   Yes   S   Agg  Syn  Col  Dis  Def  No   Err
                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              The timeouts are back to short on the switch but do also look like they're set short in pfSense.

                              There are two additional sections in the 'show lag' output. The system lag info at the top and the partner info at the bottom. Both of which might be useful here.
                              For example:

                              SSH@ICX6450-24P Switch>show lag
Total number of LAGs:          1
Total number of deployed LAGs: 1
Total number of trunks created:1 (123 available)
LACP System Priority / ID:     1 / 609c.9f54.14f2
LACP Long timeout:             90, default: 90
LACP Short timeout:            3, default: 3

=== LAG "lacp1" ID 2047 (dynamic Deployed) ===
LAG Configuration:
   Ports:         e 1/2/1 e 1/2/3 
   Port Count:    2
   Primary Port:  1/2/1
   Trunk Type:    hash-based
   LACP Key:      22047
   LACP Timeout:  long
Deployment: HW Trunk ID 1
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/2/1      Up      Forward Full 10G   2047  No  1    0   609c.9f54.150b                 
1/2/3      Up      Forward Full 10G   2047  No  1    0   609c.9f54.150b                 

Port       [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/2/1           1        1   22047   Yes   L   Agg  Syn  Col  Dis  No   No   Ope
1/2/3           1        1   22047   Yes   L   Agg  Syn  Col  Dis  No   No   Ope
                                                                  

 Partner Info and PDU Statistics 
Port          Partner         Partner     LACP      LACP     
             System ID         Key     Rx Count  Tx Count  
1/2/1    32768-00e0.ed86.a68c      690     5638     13113
1/2/3    32768-00e0.ed86.a68c      690     5625     13138

                              [2.4.5-RELEASE][admin@7100.stevew.lan]/root: ifconfig -vvv lagg0
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=500b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO>
	ether 00:e0:ed:86:a6:8c
	inet6 fe80::2e0:edff:fe86:a68c%lagg0 prefixlen 64 scopeid 0x15
	inet 172.21.16.206 netmask 0xffffff00 broadcast 172.21.16.255
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	status: active
	groups: lagg
	laggproto lacp lagghash l2,l3,l4
	lagg options:
		flags=10<LACP_STRICT>
		flowid_shift: 16
	lagg statistics:
		active ports: 2
		flapping: 0
	lag id: [(8000,00-E0-ED-86-A6-8C,02B2,0000,0000),
		 (0001,60-9C-9F-54-14-F2,561F,0000,0000)]
	laggport: ixl0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
		[(8000,00-E0-ED-86-A6-8C,02B2,8000,0001),
		 (0001,60-9C-9F-54-14-F2,561F,0001,0041)]
	laggport: ixl1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
		[(8000,00-E0-ED-86-A6-8C,02B2,8000,0002),
		 (0001,60-9C-9F-54-14-F2,561F,0001,0043)]
                              1 Reply Last reply Reply Quote 0
                              • C
                                cyberbot
                                last edited by

                                Hi Steve,

                                do you suggest to reconfigure the set up to long instead of short?
                                i can do that is not a issue at all, but the switch still blocking it

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Well it should work with either as long as both sides are set the same. The switch ports are not showing Exp (expired) which I would expect if pfSense was still using Long timeouts.

                                  I know Long works though.

                                  The port operational state (Ope) is still showing Err:

                                  Err: If there is a peer information mismatch, then that particular port is moved to the Error disable state (Err).

                                  https://docs.commscope.com/bundle/fastiron-08095-commandref/page/GUID-06AFF73D-6957-44A5-AF25-3527B2BE1580.html

                                  I would have expected that to be logged still.

                                  Can we see the full 'show lag' output?

                                  Steve

                                  C 1 Reply Last reply Reply Quote 0
                                  • C
                                    cyberbot @stephenw10
                                    last edited by cyberbot

                                    @stephenw10 said in LACP not working:

                                    Well it should work with either as long as both sides are set the same. The switch ports are not showing Exp (expired) which I would expect if pfSense was still using Long timeouts.

                                    I know Long works though.

                                    The port operational state (Ope) is still showing Err:

                                    Err: If there is a peer information mismatch, then that particular port is moved to the Error disable state (Err).

                                    https://docs.commscope.com/bundle/fastiron-08095-commandref/page/GUID-06AFF73D-6957-44A5-AF25-3527B2BE1580.html

                                    I would have expected that to be logged still.

                                    Can we see the full 'show lag' output?

                                    Steve

                                    on the switch side I don't see any logs showing anything about the LACP but only about the when I logging using ssh ect...
                                    I have change it to long but still shows blocked.

                                    === LAG "LAN" ID 1 (dynamic Deployed) ===
LAG Configuration:
   Ports:         e 1/1/2 e 2/1/2
   Port Count:    2
   Primary Port:  1/1/2
   Trunk Type:    hash-based
   LACP Key:      20001
   LACP Timeout:  long
Deployment: HW Trunk ID 1
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/1/2      Up      Blocked Full 1G    1     Yes N/A  0    LAN1
2/1/2      Up      Blocked Full 1G    1     Yes N/A  0    LAN2

Port       [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/1/2           1        1   20001   Yes   L   Agg  No   No   No   No   No   Ina
2/1/2           1        1   20001   Yes   L   Agg  Syn  Col  Dis  Def  No   Err
                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      We need to see the full output, including the partner info.

                                      The first port there is showing it's not seeing ant LACP packets from the other side.

                                      C 1 Reply Last reply Reply Quote 0
                                      • C
                                        cyberbot @stephenw10
                                        last edited by cyberbot

                                        @stephenw10 said in LACP not working:

                                        We need to see the full output, including the partner info.

                                        The first port there is showing it's not seeing ant LACP packets from the other side.

                                        Hi Steve

                                        What full output are you referring to ? On the pfsense or switch ?
                                        On the switch side it shows only information about who is connected from the ssh and what time nothing else .

                                        1 Reply Last reply Reply Quote 0
                                        • stephenw10S
                                          stephenw10 Netgate Administrator
                                          last edited by

                                          The complete output from show lag on the switch contains the Partner info and PDU stats at the end which show what is connected and on which port. It also shows the system wide lag parameters at the top on my switch. It would be good to compare those.
                                          Your switches are different to mine though, the output is very similar but not identical.

                                          You posted a more complete output here: https://forum.netgate.com/post/947906

                                          Steve

                                          C 1 Reply Last reply Reply Quote 0
                                          • C
                                            cyberbot @stephenw10
                                            last edited by cyberbot

                                            @stephenw10 said in LACP not working:

                                            The complete output from show lag on the switch contains the Partner info and PDU stats at the end which show what is connected and on which port. It also shows the system wide lag parameters at the top on my switch. It would be good to compare those.
                                            Your switches are different to mine though, the output is very similar but not identical.

                                            You posted a more complete output here: https://forum.netgate.com/post/947906

                                            Steve

                                            Hi Steve,

                                            Are you referring to these?

                                            === LAG "LAN" ID 1 (dynamic Deployed) ===
LAG Configuration:
   Ports:         e 1/1/2 e 2/1/2
   Port Count:    2
   Primary Port:  1/1/2
   Trunk Type:    hash-based
   LACP Key:      20001
   LACP Timeout:  long
Deployment: HW Trunk ID 1
Port       Link    State   Dupl Speed Trunk Tag Pvid Pri MAC             Name
1/1/2      Up      Blocked Full 1G    1     Yes N/A  0   609c.9f4b.606d  LAN1
2/1/2      Up      Blocked Full 1G    1     Yes N/A  0   609c.9f4b.606d  LAN2

Port       [Sys P] [Port P] [ Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/1/2           1        1   20001   Yes   L   Agg  Syn  Col  Dis  Def  No   Err
2/1/2           1        1   20001   Yes   L   Agg  Syn  Col  Dis  Def  No   Err


 Partner Info and PDU Statistics
Port          Partner         Partner     LACP      LACP
             System ID         Key     Rx Count  Tx Count
1/1/2    32768-e839.3511.faab      363        0    237550
2/1/2    32768-e839.3511.faab      363        0    237550

                                            lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=800008<VLAN_MTU>
        ether e8:39:35:11:fa:ab
        inet6 fe80::ea39:35ff:fe11:faab%lagg0 prefixlen 64 scopeid 0xb
        inet 192.168.73.1 netmask 0xffffff00 broadcast 192.168.73.255
        laggproto lacp lagghash l2,l3,l4
        lagg options:
                flags=10<LACP_STRICT>
                flowid_shift: 16
        lagg statistics:
                active ports: 2
                flapping: 291
        lag id: [(8000,E8-39-35-11-FA-AB,016B,0000,0000),
                 (0001,60-9C-9F-4B-80-8C,4E21,0000,0000)]
        laggport: em2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
                [(8000,E8-39-35-11-FA-AB,016B,8000,0003),
                 (0001,60-9C-9F-4B-80-8C,4E21,0001,0002)]
        laggport: em3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING>
                [(8000,E8-39-35-11-FA-AB,016B,8000,0004),
                 (0001,60-9C-9F-4B-80-8C,4E21,0001,0102)]
        groups: lagg
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                                            stephenw10S 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.