Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    LAN with quad NIC configured with LAGG-LACP and VLANs

    Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
    15 Posts 6 Posters 3.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • CrossEyeC
      CrossEye
      last edited by

      Can someone verify that after upgrading their pfSense from 2.1.x to 2.2 that their LAGG interface is passing traffic / functioning?

      I have a pfSense box with 5 NICs in it, one is for WAN and the other 4 are my LAN interface in a LAGG with the protocol set to LACP. I also have about ten VLANs running over this LAGG with corresponding virtual interfaces configured in pfSense.

      Everything has been running great in 2.1.x but once I initiated the upgrade from the WebGUI, it completed successfully and then would not pass any traffic over the LAGG. So I thought there might be a glitch with my config, so I started over with a fresh install of 2.2 and did a very basic config as a test. But whenever I configured the quad NIC in a LAGG with LACP… all traffic stop passing. I also did a couple Tcpdumps and saw nothing except pfSense trying to ARP but nothing else.

      Right now I am just trying to see if the problem is isolated to only me or if it is a wider problem.

      Thanks in advance all and take care,

      1 Reply Last reply Reply Quote 0
      • R
        Riesch
        last edited by

        Hi CrossEye,

        I am having the same problem on a Supermicro board with a C2758 chip.
        After the update i got a geom/intel raid error so used a diffirent (single disk). The problem you describe is what i got after the fresh install.

        Hoping for a quick fix for both problems, sold so many leftover gear i have nothing to replace my pfSense box now :'(

        1 Reply Last reply Reply Quote 0
        • J
          jumpace07
          last edited by

          This is interesting.. I have 2 NIC's in a LAGG group with 2.1.5 and things work fine, upgrade to 2.2 and things go pear shaped.  I just posted my own post regarding issues and this might be the same as nothing seems to work over the link.  I didn't suspect the lagg, but something else but after this it very well could be the lagg not functioning properly as all my vlans are carried by them.

          You might be onto something.

          1 Reply Last reply Reply Quote 0
          • CrossEyeC
            CrossEye
            last edited by

            I posted a bug report for the issue.

            https://redmine.pfsense.org/issues/4280

            If more information is needed, I can definitely provide it.

            However I definitely think this is an issue now, tested on two separate boxes now and both of them experienced the same issue with a fresh install and a very basic config on them.

            Take care all,

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              What NICs are you using? Do you have any hardware off-loading enabled at all? Nothing in the logs? What does ifconfig report?

              Steve

              1 Reply Last reply Reply Quote 0
              • CrossEyeC
                CrossEye
                last edited by

                @stephenw10:

                What NICs are you using? Do you have any hardware off-loading enabled at all? Nothing in the logs? What does ifconfig report?

                Steve

                Hey Steve,

                I use two Dell Optiplex 3010 i5 @ 3.10GHz with 16GB of RAM and a Intel quad NIC.

                I also just tested the same scenario with a Dell Optiplex 9010 and two additional single PCIe Intel NICs. The result was the same as before, as soon as I configured the LAGG, all traffic stopped passing.

                I have no hardware off-loading and checking the logs reveals nothing. Also ifconfig shows the carrier signal as active and the interface as up.

                As far as I can tell, It's like pfSense thinks the interface is working correctly.

                1 Reply Last reply Reply Quote 0
                • M
                  mir
                  last edited by

                  This setting: Hardware Checksum Offloading (system->advanced->network)
                  should actually be checked to prevent hardware offloading

                  1 Reply Last reply Reply Quote 0
                  • M
                    mir
                    last edited by

                    Maybe you could try to check this setting:
                    ARP Handling Suppress ARP messages
                    This option will suppress ARP log messages when multiple interfaces reside on the same broadcast domain

                    1 Reply Last reply Reply Quote 0
                    • CrossEyeC
                      CrossEye
                      last edited by

                      Ok so I created a VM with a fresh install of 2.2 and did the following,

                      System > Advanced > Networking
                      Checked "Disable hardware checksum offload"
                      Rebooted

                      Created a LAGG with the protocol set to LACP.
                      Created a new interface with the LAGG.
                      Created a new rule allowing any protocol on the new LAGG interface.

                      Started a Tcpdump on the interface and starting pinging, curling HTTP and HTTPS (to try and generate traffic) and nothing showed up on the Tcpdump.

                      Checked the routes and everything looks good. Even tried unchecking "Disable hardware checksum offload" and rebooting again and trying the above test… still did not work.

                      I am not at a loss as to why this is happening, anyone else have any ideas?

                      Thanks again all,

                      Gabriel

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Hmm. If this was some bug in FreeBSD 10.1 I'd expect to see far more mailing/forum traffic about it. Also I'm sure someone must have tested a LAGG when 2.2 was in Beta/RC (though I haven't personally).
                        VM is kind of a special case, hardware off-loading doesn't really apply. Exactly what NICs are you using, which Intel cards?

                        Steve

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          I only see this:
                          https://lists.freebsd.org/pipermail/freebsd-net/2014-February/037756.html
                          Do you have the sysctl:
                          net.link.lagg.0.lacp.lacp_strict_mode: 1

                          Change it to 0 to go back to FreeBSD 9 behaviour. You may have to cycle the LAGG or wait a while as it says in the post.

                          Steve

                          1 Reply Last reply Reply Quote 0
                          • CrossEyeC
                            CrossEye
                            last edited by

                            @stephenw10:

                            Hmm. If this was some bug in FreeBSD 10.1 I'd expect to see far more mailing/forum traffic about it. Also I'm sure someone must have tested a LAGG when 2.2 was in Beta/RC (though I haven't personally).
                            VM is kind of a special case, hardware off-loading doesn't really apply. Exactly what NICs are you using, which Intel cards?

                            Steve

                            I couldn't agree more Steve but you are more than welcome to run the same test and verify my results. I quite possibility could be misconfiguring it but I have done this setup multiple times now with great results until pfSense 2.2.

                            The NICs I use are the quad Intel Pro/1000 I340-T4,
                            http://ark.intel.com/products/49186/Intel-Ethernet-Server-Adapter-I340-T4

                            I never did any 2.2 beta testing, so I cannot speak if it worked in beta versions but I know that all of the 2.1.x stable builds work great.

                            Please if anyone can shed light on this issue, I am all ears.

                            Thanks again,

                            Gabriel

                            1 Reply Last reply Reply Quote 0
                            • CrossEyeC
                              CrossEye
                              last edited by

                              @stephenw10:

                              I only see this:
                              https://lists.freebsd.org/pipermail/freebsd-net/2014-February/037756.html
                              Do you have the sysctl:
                              net.link.lagg.0.lacp.lacp_strict_mode: 1

                              Change it to 0 to go back to FreeBSD 9 behaviour. You may have to cycle the LAGG or wait a while as it says in the post.

                              Steve

                              Great find Steve and I will run the test on the VM now and later tonight repeat the test on the physical hardware.

                              Gabriel

                              1 Reply Last reply Reply Quote 0
                              • J
                                jumpace07
                                last edited by

                                Late last night I also built a fresh 2.2 VM and rebuilt everything manually, a totally fresh setup so nothing coming across from a dodgy upgrade or anything.

                                I am going to be testing tonight with my LACP LAGG connection to my switch which is currently working perfectly with 2.1.5.

                                Will post results as well.

                                1 Reply Last reply Reply Quote 0
                                • C
                                  cmb
                                  last edited by

                                  The root issue is what I added to the 2.2 upgrade guide here.

                                  Either enable active LACP on your switch, or disable strict mode as noted there. We'll change the default back to its previous setting for 2.2.1.
                                  https://redmine.pfsense.org/issues/4308

                                  Steve - thanks for saving me some time searching by finding that first. :)

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post
                                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.