Pfsense 2.4.2-p1 LAGG LACP intel 520 possible flapping

  • Hi Pfsense community,

    Started using pfsense 2.4.2-p1 with an Intel 520-2 10G SFP+ dual port card. Firmware installed is v16.5.20
    I have setup an LACP LAGG which uses those two ports (ix0 and ix1).
    Created VLANs (12) using this new interface to assign on the pfsense.

    Also followed the information from this page:
    MTU: 9000
    System tunable:

    • hw.intr_storm_threshold=10000
      and my /boot/loader.conf.local as the following:
    • kern.ipc.nmbclusters="12255534"
    • kern.ipc.nmbjumbop="6127766"
    • kern.ipc.nmbjumbo9="5446902"
    • kern.ipc.nmbjumbo16="4085176"
    • hw.igb.num_queues=4
    • hw.bce.tso_enable=0
    • hw.pci.enable_msix=0

    The switches are two Force10 S4810 (one port of the port-channel on each). VLT is setup for the two switch.
    LACP mode active
    flow control is on for RX and TX

    The setup is working but not stable. On the pfsense I'm getting random:

    ix0: Interface stopped DISTRIBUTING, possible flapping
    ix1: Interface stopped DISTRIBUTING, possible flapping

    What should I look for to troubleshoot further? Let me know if you need more info to point me in the right direction.
    Should I disable Flow Control entirely on the switch and pfsense (via dev.ix.0.fc=0)?
    Any recommendation would be much appreciated  ;D