BRCM 10G/GbE 2+2P 57800-t rNDC Pfsense problem



  • Hi,

    I have a problem with this NIC on two Pfsense servers.

    The NIC's are shutting down and following message spams the console:

    bxe1: WARNING: Interface link is down, ignoring transmit request

    Serverhardware is Dell, with NIC type BRCM 10G/GbE 2+2P 57800-t rNDC (new servers)

    I have this in the /boot/loader.conf.local

    
    hw.bce.tso_enable=0
    kern.cam.boot_delay=10000
    kern.ipc.nmbclusters="1048576"
    net.inet.tcp.tso=0
    hw.pci.enable_msix=0
    kern.ipc.nmbjumbop="1048576"
    net.isr.bindthreads=0
    net.isr.maxthreads=1
    kern.random.sys.harvest.ethernet=0
    kern.random.sys.harvest.point_to_point=0
    kern.random.sys.harvest.interrupt=0
    net.isr.defaultqlimit=2048
    net.isr.maxqlimit=40960
    
    

    Any idea what can cause the problem?

    Pfsense version:

    
    2.2.4-RELEASE (amd64) 
    built on Sat Jul 25 19:57:37 CDT 2015 
    FreeBSD 10.1-RELEASE-p15
    
    


  • Serverhardware is Dell, with NIC type BRCM 10G/GbE 2+2P 57800-t rNDC (new servers)

    Could it be that this are Broadcom NICs they are perhaps also not supported by pfSense?

    Anyway the real failure message is that the link is not up, do you have any problems with the Internet Connection.





  • NPAR is disabled on the NIC ports.

    Most times after booting the servers, the NIC's are working, but at random times, sometimes, 24 hours or 36 hours the one of the NIC's are starting to post console messages like this:

    bxe1: WARNING: Interface link is down, ignoring transmit request

    Could also be bxe0.

    ifconfig bxe0 down && ifconfig bxe1 down works, wait like 3 minutes and than ifconfig bxe0 up && ifconfig bxe1 up resolves the problem for again like 24 hours.

    Could this be some kind of memory issue?



  • That's generally the sign of a driver issue, where an ifconfig down && up resolves things for some period of time. Reporting to freebsd-net list with specifics, or filing a FreeBSD PR after replicating with stock 11-CURRENT assuming it's replicable there, would be your best bet. Where it doesn't impact hardware we sell, we don't get much into driver issues.