Intel PRO/1000 PT Dual Port Server Adapter (82571) randomly stops working

[josh4trunks]

Hi guys, my pci-e x4 dual port NIC is acting all screwy with PfSense 2.1.

From a fresh boot/reboot the NIC works perfectly but all of a sudden will drop the connection on both ports.

• Once the connection drops an 'ifconfig down/up' doesn't even bring it back to life.
• A ping on the subnets of the downed card gives..

ping: sendto: No buffer space available

INFO
OS = PfSense 2.1 amd64
Mobo/CPU = X7SPA-H-D525

em0@pci0:1:0:0:	class=0x020000 card=0x135e8086 chip=0x105e8086 rev=0x06 hdr=0x00
    class      = network
    subclass   = ethernet
em1@pci0:1:0:1:	class=0x020000 card=0x135e8086 chip=0x105e8086 rev=0x06 hdr=0x00
    class      = network
    subclass   = ethernet

em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
	options=4019b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso>ether 00:15:17:12:b5:96
	inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255
	inet6 fe80::215:17ff:fe12:b596%em0 prefixlen 64 scopeid 0x1 
	nd6 options=1 <performnud>media: Ethernet autoselect
	status: no carrier
em1: flags=8c43 <up,broadcast,running,oactive,simplex,multicast>metric 0 mtu 1500
	options=4019b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso>ether 00:15:17:12:b5:97
	inet 192.168.3.1 netmask 0xffffff00 broadcast 192.168.3.255
	inet6 fe80::215:17ff:fe12:b597%em1 prefixlen 64 scopeid 0x2 
	nd6 options=1 <performnud>media: Ethernet autoselect (10baseT/UTP <full-duplex>)
	status: active</full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso></up,broadcast,running,oactive,simplex,multicast></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4,vlan_hwtso></up,broadcast,running,simplex,multicast> 

See attached screenshot of PfSense's 'Status > Interfaces' page, the 'In/out errors' don't show up until the NIC stops working.

I have tried these settings with no avail, though the problem did seem to take longer to happen.

kern.ipc.nmbclusters=131072
hw.em.txd=4096
hw.em.rxd=4096

The problem happens even when I boot with nothing connected to the card; I'll check the interfaces page and see the 'In/out errors' start building.

Please help, I'm going to try and get a ET chipset NIC for PfSense but want to make sure this card still works cause I'm going to use it in another FreeBSD machine. Thanks for the help!

[msi]

Hi

Since you say to have a native FreeBSD box available, maybe you can try building em(4) over there and load that module via loader.conf.local on pfSense to override the in-kernel em module.
Maybe some backporting went wrong, that could help isolate issues. The PRO/1000 PT is quite old now and remember to see that NIC supported by the base driver in 8.3-RELEASE.

[josh4trunks]

Good idea. I'll see if the newer em(4) makes a difference.

The part that is tricky is I don't know what triggers the all of a sudden malfunctioning.  I'll just put my desktop on that card and keep a constant torrent stream going to see if I can trigger it.

[jwelter99]

We had the same issue but the death would be in 24 to 72 hours but similar symptoms.  A reboot was needed to get it back functional.  We tried the same tuning entries you did but did not help.  This was on Supermicro X8SIL boards with a slightly newer NIC but still used the em driver.

Since this was production servers we moved to a box that uses the igb drivers instead and so far so good - knock on wood.

Note we did see issues using LAGG on 2.1 even with the igb based NIC's but need to do some more testing to ensure it's not a switch issue.

[josh4trunks]

So I compiled em-7.3.8 on a FreeBSD-8.3 virtual machine. I am successfully able to load that driver during boot for the virtual machine and everything seems to work.

I moved the driver over to PfSense's /boot/modules, gave proper 555 permissions, and added 'if_em_load="YES"' to /boot/loader.conf.local.

My PfSense would hang when booting on "configuring WAN interface". ifconfig in single user mode showed the interfaces seemed to work with the new driver, maybe I need to reconfigure my interfaces for pfsense? I can try it, I'll do it tonight when I have more time.

What's weird is I'm pretty sure this NIC worked flawlessly with PfSense 2.0.1 when I used it over a year ago.