In errors on Intel X722 NIC since update to 2.6.0
-
Hi,
I have been observing high inerrors on two Intel X722 NICs since exactly the time I upgraded my pfsense from 2.5.2 to 2.6.0. After the pfsense was back online my monitoring recorded the first inerrors. Before upgrade I had no problems. ixl2 is responsible for LAN and ixl3 for WAN. The amount of traffic does not seem to play a role. Pfsense is running here on a Supermicro SYS-5019D-FN8TP with X11SDV-8C-TP8F board.ifconfig
ixl2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
media: Ethernet autoselect (10Gbase-SR <full-duplex>)ixl3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
media: Ethernet autoselect (10Gbase-SR <full-duplex>)netstat
Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll
ixl3 1500 <Link#8> 3c:ec:ef:xx:xx:7d 5646273152 8457 0 3329536602 0 0Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll
ixl2 1500 <Link#7> 3c:ec:ef:xx:xx:7c 3348495308 8494 0 5655364293 0 0
pciconf
ixl2@pci0:183:0:2: class=0x020000 card=0x37d015d9 chip=0x37d08086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Connection X722 for 10GbE SFP+'
class = network
subclass = ethernetixl3@pci0:183:0:3: class=0x020000 card=0x37d015d9 chip=0x37d08086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Connection X722 for 10GbE SFP+'
class = network
subclass = ethernetpfctl -s info
Status: Enabled for 3 days 11:53:12 Debug: UrgentInterface Stats for ixl3 IPv4 IPv6
Bytes In 6746610399786 42106894
Bytes Out 1694269661081 31105578
Packets In
Passed 5715567136 619348
Blocked 304557 0
Packets Out
Passed 335 0
Blocked 620707 0State Table Total Rate
current entries 46690
searches 19731849353 65339.0/s
inserts 72608950 240.4/s
removals 72562938 240.3/s
Counters
match 82204073 272.2/s
bad-offset 0 0.0/s
fragment 162 0.0/s
short 253 0.0/s
normalize 67 0.0/s
memory 0 0.0/s
bad-timestamp 0 0.0/s
congestion 0 0.0/s
ip-option 415714 1.4/s
proto-cksum 0 0.0/s
state-mismatch 30558 0.1/s
state-insert 681 0.0/s
state-limit 0 0.0/s
src-limit 0 0.0/s
synproxy 0 0.0/s
map-failed 0 0.0/s
Since Akismet flags my post as spam I have omitted the output of sysctl dev.ixl.3 and sysctl dev.ixl.2.
Have there been any changes to the driver? Does anyone have an idea what could be the cause?
-
It's almost certainly because of the driver change going to 2.6 that now reports more input error types on that counter:
https://github.com/pfsense/FreeBSD-src/commit/9fb4d27695499501572335700418ca40b20cb6b1So the actual errors on the NIC have likely not changed you just weren't seeing them before.
Steve
-
The same change went into ix. See: https://redmine.pfsense.org/issues/12904
-
@stephenw10
Thank you for your answer. That explains a lot. Indeed, the counter for CRC checksums is increasing.sysctl -a | grep checksum_errors dev.ixl.3.mac.checksum_errors: 8472 dev.ixl.2.mac.checksum_errors: 8967
Now I wonder where the CRC errors on both interfaces come from.
Are there any known hardware checksum offloading issues for this type of card or driver? -
I'm not aware of anything specific but there are quite a few hardware variations. Disabling hardware checksum offloading is an easy test with pretty much no drawbacks.
Steve