In errors on Intel X722 NIC since update to 2.6.0

nightcore500

Hi,
I have been observing high inerrors on two Intel X722 NICs since exactly the time I upgraded my pfsense from 2.5.2 to 2.6.0. After the pfsense was back online my monitoring recorded the first inerrors. Before upgrade I had no problems. ixl2 is responsible for LAN and ixl3 for WAN. The amount of traffic does not seem to play a role. Pfsense is running here on a Supermicro SYS-5019D-FN8TP with X11SDV-8C-TP8F board.

ifconfig

ixl2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
media: Ethernet autoselect (10Gbase-SR <full-duplex>)

ixl3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
media: Ethernet autoselect (10Gbase-SR <full-duplex>)

netstat

Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll
ixl3 1500 <Link#8> 3c:ec:ef:xx:xx:7d 5646273152 8457 0 3329536602 0 0

Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll
ixl2 1500 <Link#7> 3c:ec:ef:xx:xx:7c 3348495308 8494 0 5655364293 0 0

pciconf

ixl2@pci0:183:0:2: class=0x020000 card=0x37d015d9 chip=0x37d08086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Connection X722 for 10GbE SFP+'
class = network
subclass = ethernet

ixl3@pci0:183:0:3: class=0x020000 card=0x37d015d9 chip=0x37d08086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Connection X722 for 10GbE SFP+'
class = network
subclass = ethernet

pfctl -s info

Status: Enabled for 3 days 11:53:12 Debug: Urgent

Interface Stats for ixl3 IPv4 IPv6
Bytes In 6746610399786 42106894
Bytes Out 1694269661081 31105578
Packets In
Passed 5715567136 619348
Blocked 304557 0
Packets Out
Passed 335 0
Blocked 620707 0

State Table Total Rate
current entries 46690
searches 19731849353 65339.0/s
inserts 72608950 240.4/s
removals 72562938 240.3/s
Counters
match 82204073 272.2/s
bad-offset 0 0.0/s
fragment 162 0.0/s
short 253 0.0/s
normalize 67 0.0/s
memory 0 0.0/s
bad-timestamp 0 0.0/s
congestion 0 0.0/s
ip-option 415714 1.4/s
proto-cksum 0 0.0/s
state-mismatch 30558 0.1/s
state-insert 681 0.0/s
state-limit 0 0.0/s
src-limit 0 0.0/s
synproxy 0 0.0/s
map-failed 0 0.0/s

Since Akismet flags my post as spam I have omitted the output of sysctl dev.ixl.3 and sysctl dev.ixl.2.

Have there been any changes to the driver? Does anyone have an idea what could be the cause?

stephenw10

It's almost certainly because of the driver change going to 2.6 that now reports more input error types on that counter:
https://github.com/pfsense/FreeBSD-src/commit/9fb4d27695499501572335700418ca40b20cb6b1

So the actual errors on the NIC have likely not changed you just weren't seeing them before.

Steve

stephenw10

The same change went into ix. See: https://redmine.pfsense.org/issues/12904

nightcore500

@stephenw10
Thank you for your answer. That explains a lot. Indeed, the counter for CRC checksums is increasing.

sysctl -a | grep checksum_errors
dev.ixl.3.mac.checksum_errors: 8472
dev.ixl.2.mac.checksum_errors: 8967

Now I wonder where the CRC errors on both interfaces come from.
Are there any known hardware checksum offloading issues for this type of card or driver?

stephenw10

I'm not aware of anything specific but there are quite a few hardware variations. Disabling hardware checksum offloading is an easy test with pretty much no drawbacks.

Steve