Intel Ethernet Controller I225-LM Support?

Setarcos · Jul 27, 2021, 3:41 PM

@stephenw10 said in Intel Ethernet Controller I225-LM Support?:

You are talking about?:
dev.igc.0.mac_stats.missed_packets: 1497
That does seem high, especially as a percentage of the total packets received!
dev.igc.0.mac_stats.mcast_pkts_recvd: 281
dev.igc.0.mac_stats.bcast_pkts_recvd: 14
dev.igc.0.mac_stats.good_pkts_recvd: 295
dev.igc.0.mac_stats.total_pkts_recvd: 1792

Yes, this is what I was referring to. The "in" packet count shows 0 on the UI though and I haven't managed to capture any inbound traffic on the I225-LM interface during testing. Note that the sysctl output and UI Status/Interfaces text capture for this interface (third text block from my previous reply) were taken a few minutes apart so the counters are not exactly the same.

dev.igc.3.reg_dump: General Registers
	CTRL	 58140641
	STATUS	 40780683
	CTRL_EXIT	 10000040

Interrupt Registers
	ICR	 00000000

RX Registers
	RCTL	 0444801e
	RDLEN	 00004000
	RDH	 0000000e
	RDT	 0000000d
	RXDCTL	 02040808
	RDBAL	 04a65000
	RDBAH	 00000000

TX Registers
	TCTL	 a503f0fa
	TDBAL	 04a27000
	TDBAH	 00000000
	TDLEN	 00004000
	TDH	 000001d0
	TDT	 000001d0
	TXDCTL	 0201011f
	TDFH	 00000000
	TDFT	 00000000
	TDFHS	 00000000
	TDFPC	 00000000


dev.igc.3.%pnpinfo: vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000 class=0x020000
dev.igc.3.%location: slot=0 function=0 dbsf=pci0:7:0:0 handle=\_SB_.PCI0.PEX6.PXSX
dev.igc.3.%driver: igc
dev.igc.3.%desc: Intel(R) PRO/1000 PCI-Express Network Driver

So it's a different device ID. The registers are set differently.

Yes, but isn't the device ID difference to be expected? The I225-LM and I225-V have different IDs (0x15F2 and 0x15F3 respectively per https://github.com/pfsense/FreeBSD-src/blob/devel-12/sys/dev/igc/igc_hw.h#L44) Are there any pertinent differences in the registers between the two?

dev.igc.0.reg_dump: General Registers
        CTRL     181c0641
        STATUS   40380683
        CTRL_EXIT        10000040

Interrupt Registers
        ICR      00000000

RX Registers
        RCTL     0440801e
        RDLEN    00004000
        RDH      00000000
        RDT      00000080
        RXDCTL   02040808
        RDBAL    033a8000
        RDBAH    00000001

TX Registers
        TCTL     a503f0fa
        TDBAL    0336a000
        TDBAH    00000001
        TDLEN    00004000
        TDH      00000013
        TDT      00000375
        TXDCTL   0201011f
        TDFH     00000000
        TDFT     00000000
        TDFHS    00000000
        TDFPC    00000000


dev.igc.0.%pnpinfo: vendor=0x8086 device=0x15f2 subvendor=0x1baa subdevice=0xc002 class=0x020000
dev.igc.0.%location: slot=0 function=0 dbsf=pci0:4:0:0
dev.igc.0.%driver: igc
dev.igc.0.%desc: Intel(R) PRO/1000 PCI-Express Network Driver

stephenw10 · Jul 27, 2021, 4:25 PM

@setarcos said in Intel Ethernet Controller I225-LM Support?:

Yes, but isn't the device ID difference to be expected?

Yes it is. I'm just saying it's different to the NIC in the 6100 so it may not have been tested.

Are there any pertinent differences in the registers between the two?

I don't know. The spec sheets don't look significantly different though.

Setarcos · Jul 28, 2021, 7:44 AM

@stephenw10

Does your 6100 have any igc driver tunables set? Here is what I am currently using (defaults):

[21.05-RELEASE][root@cerberus.setarcos.lan]/root: sysctl -a hw.igc
hw.igc.max_interrupt_rate: 8000
hw.igc.eee_setting: 1
hw.igc.rx_process_limit: 100
hw.igc.sbp: 1
hw.igc.smart_pwr_down: 0
hw.igc.rx_abs_int_delay: 66
hw.igc.tx_abs_int_delay: 66
hw.igc.rx_int_delay: 0
hw.igc.tx_int_delay: 66
hw.igc.disable_crc_stripping: 0

I have a 2.5Gbe device on the way and will be able to test with that shortly, but have otherwise run out of things to try.

stephenw10 · Jul 29, 2021, 2:44 AM

Just the same default values you're using:

[21.05-RELEASE][admin@6100-2.stevew.lan]/root: sysctl hw.igc
hw.igc.max_interrupt_rate: 8000
hw.igc.eee_setting: 1
hw.igc.rx_process_limit: 100
hw.igc.sbp: 1
hw.igc.smart_pwr_down: 0
hw.igc.rx_abs_int_delay: 66
hw.igc.tx_abs_int_delay: 66
hw.igc.rx_int_delay: 0
hw.igc.tx_int_delay: 66
hw.igc.disable_crc_stripping: 0

Steve

lra · Jul 29, 2021, 2:22 AM

@setarcos

You may want to try the card in a Linux 5.10+ system just to prove it is functioning properly.

Setarcos · Jul 29, 2021, 2:44 AM

This post is deleted!

Setarcos · Jul 29, 2021, 3:01 AM

@stephenw10

My 2.5GbE device arrived today, and while the port does seem to negotiate the line rate correctly when both forced and autodetected, the original problem remains and no inbound packets from the device attached to the i225-LM port are making it through.

Just for grins I tried disabling CRC stripping with hw.igc.disable_crc_stripping=1, and while I still don't see any in packets making it through, the error in counter is no longer incrementing for every packet received as it had before.

dev.igc.0.mac_stats.tso_txd: 0
dev.igc.0.mac_stats.tx_frames_1024_1522: 0
dev.igc.0.mac_stats.tx_frames_512_1023: 0
dev.igc.0.mac_stats.tx_frames_256_511: 0
dev.igc.0.mac_stats.tx_frames_128_255: 0
dev.igc.0.mac_stats.tx_frames_65_127: 0
dev.igc.0.mac_stats.tx_frames_64: 8
dev.igc.0.mac_stats.mcast_pkts_txd: 0
dev.igc.0.mac_stats.bcast_pkts_txd: 8
dev.igc.0.mac_stats.good_pkts_txd: 8
dev.igc.0.mac_stats.total_pkts_txd: 8
dev.igc.0.mac_stats.good_octets_txd: 512
dev.igc.0.mac_stats.good_octets_recvd: 0
dev.igc.0.mac_stats.rx_frames_1024_1522: 0
dev.igc.0.mac_stats.rx_frames_512_1023: 0
dev.igc.0.mac_stats.rx_frames_256_511: 0
dev.igc.0.mac_stats.rx_frames_128_255: 0
dev.igc.0.mac_stats.rx_frames_65_127: 0
dev.igc.0.mac_stats.rx_frames_64: 0
dev.igc.0.mac_stats.mcast_pkts_recvd: 0
dev.igc.0.mac_stats.bcast_pkts_recvd: 0
dev.igc.0.mac_stats.good_pkts_recvd: 0
dev.igc.0.mac_stats.total_pkts_recvd: 0
dev.igc.0.mac_stats.xoff_txd: 0
dev.igc.0.mac_stats.xoff_recvd: 0
dev.igc.0.mac_stats.xon_txd: 0
dev.igc.0.mac_stats.xon_recvd: 0
dev.igc.0.mac_stats.alignment_errs: 0
dev.igc.0.mac_stats.crc_errs: 0
dev.igc.0.mac_stats.recv_errs: 0
dev.igc.0.mac_stats.recv_jabber: 0
dev.igc.0.mac_stats.recv_oversize: 0
dev.igc.0.mac_stats.recv_fragmented: 0
dev.igc.0.mac_stats.recv_undersize: 0
dev.igc.0.mac_stats.recv_no_buff: 0
dev.igc.0.mac_stats.missed_packets: 0
dev.igc.0.mac_stats.defer_count: 0
dev.igc.0.mac_stats.sequence_errors: 0
dev.igc.0.mac_stats.symbol_errors: 0
dev.igc.0.mac_stats.collision_count: 0
dev.igc.0.mac_stats.late_coll: 0
dev.igc.0.mac_stats.multiple_coll: 0
dev.igc.0.mac_stats.single_coll: 0
dev.igc.0.mac_stats.excess_coll: 0

@lra that is a good idea, but I don't have a spare system to give this a try on at the moment. I will be building a TrueNAS SCALE box in the next month or so and can give it a try then. In the mean-time I'll probably try picking up another NIC to test.

stephenw10 · Jul 29, 2021, 3:23 PM

Hmm, there it's showing 0 packets received at all not just bad packets as it was before.
Was it actually connected when that was taken?

Setarcos · Jul 29, 2021, 3:23 PM

@stephenw10 said in Intel Ethernet Controller I225-LM Support?:

Hmm, there it's showing 0 packets received at all not just bad packets as it was before.
Was it actually connected when that was taken?

Yes, the port was actually connected to a test device and had received a few ARP replies from the test device when this was taken, but the system had been recently restarted resetting the counters and had only been online a few minutes at that point.

As another experiment, I modified the interface settings to use the second physical interface on the dual I225-LM card and it was exhibiting the same issues, but I let it sit overnight, and it does appear to have accumulated errors after all:

dev.igc.1.mac_stats.tso_txd: 0
dev.igc.1.mac_stats.tx_frames_1024_1522: 0
dev.igc.1.mac_stats.tx_frames_512_1023: 0
dev.igc.1.mac_stats.tx_frames_256_511: 0
dev.igc.1.mac_stats.tx_frames_128_255: 0
dev.igc.1.mac_stats.tx_frames_65_127: 0
dev.igc.1.mac_stats.tx_frames_64: 17
dev.igc.1.mac_stats.mcast_pkts_txd: 0
dev.igc.1.mac_stats.bcast_pkts_txd: 17
dev.igc.1.mac_stats.good_pkts_txd: 17
dev.igc.1.mac_stats.total_pkts_txd: 2860
dev.igc.1.mac_stats.good_octets_txd: 1088
dev.igc.1.mac_stats.good_octets_recvd: 32321
dev.igc.1.mac_stats.rx_frames_1024_1522: 0
dev.igc.1.mac_stats.rx_frames_512_1023: 28
dev.igc.1.mac_stats.rx_frames_256_511: 0
dev.igc.1.mac_stats.rx_frames_128_255: 37
dev.igc.1.mac_stats.rx_frames_65_127: 34
dev.igc.1.mac_stats.rx_frames_64: 26
dev.igc.1.mac_stats.mcast_pkts_recvd: 100
dev.igc.1.mac_stats.bcast_pkts_recvd: 3
dev.igc.1.mac_stats.good_pkts_recvd: 125
dev.igc.1.mac_stats.total_pkts_recvd: 2967
dev.igc.1.mac_stats.xoff_txd: 2843
dev.igc.1.mac_stats.xoff_recvd: 0
dev.igc.1.mac_stats.xon_txd: 0
dev.igc.1.mac_stats.xon_recvd: 0
dev.igc.1.mac_stats.alignment_errs: 0
dev.igc.1.mac_stats.crc_errs: 0
dev.igc.1.mac_stats.recv_errs: 0
dev.igc.1.mac_stats.recv_jabber: 0
dev.igc.1.mac_stats.recv_oversize: 0
dev.igc.1.mac_stats.recv_fragmented: 0
dev.igc.1.mac_stats.recv_undersize: 0
dev.igc.1.mac_stats.recv_no_buff: 0
dev.igc.1.mac_stats.missed_packets: 2842
dev.igc.1.mac_stats.defer_count: 0
dev.igc.1.mac_stats.sequence_errors: 0
dev.igc.1.mac_stats.symbol_errors: 0
dev.igc.1.mac_stats.collision_count: 0
dev.igc.1.mac_stats.late_coll: 0
dev.igc.1.mac_stats.multiple_coll: 0
dev.igc.1.mac_stats.single_coll: 0
dev.igc.1.mac_stats.excess_coll: 0

I decided to go a different route and picked up an X710-T2L and will be returning the QNAP QXG-2G2T-I225 as my return window will have closed by the time I have a chance to test this card on a Linux box to see if the issues persist. That said, I will still have the I225-LM card installed and available for testing through early next week so if you would like me to do any additional testing beforehand, please let me know.

stephenw10 · Aug 2, 2021, 4:30 PM

I'm not sure what else we can do there. It's just rejecting almost all the traffic it sees at some low level. It 'feels' like a hardware off loading issue to me. It all reports as disabled but maybe on that particular chip some register is set/not set that the driver doesn't see? No easy way to tell.

jerseymike · Aug 2, 2021, 4:30 PM

@stephenw10 Not sure if it's 100% relevant but I am using two of the single-port versions of the QNAP 2.5Gbe card, and could not get the DNS resolver to work at all. After disabling hardward crc offloading, the issue was resolved, so there's definitely something wonky with this driver/card combo.

Setarcos · Aug 2, 2021, 4:48 PM

@jerseymike said in Intel Ethernet Controller I225-LM Support?:

@stephenw10 Not sure if it's 100% relevant but I am using two of the single-port versions of the QNAP 2.5Gbe card, and could not get the DNS resolver to work at all. After disabling hardward crc offloading, the issue was resolved, so there's definitely something wonky with this driver/card combo.

This seems to align with @slk2k 's findings in the thread here, however, it seems to have no effect with the dual port QNAP QXG-2G2T-I225 card I have been testing with. I had initially tried disabling TX and RX CRC offloading from the UI, but later did so directly with ifconfig and both yielded the same results (no inbound packets are making it through).

stephenw10 · Aug 2, 2021, 7:07 PM

Hmm, have either of you tried FreeBSD directly?

That should be no different but....

slk2k · Aug 3, 2021, 11:40 AM

@setarcos

Now that the system has been in place for a long weekend, I checked the stats and nothing is unusual - no missed packets. Things looks reasonably well (other than a crap-ton of interrupts, but that's because all the HW offloading is turned off).

/root: sysctl -a dev.igc.0
dev.igc.0.interrupts.rx_desc_min_thresh: 0
dev.igc.0.interrupts.asserts: 59793578
dev.igc.0.mac_stats.tso_txd: 0
dev.igc.0.mac_stats.tx_frames_1024_1522: 29552898
dev.igc.0.mac_stats.tx_frames_512_1023: 337783
dev.igc.0.mac_stats.tx_frames_256_511: 2041523
dev.igc.0.mac_stats.tx_frames_128_255: 867472
dev.igc.0.mac_stats.tx_frames_65_127: 11716942
dev.igc.0.mac_stats.tx_frames_64: 9123432
dev.igc.0.mac_stats.mcast_pkts_txd: 67
dev.igc.0.mac_stats.bcast_pkts_txd: 268
dev.igc.0.mac_stats.good_pkts_txd: 53640050
dev.igc.0.mac_stats.total_pkts_txd: 53640050
dev.igc.0.mac_stats.good_octets_txd: 47517179206
dev.igc.0.mac_stats.good_octets_recvd: 94980575076
dev.igc.0.mac_stats.rx_frames_1024_1522: 61648203
dev.igc.0.mac_stats.rx_frames_512_1023: 760161
dev.igc.0.mac_stats.rx_frames_256_511: 1040034
dev.igc.0.mac_stats.rx_frames_128_255: 1551643
dev.igc.0.mac_stats.rx_frames_65_127: 6856810
dev.igc.0.mac_stats.rx_frames_64: 15037895
dev.igc.0.mac_stats.mcast_pkts_recvd: 128920
dev.igc.0.mac_stats.bcast_pkts_recvd: 623916
dev.igc.0.mac_stats.good_pkts_recvd: 86894746
dev.igc.0.mac_stats.total_pkts_recvd: 86894746
dev.igc.0.mac_stats.xoff_txd: 0
dev.igc.0.mac_stats.xoff_recvd: 0
dev.igc.0.mac_stats.xon_txd: 0
dev.igc.0.mac_stats.xon_recvd: 0
dev.igc.0.mac_stats.alignment_errs: 0
dev.igc.0.mac_stats.crc_errs: 0
dev.igc.0.mac_stats.recv_errs: 0
dev.igc.0.mac_stats.recv_jabber: 0
dev.igc.0.mac_stats.recv_oversize: 0
dev.igc.0.mac_stats.recv_fragmented: 0
dev.igc.0.mac_stats.recv_undersize: 0
dev.igc.0.mac_stats.recv_no_buff: 0
dev.igc.0.mac_stats.missed_packets: 0
dev.igc.0.mac_stats.defer_count: 0
dev.igc.0.mac_stats.sequence_errors: 0
dev.igc.0.mac_stats.symbol_errors: 0
dev.igc.0.mac_stats.collision_count: 0
dev.igc.0.mac_stats.late_coll: 0
dev.igc.0.mac_stats.multiple_coll: 0
dev.igc.0.mac_stats.single_coll: 0
dev.igc.0.mac_stats.excess_coll: 0
dev.igc.0.queue_rx_3.rx_irq: 0
dev.igc.0.queue_rx_3.rxd_tail: 287
dev.igc.0.queue_rx_3.rxd_head: 288
dev.igc.0.queue_rx_2.rx_irq: 0
dev.igc.0.queue_rx_2.rxd_tail: 374
dev.igc.0.queue_rx_2.rxd_head: 375
dev.igc.0.queue_rx_1.rx_irq: 0
dev.igc.0.queue_rx_1.rxd_tail: 165
dev.igc.0.queue_rx_1.rxd_head: 166
dev.igc.0.queue_rx_0.rx_irq: 0
dev.igc.0.queue_rx_0.rxd_tail: 363
dev.igc.0.queue_rx_0.rxd_head: 364
dev.igc.0.queue_tx_3.tx_irq: 0
dev.igc.0.queue_tx_3.txd_tail: 647
dev.igc.0.queue_tx_3.txd_head: 647
dev.igc.0.queue_tx_2.tx_irq: 0
dev.igc.0.queue_tx_2.txd_tail: 789
dev.igc.0.queue_tx_2.txd_head: 789
dev.igc.0.queue_tx_1.tx_irq: 0
dev.igc.0.queue_tx_1.txd_tail: 754
dev.igc.0.queue_tx_1.txd_head: 754
dev.igc.0.queue_tx_0.tx_irq: 0
dev.igc.0.queue_tx_0.txd_tail: 294
dev.igc.0.queue_tx_0.txd_head: 294
dev.igc.0.fc_low_water: 32752
dev.igc.0.fc_high_water: 32768
dev.igc.0.rx_control: 71335966
dev.igc.0.device_control: 404489793
dev.igc.0.watchdog_timeouts: 0
dev.igc.0.rx_overruns: 0
dev.igc.0.link_irq: 2
dev.igc.0.dropped: 0
dev.igc.0.eee_control: 1
dev.igc.0.itr: 488
dev.igc.0.tx_abs_int_delay: 66
dev.igc.0.rx_abs_int_delay: 66
dev.igc.0.tx_int_delay: 66
dev.igc.0.rx_int_delay: 0
dev.igc.0.rs_dump: 0

But my thruput seems off. My previous modem and connection I would get a higher thruput on the speedtest (like 940M down, 42M up). Now I can't test higher than 860M down, 36M up). Might have more to do with the modem as much as the NIC ports. I get the same directly connected to the modem.

Setarcos · Aug 4, 2021, 4:26 AM

@stephenw10 said in Intel Ethernet Controller I225-LM Support?:

Hmm, have either of you tried FreeBSD directly?

That should be no different but....

I have not, but a fair comparison would include both vanilla FreeBSD 12.2 with the igc driver added and a Linux kernel 5.10+ based distro with a variety of I225-LM and I225-V hardware. I just don't have the physical hardware to test this out in my homelab.

stephenw10 · Aug 4, 2021, 2:01 PM

I would test with any FreeBSD version that has the driver just to know if it works at all there. That looks like only Main right now: https://github.com/freebsd/freebsd-src/tree/main/sys/dev/igc
So I'd test a snapshot.

Steve

bk150 · Aug 5, 2021, 5:30 AM

Wow I'm glad i found this post. I have 2 of the QNAP I225-LM cards installed with pfsense 2.5.2 and the DNS resolver wasn't working on a fresh install. I was absolutely tearing my hair out trying to figure out why it wasn't working (blaming my AT&T fiber gateway that I'm forced to use...).

ipv6 also wasn't working using DHCPv6 prefix delegation. The WAN interface I225-LM could get an address but none of the clients on my LAN interface I225-LM network could.

Using two integrated I-211 and I-219 gigabit controllers on the motherboard yields no such issues. Let me know if there is anything specific that anyone wants me to test while I still have the 2 troublesome I225-LM cards.

Edit: I reinstalled 2.5.2 and disabled hardware offloading and now everything is working great.

Setarcos · Aug 5, 2021, 7:49 AM

I am not sure if it matters, but both @bk150 and @slk2k are using CE 2.5.2 and I am using Plus 21.05. @jerseymike what were you running when you did your test with the single port QNAP I225-LM cards?

jerseymike · Aug 5, 2021, 11:17 AM

@setarcos I'm running CE 2.5.2

stephenw10 · Aug 5, 2021, 1:38 PM

@bk150 said in Intel Ethernet Controller I225-LM Support?:

I reinstalled 2.5.2 and disabled hardware offloading and now everything is working great.

Hardware Checksum Offloading? Using the GUI checkbox?