Intel Ethernet Controller I225-LM Support?

ndemmon

@stephenw10
This is where the "I've never touched freeBsd" bit is hitting me. I got the pfsense source code loaded onto a freebsd vm. While I couldn't get "make" working from {SRCROOT}/sys/modules/igc (some error about a bad definition of struct thread that implies to me that the include path was messed up but that was about as far as I was going to take it), I did get "make buildkernel" working fine from {SRCROOT}, and I went and found if_igc.ko in /usr/obj/...

And I think I'm stuck. I added a string/device_printf to an always-run path (right before the call to igc_setup_init_funcs() that's returning the error). I can grep for the string in my built if_igc.ko file and it's there. I made an assumption that if I replaced the /boot/kernel/if_igc.ko file on the pfsense box with the one I built, I'd get my debug printf in either dmesg -a, /var/log/dmesg.boot, or /var/log/* somewhere.

I did not, it's like it's still using the original driver to boot. What bit of FreeBSD magic am I missing?

stephenw10

You need to load it at boot by adding:

if_igc_load="YES"

to /boot/loader.conf.local. Create that file if it doesn't exist.

I've done that in the past with quite a few different modules but not with igc specifically.

Steve

ndemmon

@stephenw10
Woo hoo, "phy initialization error". Now I just have to fix the debug messages to actually print out debug information (like what values are being read and written to the phy so I can tell what it doesn't like in the error case). Just as an FYI, with more test cycles it's not tracking the link state of em0 anymore. The most reliable reproduction method is to halt the system (option 6 from the login menu) and then when I start the system via the on button the attach fails. Thinking maybe something doesn't get reset on that particular path, will update again if I find a fix and/or give up and just give an info dump.

Thanks for saving me the hours trying to find the loader.conf.local magic by myself.

ndemmon

@stephenw10

Found it.

        /* Make sure the PHY is in a good state. Several people have reported
         * firmware leaving the PHY's page select register set to something
         * other than the default of zero, which causes the PHY ID read to
         * access something other than the intended register.
         */
        ret_val = hw->phy.ops.reset(hw);

(igc_i225.c, line 168-ish)

phy->ops.reset = igc_null_ops_generic;

(igc_phy.c, line 37)
So something about that boot path leaves the phy's page select at non-default, which makes the phy revision come out unexpected. The code tries to fix this by issuing a reset of the phy, but nobody wrote and/or assigned the phy reset function. (Function tracing shows that igc_null_ops_generic() is getting called there instead.)

The problem is that that's not exactly a risk-free fix of throwing an actual phy reset into the boot path.

ndemmon

So if I throw in:

        phy->ops.acquire        = igc_acquire_phy_base;
        phy->ops.check_reset_block = igc_check_reset_block_generic;
        phy->ops.commit         = igc_phy_sw_reset_generic;
//Adding a reset function so phy_id always works.
        phy->ops.reset          = igc_phy_hw_reset_generic;
        phy->ops.release        = igc_release_phy_base;

it passes my initial tests. Assuming that this phy works like most of the dozens of phys that I've worked with, it implements two copies of the first few standard registers, one copy that tracks the "host side" of the phy that connects to the MAC and one copy that tracks the "line side" where you plug in the cable. The software reset bit is in that register window, so software reset doesn't affect the page you're reading from (you just reset the half of the phy that the page select currently has selected) so it has to be the hw_reset function.

This WILL cause an extra link-flap at boot time...I don't really care for my home setup. If I was looking for something safer to do in public code, I'd probably leave the path that grabs the phyID alone, but if it failed I'd do the hardware reset and retry. Or you could manually select page 0 before grabbing the phyID, but who knows what other stale info the PHY has that might screw something else up.

Thanks for the help!

stephenw10

Ah, nice!

Might be time to open a FreeBSD bug for that if nothing already exists.

Is this you?: https://reviews.freebsd.org/D32772

If not it looks like this is already being addressed.

That is in FreeBSD source now but not our fork yet.

Steve

xiazuojie

I think I have the same issue as @setarcos, that is very poor inbound packages (drops to zero) while outbound packages are working perfect at full 2.5GB speed.

I've tested the same setup with debian 11 and everything is working fine. So there's definitely something wrong with the freebsd driver.

The following is my setup and I've tried all the tricks mentioned in this thread:

i225-v b3 (which has supposedly fixed issues of early steppings) PCI-E adapter.
disabled onboard Broadcom BCM570x NIC at BIOS to avoid conflicts.
the NIC is connected to a TP-LINK TL-SH1008 switch at 2.5GB speed.
Truenas 12.0-U6.1 with the igc driver compiled from latest source from freebsd 12.0-stable dated Nov 13, 2021 (commit "igc: Use hardware routine for PHY reset").
disabled all hardware checksum and VLAN support as show below.
disabled all jails.

root@freenas:~ # pciconf -lv | grep -A1 -B2 network
igc0@pci0:2:0:0:        class=0x020000 card=0x00008086 chip=0x15f38086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    class      = network
    subclass   = ethernet

root@freenas:~ # ifconfig
igc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=802020<JUMBO_MTU,WOL_MAGIC>
        ether 2c:53:4a:0a:f5:ec
        inet 10.1.1.12 netmask 0xffffff00 broadcast 10.1.1.255
        media: Ethernet autoselect (2500Base-T <full-duplex>)
        status: active
        nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
        groups: pflog

stephenw10

@xiazuojie said in Intel Ethernet Controller I225-LM Support?:

Truenas 12.0-U6.1 with the igc driver compiled from latest source from freebsd 12.0-stable

You see the same running pfSense?

xiazuojie

Something I've noticed is that 6100 has i225-V B1 revision while both @setarcos and I have B3 revisions. I wonder if that makes any difference.

@stephenw10 said in Intel Ethernet Controller I225-LM Support?:

as far as I know though the only tested NIC is that in the 6100:
igc0@pci0:4:0:0: class=0x020000 card=0x00008086 chip=0x15f38086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller I225-V'
class = network
subclass = ethernet

B3 revision supposedly fixes hardware issues of B1 revision.

stephenw10

@xiazuojie said in Intel Ethernet Controller I225-LM Support?:

very poor inbound packages (drops to zero) while outbound packages are working perfect at full 2.5GB speed.

Is it actually zero of just close to zero. @Setarcos was seeing nothing come in at all, not even ARP replies. Reading back now I see I didn't ask if they were running the current Coreboot version. Some early versions had a bug that presented like this.

Steve

Setarcos

@stephenw10 I was (and still am) running ADI_PLCC-01.00.00.11 on my XG-7100

stephenw10

Hmm, not that then. Or at least not a known issue.

xiazuojie

@stephenw10 What I see is unstable connection with the NIC actually goes to down status after a short period with high inbound traffic (while outbound traffic is working fine). However, it's tested with TrueNAS 12.0-U6.1.
Since the igc driver is contributed to FreeBSD from netgate, I report the issue here.

With the same setup on pfsense 2.6.0-DEVELOPMENT snapshot and the i225-v b3 port as lan port, it does not show the issue.

Debian 11 does not have the same issue.

stephenw10

The development was sponsored by Netgate but issue still need to be reported upstream to FreeBSD. I have no idea how TrueNAS might be different to what we're running but it could be significantly.

Steve

n1ck31

I'm seeing mixed results online. Is the I225 working in 2.5.2 or is 2.6 basically a requirement?

I have the 2 port QNAP card coming tonight, and I'm not sure if I need to reinstall it or just plop it in and go.

I'd prefer to stay away from 2.6 for reliability sake, if this card is not "working", would an x550-t2 work? I saw posts that there's no 2.5g option in the UI with this card (only x710?)

bk150

@n1ck31 The QNAP cards work in 2.5.2 with UDP Checksum offloading disabled and they work in 2.6 snapshots without any modifications.

n1ck31

@bk150 I also saw some posts that people were claiming after a little while, the connection just drops or the ping goes crazy high?

I'm assuming that's due to that one setting being ticked?

I'm trying to decide if I put this new card in now or wait for a stable 2.6 release...

bk150

@n1ck31 I've seen mixed info on if UDP checksum offloading even does anything worthwhile. I'm no software engineer but in my limited knowledge, assuming your CPU can keep up with doing everything in software, you really shouldn't notice any difference. I've been running 2.5.2 with checksum offloading turned off for around 4 months now with multiple VLANs, a Wireguard tunnel, and am hosting a handful of gaming servers. I've seen absolutely no issues.

FWIW OPNSense has that setting off by default and PfSense has it on by default so who knows.

n1ck31

@bk150 Just tried on My Dell XE2, the same issue as someone had mentioned in this thread, not being detected correctly by pfSense. It sees the card is there but has an error -2 when trying to load the driver/attach.

Tested both 2.5.2 and 2.6, integrated intel NIC disabled.

stephenw10

Hmm, you probably need this:
https://github.com/freebsd/freebsd-src/commit/561cd74b1704ab76a3286733fb43da53b6892ea4#diff-6bb19804656a668f9aa22575da6a622a983af20b269eb5b35dbfda026c2499b9
That's not in pfSense, yet.

Though we'd need to see exactly how it fails at boot to know for sure. It's odd that card works at all for others if that's the case. Maybe there are different revisions?

Steve