NBASE-T Support for Intel X550

stephenw10

I don't think so, not in ixgbe. As far as I know those NICs support a real fixed speed if you set it. It's igc that does not and setting a speed only sets a single advertisement speed.

w0w

@stephenw10
Ok, but, I've been checked this right now. Gui selection for 2500, sets
dev.ix.0.advertise_speed: 16
5000:
dev.ix.0.advertise_speed: 32
1000:
dev.ix.0.advertise_speed: 3
autoselect:
dev.ix.0.advertise_speed: 7
10G:
dev.ix.1.advertise_speed: 7

stephenw10

Right, those are all multiple advertised values rather than a single available value which is what igc does.

It appears it needs to include the 2.5 and/or 5G bits when set to autoselect and is not.

w0w

@stephenw10
I think it is commit

Source

Setting 63 as value is not possible, because in fact it reports that card is not supporting 10M, but if I set 37 and select "default" it sets media to the 100M and at least it is working, getting an IP address also!
If i select 100M in GUI then it sets the sysctl value to 1, but no IP then. If I change sysctl value to 37 it gets the IP.
36, 32 causes no link on the port, but it is accepted by sysctl.
Setting it to 2 in sysctl the link is active but no IP then. Looks like there are some values are hardly encoded in the other part of code and maybe not on the pf side. I will try the Ubuntu again to see if I can change the port speed to different values.

w0w

It looks like Ubuntu have similar problems and even worse.
Link negotiation "Ignore" — this mode just means that some other manager or driver itself somehow auto-selecting the link speed, that puts the maximum speed for the link, like 2.5Gbps and 10Gbps.
Selecting 'auto' means 1Gbps in Ubuntu, at least on my cards.
Manual selecting is some kind of lottery that depends on a switch, for example if I select 1Gbps speed on 10G capable switch, it stays on 10Gbps, just ignoring the setting.
On other switch that is capable for 2.5Gbps and fully compliant with NBASE-T, I select 1Gbps, and it changes the link speed, but then no IP received from DHCP, like it happening on the pfSense side.

I have found one x550 user report that in the Windows you need manualy select the link speed for X550 when the other side is 226 intel card.

I hope those findings will help someone to make things a bit better.

At least it would be good to use dev.ix.1.advertise_speed: 37 as default value, so you will have a working connection, but this need to be tested.

stephenw10

Yes it would need to be extensively tested. Because it looks like that was deliberately not set as a default in order to prevent negotiation issues with some other hardware. So if we set that as a default it could break links for anyone with that hardware.

w0w

@stephenw10
Yes, sure…
The fastest way is just to add the explanation to the Netgate pfSense documentation?
I think it can be helpful when installing on the hardware when the ix550 is on the LAN side and connected to some 2.5/5Gbps switch that have no link, because of this setting missing or non-compatible by default.

stephenw10

I agree, let me see what I can do here....

stephenw10

https://redmine.pfsense.org/issues/14647

indigo88

@stephenw10 I can confirm that setting

sysctl dev.ix.X.advertise_speed=N

Where X is the interface number (e.g. for ix0, X=0 in the command above) and N is the sum of the advertised speeds, works.

In hex and decimal:

Control advertised link speed using these flags:
    0x1 - 1 - advertise 100M
    0x2 - 2 - advertise 1G
    0x4 - 4 - advertise 10G
    0x8 - 8 - advertise 10M

    0x10 - 16 - advertise 2.5G
    0x20 - 32 - advertise 5G

    100M and 10M are only supported on certain adapters.

My default value was 7, (in decimal, so it was 1+2+4, 100M/1G/10G) so to add advertisement of 2.5G and 5G on ix0, I ran:

sysctl dev.ix.0.advertise_speed=55

Since 7 + 16 + 32 = 55.

Now I see:

I also had to add dev.ix.0.advertise_speed as a system tunable, with value 55, to make it persist between reboots

w0w

@indigo88
Thank you.

I can confirm that setting 55 works fine on my cards also. Actually it's 1+2+4+16+32
8 (10M) not supported AFAIK on those cards.

indigo88

@w0w Right, the default of 7 is the result of 1 + 2 + 4 (100M/1G/10G)

It’s nice that we have the option, even if it’s a bit esoteric. Also nice that the GUI displays the link speed properly once it’s enabled

DeviceMangler

I just slotted an X550-T2 into my pfSense box on 2.7.0 and just wanted to chime in that it appears to be advertising NBASE-T speeds despite me not having added the system tunable. When I go to Interfaces > LAN, in the dropdown box for Speed and Duplex, along with autoselect, I have 10G, 5000, 2500, 1000BASE-T and 100BASE-TX all available for selection as media supported by this interface. I have yet to verify that it will link at anything above 1G, but will do so once I have something to try.

For what it's worth, it's a relatively new (2022 date of MFG) card and dmesg states:
ix0: Option ROM V1-b1313-p0 eTrack 0x80000760 PHY FW V523

I'm not sure what that corresponds to firmware-wise, but maybe it can be used for comparison. If there's anything you can think of that I can run from the shell for more detail, feel free to let me know.

w0w

@DeviceMangler
The current actual issue and mentioned system tunable is not related to what you see in the speed and duplex dropdown list. Issue is related to autonegotiation, all three X550-T2 I have, shows those NBASE-T speeds in drop-down list as well. The problem is that it does not negotiate on the correct speed, when using NBASE-T switch, and you should select it manually. When system tunable is added, the card automagically connects on the maximum NBASE-T speed. There is no new revisions of X550-T2 and never to be released AFAIK.

DeviceMangler

@w0w
Ah, all right. I assumed that the dropdown box wouldn't have the entries for the other speeds without the system tunable. Thought it may be firmware-related.

Guess I'll go ahead and add the tunable, then.

Thanks!

JBD

I am new to pfsense and plan build a router and install pfsense in the next few months. Hardware is atom c3758 with the dual 10gb ports. The c3000 soc has the intel x553 lan controller. Does anyone know if the modifications described here will allow for nbase-t on those ports like on the x550 cards?

stephenw10

AFAIK this works only on the x550 NIC. I do not expect it to work on the x553 NICs that are part of the C3K SoC. It does not work on the 7100 for example.

JBD

Ok and Thank you. Well it works on the x550 which can be had and I may have an opportunity to use an X710-t2l. It is 8 lane (I assume intel wanted to use the same board for the 2 port and 4 port which needs 8 lanes to save on costs), but pcie 3.0 with 4 lanes is all that is needed for a 2 port card and I have understand that it will work in a 4 lane slot at full speed. If it did not, there is always the intel i226.

Darkk

It's a bit of a bummer that the Intel X550-T2 is not a drop in part for latest version of pfsense without going through some hacks and tweaks. So makes me wonder are we better off going with a different card that has better support in FreeBSD? My concern is since you guys had to go through all of that just how safe is it going to be in future FreeBSD updates? I.e. removal of unsupported drivers.

I recently purchased a couple of I350-T4V2 DELL/INTEL cards off of e-bay and about to replace my older quad NIC in my pfsense whitebox firewall when I realize because I have a paid support for pfsense plus Netgate will only transfer the TAC lite over to the new Netgate Hardware ID as one time courtesy. Lesson learned for me is to only buy one year of TAC Lite instead of two since I like to tinker hardware often.

I'll search another thread for recommendation of a different card that will support 1,2.5,5 and 10 gig copper. Really wish consumer cable modems would have SFP+ ports which would make things alot easier. Ah well.

stephenw10

As far as I know the X550-T is the only card that supports N-Base-T and works in pfSense.

If you know you're going to have to change your NDI you should email support in advance to check. There is some flexibility there.