NBASE-T Support for Intel X550
-
Hi all,
I have an Intel X550 T2 and I’m going to be upgrading the switch to one that supports NBASE-T.
According to the Intel ARK page, NBASE-T is supported on this NIC in Linux. Has anyone tried this for the X550 on pfSense or FreeBSD?
Thanks
-
yeah, works great, plug and play under pfSense
ix0@pci0:1:0:0: class=0x020000 card=0x00028086 chip=0x15d18086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'Ethernet Controller 10G X550T' class = network subclass = ethernet
-
@indigo88 said in NBASE-T Support for Intel X550:
NBASE-T
I assume you mean 2.5Gbps and/or 5Gbps?
Not sure I've seen an ix card do that under FreeBSD. The only card I'm aware of that can do 2.5Gbps is a Broadcom card but that requires changing the NIC firmware settings and patching the driver.Steve
-
what if i put a cat5e cable on mine to test at which speed it connect ?
do you think it can only go from 1000 to 10gb under freebsd ? -
@stephenw10 Yes, here’s the ARK page: https://ark.intel.com/content/www/us/en/ark/products/88209/intel-ethernet-converged-network-adapter-x550-t2.html
It’s been in my system since 2017, works fine at 1G and 10G.
You can see it supports 2.5G and 5G. Just wondering if anyone has tried it with either standard.
-
i can't change media
[2.5.0-DEVELOPMENT][root@pfSense.localdomain]/root: ifconfig -m ix0 ix0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8538b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO> capabilities=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether a0:36:9f:17:c7:06 inet 192.168.10.254 netmask 0xffffff00 broadcast 192.168.10.255 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active supported media: media autoselect media 100baseTX media 1000baseT media 10Gbase-T nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
there is no 5 or 2,5 that can be selected
-
The only reference I see for 2.5Gbps even in the most recent ix driver is for 2500base-KX like we use for the internal link in the XG-7100, where no PHY exists:
https://github.com/freebsd/freebsd/blob/fe710b3242abb89deab33c466c1911d45dbde795/sys/dev/ixgbe/if_ix.c#L1238Steve
-
AFAIK NBASE-T for X550 is supported only for Windows and Linux OSes, moreover, on release NBASE-T was stated only for Linux by INTEL ARK.
I've been running into the same issue with speeds and same card on FreeBSD and the solution was to place the cheapest unmanaged switch with NBASE-T supported, but the other end must support NBASE-T speed switching and you should manually select the desired speed or you need the managed switch. -
[2.4.5-RELEASE][admin@pfSense.localdomain]/root: ifconfig -m ix1 ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO> capabilities=f507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether a0:36:9f: hwaddr a0:36:9f: inet6 fe80::a236:9fff:fe27:b1f6%ix1 prefixlen 64 scopeid 0x2 inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet autoselect (Unknown) status: active supported media: media autoselect media 100baseTX media 1000baseT media 10Gbase-T
this is how it looks when runing at 2.5gbps. it seems to work fine was passing traffic, but has that scary unknown on ui. also still not able to select 2.5g/5g as listed under linux drivers by intel ARK for x550 specs. maybe its not been updated yet under pfsense.
-
Interesting. The other end shows it linked at 2.5G though?
-
Just re-tested it also.
I can confirm that selecting on the other end speeds below 10G is now working and yes, it's "unknown" mode on pfSense side. But I do not think there is anything to do with that since Intel driver does not report any speed, not pfSense. -
@stephenw10 yes , i had to use a realtek 2.5g on a new board because i could not get x550 to change to 2.5g under PFsense or on the other side on windows system. it would be nice if PFsense suported 2.5 and 5g as selectable on its end if posible. idk where there is a hiccup in support, intel - bsd - pfsense. intel says under linux driver it should be setectable so idk,
thank you for keeping an eye on this issue.
-
under windows driver nbase t is only for auto-negotiating depending on link partner. but under Linux it should be fully supported and select-able. unless i am missing something.
https://www.intel.com/content/dam/www/public/us/en/documents/release-notes/ethernet-controller-x550-feature-support-matrix.pdf -
If Intel says it's Linux only then that's probably true since they write the FreeBSD drivers themselves. There is probably nothing much we can do about that but wait for an update.
If the driver shows it as selectable then pfSense should also show that, the values from the driver are just passed through.Steve
-
so even its not user select-able under BSD why is the driver seeing the current state as unknown. it work at 2.5/5 with auto negotiating when at link partner request under every other OS and displays it under ui and driver but its not displaying the current state under BSD. the HW support is there but the driver just does not know what is going on ?
i don't have a spare box up atm so i can just test on windows and pfsense(bsd)
here is X550 under windows driver connecting to the same realtek 2.5 board working and displaying fine. only the user selectablity to NBase-t is limited to linux. auto negotiating to nbase-t should be working on all drivers and os, should be displayed on ui.
-
Pretty much, yeah. The driver in FreeBSD 11.3 / pfSense 2.4.5 cannot return a link speed of 2.5G or 5G but it seemingly can show unknown.
The most recent drivers will be in FreeBSD 12 or those Intel offers for download.Try a pfSense 2.5 snapshot, currenntly based on 12-rel. Or a newer FreeBSD image.
Steve
-
[2.5.0-DEVELOPMENT][admin@pfSense.localdomain]/root: ifconfig -m ix1 ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e538bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> capabilities=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether a0:36:9f: inet6 fe80::a236:9fff:fe27:b1f6%ix1 prefixlen 64 scopeid 0x2 inet6 fe80::1:1%ix1 prefixlen 64 scopeid 0x2 inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 media: Ethernet autoselect (Unknown) status: active supported media: media autoselect media 100baseTX media 1000baseT media 10Gbase-T
same in 2.5.0 :( definitely an FreeBSD/PFsense driver only issue . tested with Ubuntu bootable usb as well. as with windows it displays and auto negotiates at 2.5g just fine, i may test under newest FreeBSD tomorrow on the pfsense box it self.
-
Mine was always 2.5. Definitely FreeBSD/Intel issue, nothing to do on pfSense side. But at least it works now and this should not be a problem anyway.
-
Even the latest driver from Intel does not list 2.5G or 5G link speeds.
https://downloadcenter.intel.com/download/14688/Intel-Network-Adapters-Driver-for-PCIe-10-Gigabit-Network-Connections-Under-FreeBSD-?product=192957Steve
-
@stephenw10
This is the same driver we have in pfSense 2.5.0. Don't know are there any plans to implement full support for NBASE-T or not, can't find anything about it on FreeBSD side. -
Installed a Netgear CM2000 modem today. It's a DOCSIS 3.1 modem with a single 2.5GBASE-T port. The modem reports a 2.5G connection and pfSense reports "Unknown" as @WanTime showed earlier. I also see "<rxpause,txpause>" but it does not affect the connection as far as I can tell.
I'm getting an aggregate bandwidth of about 1.2 Gbps between devices so Comcast is overprovisioning my gigabit service. Upload is still terrible however.
-
Can anyone recommend a 1-port and/or 2-port low profile card?
-
Just another data point:
pfsense 2.5.1-RELEASE (amd64)
Intel X550-T2 connected to the 2.5Gb port on an Arris Surfboard S33 Cable Modem support page:
Standard Specification Compliant DOCSIS 3.1 Hardware Version V1.0 Software Version TB01.01.001.14_102120_192.S3 Energy Efficient Ethernet Enabled
Connected with a 10m Cat8 Cable.
ifconfig -m ix1 reports:
media: Ethernet autoselect (Unknown <rxpause,txpause>) status: active supported media: media autoselect media 100baseTX media 1000baseT media 10Gbase-T
From the transfer speeds it definitely seems to be connected at greater than 1Gb and seems to be 2.5Gb from what I can tell. Obviously it's not paused.
The latest Intel Driver is 3.3.24
The README still says:
This driver supports the following media type options: Media Type Description ---------- ----------- autoselect Enables auto-negotiation for speed and duplex. 10baseT/UTP Sets speed to 10 Mbps. Use the ifconfig mediaopt option to select full-duplex mode. 100baseTX Sets speed to 100 Mbps. Use the ifconfig mediaopt option to select full-duplex mode. 1000baseTX Sets speed to 1000 Mbps. In this case, the driver supports only full-duplex mode. 1000baseSX Sets speed to 1000 Mbps. In this case, the driver supports only full-duplex mode.
Here's the dmesg output from boot:
... ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver> mem 0xfcc00000-0xfcffffff,0xfcbfc000-0xfcbfffff irq 19 at device 0.0 on pci2 ix0: Using 2048 TX descriptors and 2048 RX descriptors ix0: Using 8 RX queues 8 TX queues ix0: Using MSI-X interrupts with 9 vectors ix0: allocated for 8 queues ix0: allocated for 8 rx queues ix0: Ethernet address: a0:36:9f:28:1f:a6 ix0: PCI Express Bus: Speed 5.0GT/s Width x4 ix0: netmap queues/slots: TX 8/2048, RX 8/2048 ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver> mem 0xfc400000-0xfc7fffff,0xfcbf8000-0xfcbfbfff irq 18 at device 0.1 on pci2 ix1: Using 2048 TX descriptors and 2048 RX descriptors ix1: Using 8 RX queues 8 TX queues ix1: Using MSI-X interrupts with 9 vectors ix1: allocated for 8 queues ix1: allocated for 8 rx queues ix1: Ethernet address: a0:36:9f:28:1f:a4 ix1: PCI Express Bus: Speed 5.0GT/s Width x4 ix1: netmap queues/slots: TX 8/2048, RX 8/2048 ... lo0: link state changed to UP ix1: link state changed to UP ix1: link state changed to DOWN igb1: link state changed to UP ix1: link state changed to UP pflog0: promiscuous mode enabled re0: link state changed to DOWN igb1: promiscuous mode enabled ...
WAN is on ix1, LAN is on igb1, ix0, igb0, and re0 (motherboard) are not currently connected. Eventually ix0 will connected to a 10GBASE-T SFP+ transceiver on a 10G switch as soon as I get around to installing the switch with another 10G Cat8.
So the hardware still seems happy to auto-negotiate to 2500Mbps, even though the FreeBSD ix driver can't report it. ¯_(ツ)_/¯ Annoying, but usable. Where do I sign the "Dear Intel" "metoo" petition? ;-)
-
The presence of
<rxpause,txpause>
there indicates that flow control is active on that link. Not that it's actually paused at that moment. -
I knew that! :-)
Sorry, I should have rephrased that as “Flow Control appears to be working.” Rather than saying the NIC does not appear to be paused. Doh! ;-)
Any guess on when/if this could be addressed by Intel or someone else?
-
No new information there I'm afraid.
-
2.5/5G added to the kernel's ixgbe driver with this commit: https://cgit.freebsd.org/src/commit/sys/dev/ixgbe/ixgbe.h?id=d381c807510de2ebb453a563540bd17e344a2aab
-
@brian-cunnie said in NBASE-T Support for Intel X550:
https://cgit.freebsd.org/src/commit/sys/dev/ixgbe/ixgbe.h?id=d381c807510de2ebb453a563540bd17e344a2aab
Any idea on approximately when this should show up in a pfsense RELEASE? Would I have to change anything?
-
It's only in 13-stable right now so at the very least it would need either to brought into 12 stable or pfSense moved to 13 stable (or newer).
-
@daplumber I don't have any idea when it would show up in a pfsense release.
Also, I've been struggling to get this to work on my non-pfsense straight-FreeBSD 13 firewall, and in spite of several attempts the Intel X550 negotiates a 1Gbe not 2.5Gbe connection. When I get it working I'll update the thread.
-
@brian-cunnie I have found that the connection needs to be pretty clean. I use a Cat8 factory made cable. The chipset also runs pretty warm, I had to ensure good thermal grease and adequate air flow.
-
Interesting. One of the big advantages of 2.5G is that it runs over cat5e. I've never had an issue using igc. Not tried an x550 though.
Steve
-
@stephenw10 I know that’s the spec. I just had zero auto negotiation issues when I switched to a Cat8 cable, I went from Cat5 straight to Cat8 as they were weirdly cheaper at the time, so I skipped Cat5e, Cat6, and Cat7. I just wanted it to work. The other end is an Arris S33 Cable modem which I think is some kind of Broadcom chipset? I don’t know if different chipsets/drivers are better at auto-negotiating?
-
Solution: the reason I've been having trouble getting my FreeBSD-based firewall running on an Atom C3000 SoC to negotiate 2.5Gbe with my Arris S33 modem is that the custom Intel 10G X550 on the SoC does not do NBASE-T—either you get 1Gbe or 10Gbe and nothing in between. Sigh.
On the bright side, I learned a lot about building the FreeBSD kernel and a reasonable amount about the
ixgbe
driver source code and about reading Intel datasheets. I must admit it was kind of fun.I'll revert to plugging the S33 & the firewall into a 1/2.5/5/10 switch, and that'll make it work. I was hoping for a direct connect, but that's not possible.
-
@brian-cunnie Well that’s a bummer. I guess they simplified the network part on the SoC version. I didn’t even realize that existed!
-
Hmm, X553 claims to support 2.5G, but not autonegotiate at that:
https://www.intel.co.uk/content/dam/www/public/us/en/documents/manuals/atom-c3000-family-programmer-reference-manual.pdf#G2.1229696I imagine the external components would also need to.
Steve
-
@stephenw10 Good catch! I didn't realize that C3000 SoC X553 allows 2.5Gbe.
I hacked the
ixgbe
driver to allow 2.5Gbe for my card, ransudo ifconfig ix0 media 2500Base-T
to set it to 2.5Gbe, and then attached it to the Arris S33, but the carrier never came up, and I couldn't communicate with the Arris. -
Hmm, could be a number of things. You think your code hack was good though?
-
@stephenw10 I would say I'm 80% confident my hack was correct. Here's the code diff:
--- a/sys/dev/ixgbe/if_ix.c +++ b/sys/dev/ixgbe/if_ix.c @@ -1286,7 +1286,8 @@ ixgbe_add_media_types(if_ctx_t ctx) if (layer & IXGBE_PHYSICAL_LAYER_10BASE_T) ifmedia_add(sc->media, IFM_ETHER | IFM_10_T, 0, NULL); - if (hw->mac.type == ixgbe_mac_X550) { + if (hw->mac.type == ixgbe_mac_X550 || + hw->mac.type == ixgbe_mac_X550EM_a) { ifmedia_add(sc->media, IFM_ETHER | IFM_2500_T, 0, NULL); ifmedia_add(sc->media, IFM_ETHER | IFM_5000_T, 0, NULL); } @@ -1438,8 +1439,6 @@ ixgbe_config_link(if_ctx_t ctx) * autonegotiation preferences and the default * set of speeds should exclude 2.5G and 5G. */ - autoneg &= ~(IXGBE_LINK_SPEED_2_5GB_FULL | - IXGBE_LINK_SPEED_5GB_FULL); } if (hw->mac.ops.setup_link) @@ -2265,7 +2264,8 @@ ixgbe_if_media_status(if_ctx_t ctx, struct ifmediareq * ifmr) ifmr->ifm_active |= IFM_10_T | IFM_FDX; break; } - if (hw->mac.type == ixgbe_mac_X550) + if (hw->mac.type == ixgbe_mac_X550 || + hw->mac.type == ixgbe_mac_X550EM_a) switch (sc->link_speed) { case IXGBE_LINK_SPEED_5GB_FULL: ifmr->ifm_active |= IFM_5000_T | IFM_FDX; @@ -4283,8 +4283,6 @@ ixgbe_get_default_advertise(struct ixgbe_softc *sc) * are disabled by default due to reported * interoperability issues with some switches. */ - link_caps &= ~(IXGBE_LINK_SPEED_2_5GB_FULL | - IXGBE_LINK_SPEED_5GB_FULL); } speed = --- a/sys/dev/ixgbe/ixgbe_x550.c +++ b/sys/dev/ixgbe/ixgbe_x550.c @@ -1945,6 +1945,8 @@ s32 ixgbe_get_link_capabilities_X550em(struct ixgbe_hw *hw, /* fall through */ default: *speed = IXGBE_LINK_SPEED_10GB_FULL | + IXGBE_LINK_SPEED_2_5GB_FULL | + IXGBE_LINK_SPEED_5GB_FULL | IXGBE_LINK_SPEED_1GB_FULL; break; }
-
Mmm, well that doesn't look wrong.
But I couldn't tell you if that's all that's required there.