Cannot get 10Gbase-T on LAN (only 1000baseT)



  • Dear PFSense community,

    I've installed an intel x540-t2 in my pfsense Box for the LAN but it's only detecting as 1000baseT. I've been stumped on this for a while.

    If anyone out there has experience with this and can throw me some help, I would really appreciate it.

    Good day!



  • What's on the other end of the cable? 10G has to be supported on both ends.


  • Netgate Administrator

    Assuming the other end is 10G capable perhaps it's fixed to 1G?

    If it isn't have you tried setting a fixed 10G link speed in pfSense?

    Steve



  • @jknott Hi JKnott, the Intel NIC is dual port. So, so one port is connected to a 10 gigabit pro safe switch and second port on the nic is connected to my 1 gigabit shaw modem. Could that be why? Is pfsense normally supposed to detect 10g automatically?



  • @stephenw10 Hi stephenw10, the Intel NIC is dual port. So, so one port is connected to a 10 gigabit pro safe switch and second port on the nic is connected to my 1 gigabit shaw modem. I think I have tried forcing 10 gigabit on the LAN interface and it didn't work. I'll try it again right now though to double check



  • @techgeek055 odd, when i click apply I get:
    PPP Password and confirmed password must match!
    PTPP Password and confirmed password must match!

    Never got that before :D


  • Netgate Administrator

    Sounds like your browser might be filling in some form field when you try to edit the interface. Or maybe you have a left over setting somewhere.

    I would expect that NIC to auto-negotiate 1G/10G but I have seen some Intel cards that did not and would only establish a link when hard set.

    Steve



  • @techgeek055 Actually Pfsense, the App, doesn't preocuppy itself with NIC speeds, other than ask the OS what is and display it to you. This maybe a case of living on the bleeding edge, may need a proper NIC driver for FreeBSD.



  • @stephenw10 You were right about the browser trying to fill in a password causing the two errors :D

    I am going to try moving the WAN connected to the 1gigabit shaw modem off of the 10 gigabit intel nic and onto it's own NIC and see if that makes any difference!



  • looks like leaving just the 10g connection for lan on the intel nic didn't get pfsense to detect 10g conneciton. (and removing the 1Gb connection from the intel nic)

    It let's me pick 10Gbase-T on the interface settings but when I look at the main page that monitors the interfaces, the LAN still says: 1000baseT <full-duplex,rxpause,txpause>

    On my WAN interface settings there is no 10Gbase-T option, only 1000Base-T. So it appears that the drop down at least detects 10Gbase-T and allows you to select it as an option but it appears to do nothing and the monitor still says 1000base-T.

    :(



  • @sammywoo It lets me pick 10Gbase-T on the LAN interface settings but when I look at the main page that monitors the interfaces, the LAN still says: 1000baseT <full-duplex,rxpause,txpause>.

    If I try to pick 10Gbase-T on my WAN connection which is using a 1000base-T nic, the option of 10Gbase-T is not there. So at least in the interface settings, it seems to detect the 10Gbase-T and you can pick it. But monitor still says my LAN is on 1000base-T :(

    Do you think it's the driver still? If so, is the process easy? do you have a resource for instructions?


  • Netgate Administrator

    If you set it to 10G and it make no difference I'd say that setting hasn't applied. Reboot to be sure.

    If the other end is set to 1G for some reason then setting it to 10G fixed should cause the link to fail.

    It could be a cable that can't support 10G perhaps?

    Steve



  • @stephenw10 Hi, rebooting made no difference. The other end is 10G and the cable is CAT 7 I believe or at least CAT 6A.

    Maybe I just need to rebuild the whole pfsense firewall. :(



  • @techgeek055 said in Cannot get 10Gbase-T on LAN (only 1000baseT):

    If I try to pick 10Gbase-T on my WAN connection which is using a 1000base-T nic, the option of 10Gbase-T is not there.

    That might be a clue right there. If your NIC doesn't support 10G, then no setting will make it. 10G is 10 Gigabit, which is 10x what your NIC supports.



  • One other thing, normally there's no need to set the bandwidth on an interface. NICs use auto-negotiation to automagically determine the best possible bandwidth. In fact, manually setting it can cause problems, if you mismatch the interfaces.



  • @jknott Thanks JKnott, I will put WAN and LAN back on auto negotiate. But my interfaces on the monitor on the main page still says 1000Gbase-T. WAN is correct, since my modem is only 1Gigabit but for my LAN everything on both ends and inbetween are 10Gigabit hardware.

    I'm probably just going to re-build the firewall from scratch over the weekend. I'll let ya'll know how it goes.



  • @techgeek055 said in Cannot get 10Gbase-T on LAN (only 1000baseT):

    WAN is correct, since my modem is only 1Gigabit but for my LAN everything on both ends and inbetween are 10Gigabit hardware.

    What's on the other side of the switch is irrelevant. Each link will auto-negotiate independent of what other links on your LAN are doing. So, for you to get 10Gb, both the device and the switch need to support 10 GB. Does your switch really support 10G? Switches that do tend to be a bit on the expensive side.



  • @jknott Yes sir! My switch is 10 gigabit as well. Model: Netgear ProSAFE 16-Port 10-Gigabit Ethernet Smart Managed Switch (XS716T-100NES). My computer and VMs all say 10 gigabit without any tinkering.



  • How long is the cable?

    Double-check that the switch port is set to Auto. If it is, try with another cable. The cable could be damaged.

    I've had no problems with the similar Netgear XS712T. It have negotiated properly with Intel X550-T2, QNAP QXG-10G1T and even low quality Startech ST10000SPEX but always only with short (2-5 m) cables.



  • @p3r said in Cannot get 10Gbase-T on LAN (only 1000baseT):

    Double-check that the switch port is set to Auto. If it is, try with another cable. The cable could be damaged.

    As mentioned earlier, he was trying to get 10 Gb from a Gb NIC. That tends not to work that well. Also, handshaking takes place at 10 Mb, so if the cable is good at that rate, the NICs will try to connect at the best speed, not knowing the cable may be defective.



  • @p3r Hi P3R, my cable going from LAN port to my netgear switch is very short too, 2-4 ft. I turned the LAN port to auto negotiate in my pfsense gui. I'll try swapping out the cable!



  • @jknott said in Cannot get 10Gbase-T on LAN (only 1000baseT):

    As mentioned earlier, he was trying to get 10 Gb from a Gb NIC.

    I'm sorry but I can't find that in the thread but Intel X540-T2 (as reported in the first post) is a 10 GbE NIC.



  • @techgeek055 said in Cannot get 10Gbase-T on LAN (only 1000baseT):

    I turned the LAN port to auto negotiate in my pfsense gui.

    Yes but I asked you to double-check that also the switch port is set to Auto negotiation:
    0_1543091180628_XS712T.jpg



  • @techgeek055 said in [Cannot get 10Gbase-T on LAN (only 1000baseT)]

    I'm probably just going to re-build the firewall from scratch over the weekend. I'll let ya'll know how it goes.

    Well first of all make sure 10Giga nics works in p2p mode, I think some like linux live distro loaded on pfsense box and watch it shows about auto configured ethernet link!, Best simply pratice to remove iOperating system and driver issue

    So go to load your favourite linux distro on it! Yeah, it's a way to figure it out for "ensure" it is OK at full speed or NOK right now.
    Bye



  • I "think" (i'm not 100% sure) I know what happened now. I was using a dual port 10 gigabit nic (intel x540-t2). 1 port was the WAN and one port was the LAN. I installed another 10 gigabit nic card to use as the LAN and it is detected as 10Gbase-T on LAN now. Maybe both ports on the 10 gigabit intel x540-t2 cannot run 10 gigabit simultaneously?



  • @techgeek055 said in Cannot get 10Gbase-T on LAN (only 1000baseT):

    intel x540-t2

    From Datasheet :Product Overview
    "The X540 is a derivative of the 82599, the Intel 10 GbE Network Interface Controller
    (NIC) targeted for blade servers. Many features of its predecessor remain intact;
    however, some have been removed or modified as well as new features introduced.
    The X540 includes two integrated 10GBASE-T copper Physical Layer Transceivers (PHYs).
    A standard MDIO interface, accessible to software via MAC control registers, is used to
    configure and monitor each PHY operation.
    The X540 also supports a single port configuration."

    So, if "single" configuration is "supported", maybe better check defaults intel bsd driver variable, with sysctl...
    And reading datasheet of hardware is also good pratice. Look at "Network Features" you can see X540 NOT support of non Auto-Negotiation Partner
    and others useful info, under good Intell product datasheet 🌮
    alt text

    Oh and when I reading datasheet and is very full of hints about how X540 chipset work.

    Last advice, move your nic to other next 8X PCIexpress slot and check PCIe bandwidth will help, I mean, you need to "reserve" almost 20 Gbits of bandwidth from PCIe bus, for allowing proper work of this chipset with high load
    . Server motherboard must do it if enough resoures is available, so handle this resouce with a motherboard full of dual 10Gbe nic's 🍎 (just kidding) may lower overall performance.
    Or I'm just go to bananas 🍺



  • @p3r said in Cannot get 10Gbase-T on LAN (only 1000baseT):

    @jknott said in Cannot get 10Gbase-T on LAN (only 1000baseT):

    As mentioned earlier, he was trying to get 10 Gb from a Gb NIC.

    I'm sorry but I can't find that in the thread but Intel X540-T2 (as reported in the first post) is a 10 GbE NIC.

    He said it in the line "If I try to pick 10Gbase-T on my WAN connection which is using a 1000base-T nic, the option of 10Gbase-T is not there.", about 6 months ago.



  • @jknott said in Cannot get 10Gbase-T on LAN (only 1000baseT):

    He said it in the line "If I try to pick 10Gbase-T on my WAN connection which is using a 1000base-T nic, the option of 10Gbase-T is not there.", about 6 months ago.

    If you read it together with the sentence following it, you'll understand that was mentioned only to show that his pfSense does detect the difference between that gigabit NIC compared to the 10 GbE Intel X540-T2 NIC.

    Unless I've misunderstood everything the WAN connection mentioned there isn't what he's having problems with (as that's connected to a gigabit modem), it's the LAN connected to the Netgear 10 GbE switch that doesn't give him 10 Gbps.


  • Netgate Administrator

    I would certainly expect to see 10G on both ports of that card. It looks like there may be some low level incompatibility there if you have tried swapping everything.
    What is the other card you have installed that can link at 10G?

    Steve