Intel X550-T2 proper setup?
-
I've read the https://forum.netgate.com/topic/146913/nbase-t-support-for-intel-x550/77 in its entirety in anticipation of an Intel X550-T2 arriving by 10pm this evening. There are requirements therein that I am uncertain how best to achieve.
pfSenseCE v2.7.2 is installed as a VMware Workstation 17 virtual machine on an HP z840 w/ dual Xeon E5-2690 v3 and 512GB memory. The pfSenseCE is allocated 8 CPUs and 16GB RAM; nonetheless, it has proven incapable of negotiating a WAN IP under all flavors Intel & RealTek 2.5Gbs NICs. Only if the ISP modem data-stream is attenuated by introducing a router with max 1Gbps downstream between the modem and the pfSense WAN can a WAN IP be gained.
The Intel X550-T2 setup appears to be the best solution.
In the NBASE-T support thread it is asserted:
I can confirm that setting
sysctl dev.ix.X.advertise_speed=N
Where X is the interface number (e.g. for ix0, X=0 in the command above) and N is the sum of the advertised speeds, (N = 55) works. I also had to add dev.ix.0.advertise_speed as a system tunable, with value 55, to make it persist between reboots.First question: Exactly how and where does one issue this sysctl command?
Is Diagnostics/ Command Prompt sufficient? Meaning one can simply enter the sysctl line above into the Execute Shell Command space and then click on the Execute button?
Or does one need to SSH into pfSenseCE and regain familiarity with Linux? The pfSense documentation speaks of a Console Menu appearing upon starting an SSH session. Sadly, I am not seeing that.
Secondly, should both X550-T2 ports be set to WAN now leaving one simply with no Ethernet attachment or just ignore its presence entirely?
Knowing these will at least get me started when it arrives.
-
@gunnyp said in Intel X550-T2 proper setup?:
pfSenseCE v2.7.2 is installed as a VMware Workstation 17 virtual machine on an HP z840 w/ dual Xeon E5-2690 v3 and 512GB memory. The pfSenseCE is allocated 8 CPUs and 16GB RAM; nonetheless, it has proven incapable of negotiating a WAN IP under all flavors Intel & RealTek 2.5Gbs NICs. Only if the ISP modem data-stream is attenuated by introducing a router with max 1Gbps downstream between the modem and the pfSense WAN can a WAN IP be gained.
What is the WAN connected to otherwise? How is that configured?
How is the NIC passed to pfSense? If it's presented as vmxnet then the link negotiation is all down to VMWare and pfSense never sees it.
Steve
-
@stephenw10
Thanks for the reply!Just finished installing the X550-T2 an hour ago. The WAN is connected by CAT8 to a Netgear CM2000 which has on only 3 occasions in 3 weeks ownership negotiated a WAN IP successfully; each time with a blue LED indicative of over 1Gbs speed. Else a wifi router was added ISP/CM2000/Router/pfSense which always worked instantly but at under 1Gbs speed. On Spectrum which seems to max at 1.2Gbs.
Attempted to update X550 drivers to "Wired driver 29.1 for Windows Server 2019" downloaded from Intel but get "The best drivers for your device are already installed." for driver version 4.1.229 from 2020.
VMware Virtual Network Editor was used to add a Bridged connection to the X550 on VMnet0. Which was then chosen as the Custom: Specific virtual network for the WAN pfSense Network Adapter. The pfSense LAN is likewise a bridged X540-T2 connected to a 12 port 10GB Managed Switch. Believe this is as close to bare metal as possible. Don't see any vmxnet3 until vms on ESXi servers further downstream.
On bootup of the pfSense it was unable to negotiate with the CM2000 modem at all and presented recorded leases:
Jul 8 18:27:42 dhclient 67162 FAIL
Jul 8 18:27:42 dhclient 38012 No working leases in persistent database - sleeping.
Jul 8 18:27:42 dhclient 65607 Deleting old routes
Jul 8 18:27:41 dhclient 62713 New Routers (em0): 192.168.1.1
Jul 8 18:27:40 dhclient 21136 New Routers (em0): 192.168.1.1
Jul 8 18:27:40 dhclient 20443 New Broadcast Address (em0): 192.168.1.255
Jul 8 18:27:40 dhclient 19759 New Subnet Mask (em0): 255.255.255.0
Jul 8 18:27:40 dhclient 18647 New IP Address (em0): 192.168.1.3
Jul 8 18:27:40 dhclient 16429 ifconfig em0 inet 192.168.1.3 netmask 255.255.255.0 broadcast 192.168.1.255
Jul 8 18:27:40 dhclient 15247 Starting add_new_address()
Jul 8 18:27:40 dhclient 14362 TIMEOUT
Jul 8 18:27:40 dhclient 38012 Trying recorded lease 192.168.1.3
Jul 8 18:27:40 dhclient 38012 No DHCPOFFERS received.
Jul 8 18:27:34 dhclient 38012 DHCPDISCOVER on em0 to 255.255.255.255 port 67 interval 6The Interfaces/WAN/Speed & Duplex offered only 10/100/1000baseT speeds. Only by setting this to 1000baseT full-duplex was the CM2000 modem able to see the pfSense dhclient DHCPDISCOVER and respond with a private address DHCPOFFER which 2 minutes later was dropped for the necessary public WAN address. This is the fastest presentation of a public WAN but speed is abysmal at 285Mbps down.
Attempted to issue the command sysctl dev.ix.X.advertise_speed=N after entering the Shell from the Console Menu and also via the Diagnostics/Command Prompt to no avail. For some reason on this pfSense the WAN is em0, LAN em1 rather the ix0/1 most seem to see.
Probably need to manually install the 2024 drivers for the X550 to make any headway and finally see the additional Speed and Duplex settings. -
@gunnyp
https://www.reddit.com/r/vmware/comments/xlvopc/vmware_workstation_25gbps_adapter/?rdt=35176If you want to pass thru the pcie device to virtual machine you should use ESXi not Workstation.
-
Hmm, yeah I'm not sure how you would set that up in VMWare workstation. pfSense there is seeing that as an em NIC which can only ever connect at 1G and is single queue.
But that wouldn't prevent the host linking at whatever speed it wants to. The NIC pfSense sees in that setup is entirely virtual.
-
@w0w said in Intel X550-T2 proper setup?:
https://www.reddit.com/r/vmware/comments/xlvopc/vmware_workstation_25gbps_adapter/?rdt=35176
Yes! Have to find time to reconfigure it properly as its used primarily as a server now. Frankly, all has lain nearly dormant since 2019; relearning everything.
Managed to get the X550-T2 drivers updated to version 4.1.254.0 dated 2/8/2024 which must be the Windows Server 2019 designation as it was billed as Drivers Release 29.2. With the updated driver, was able to switch Interfaces/WAN/Speed to autoselect rather than forcing 1000baseT and receive WAN IP albeit at 1000baseT speed still. It remains that only 10/100/1000baseT speed selections are offered.
Next will install the X550 NVM Update v3.6 Package and Intel ProSet drivers.
-
@stephenw10
That is bewildering!
When the Spectrum modem was replaced with the Netgear CM2000 and a TP-link 2.5Gbps NIC nearly a month ago both devices LED indicators were lit for above 1Gbps traffic and Spectrum speed test hit just under 1200Mbps. This was run on a vm connected via vmxnet3 to a 10Gb switch.
Just last week after new cable was run the Spectrum tech measured the speed on his equipment as I watched and it was over 1Gbps. Ah, this was just at the modem so doesn't count.
The problem was after he left even with the new cable, it dropped the connection so often and endless power cycles succeeded very, very rarely.
What has been gained with the X550-T2 is an ability to at least negotiate a connection that seems repeatable and hopefully dependable.
So you think it would be best to stop now, wipe the C drive which has only windows server and vmware workstation as the sole apps, to install ESXi?
Oh, you know what, I'm literally tripping over ethernet cables attached to a humongous water-cooled ESXi v7.03 server. If ESXi is the solution, I'll just pull the X550 out of the HP and bung it into an ESXi server! -
Well at least change the NIC type in the vm to vmxnet. em(4) is a gigabit NIC so you will never see >1Gbps through it. And since it's single queue it will only use one CPU core each way.
But yes move it to ESXi and use real hardware pass-through.
-
@stephenw10
Hours later... As suggested moved pfSense vm to exsi host using X550-T2 as WAN and the motherboard had built-in dual 10Gb ethernet that have always been LAN. It would not accept pass-through of the X550; kept getting "config error" with no further elaboration. Also unable to find exsi drivers for the X550-T2 amongst the Intel all available drivers download. Apparently only accessible through vmware.And again... Completely forgot one can update VCSA and download drivers. Will have a go after a read-up on how that's done again.
The modem negotiation took about a minute to produce a public WAN address so that headache at least remains resolved. However, the only WAN speed available on Interfaces/WAN/Speed is autoselect. The Status/Dashboard/Interfaces also has autoselect rather than the baseT indicator as well. A Okla speedtest has improved moderately to 670Mbps but modem LED indicates under 1Gb only.
The ESXi drivers on the server are of the ixgben variety. Stopped updating vCenter with the v7.03c release in early 2022 so these drivers aren't that long in the tooth. The WAN/LAN present as vmx1/vmx0. Is it possible to introduce the system tunables at this juncture or is pass-through a necessity before that can be attempted?
-
You can add the tunables for vmxnet:
https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#vmware-vmx-4-interfacesUntil you get PCIe passthrough working though pfSense cannot 'see' the real NIC at all. That's all down to ESXi. I have no idea if that will link at 2.5G or how to do it.
You could always try pfSense bare metal if you have hardware.
-
@gunnyp
Is it original Intel card? What firmware?
esxi and x550-t2 -
@w0w
Yup! Original! Made in China! Manufactured in June 2024, so hot off the boat from the Vogzone Store, sellers on Amazon.
Was going to check further into their credentials but inside the box was a 2"x2" white cardboard square with the word "CERTIFICATE"!
Why do you ask?https://www.amazon.com/dp/B07DHDW59N?th=1
As to firmware, there is very little information
All Properties CDP LLDPAdapter Intel(R) Ethernet Controller X550
Name vmnic2
Location PCI 0000:81:00.0
Driver ixgben
Status
Status Disconnected
Actual speed, Duplex Down
Configured speed, Duplex Auto negotiate
Networks No networks
SR-IOV
Status Disabled
Cisco Discovery Protocol
Cisco Discovery Protocol is not available on this physical network adapter
Link Layer Discovery Protocol
Link Layer Discovery Protocol is not available on this physical network adapter -
@gunnyp
This is definitely not an Intel verified card, there is no yotamark on it
This one is original:
Also yours is made "for Intel"
You can try to update nvme firmware on it and see what happens.As for the current fw version determination you can try
This link -
@w0w
Admit next day delivery trumped all else. Tomorrow returning two modems, three purportedly Intel i-226 2.5Gb cards and a RealTek as well.
And thanks! Had just stumbled upon the VMware Compatibility Guide. Oh, the hoops they make one jump! And its 1am. Good night! -
@w0w
5 days later...After quite the ordeal, can confirm that the Vogzone X550-T2 works pleasing well despite its questionable origins. It is managing 1150 Mbps / 39.5 Mbps avg on Ookla speedtest on Spectrum 1G.
Decided to switch all servers from ESXi to Hyper-V as twas embedded on the Windows Server 2019s which provided DC/DNS/DHCP on every ESXi server here already. With updated Intel drivers, PROSet and NVM was easily able to select the 2.5 Gbps speed, not on pfSense but on PROSet wherein one can hard set the negotiated speed. pfSense Interfaces show 10GbaseT speed and Interfaces / WAN / Speed Duplex is on Default.
Now to become Hyper-V aware.
Thanks to both of you for your immeasurable help!
