PSA: Do not attempt Intel NVM Updates on NICs embedded onto Supermicro MBs
-
Just a heads up that if you attempt to install updated Intel NVM firmware onto an embedded NIC like the Intel X550-T2 you could break one of its ports. This applies to these MBs and possibly others too:
X12SDV-4C-SPT4F
X12SDV-8C-SPT4F
X12SDV-10C-SPT4FIf this happens to you, contact Supermicro and they can provide the correct Intel NVM update with a special config to get it back to functioning firmware.
Lesson learned.
-
@InstanceExtension
Thanks for the warning. Does the revised firmware fix the upgrade issue permanently or will we need to approach Supermicro for all future updates?What prompted you to try an update?
️ -
@RobbieTT The Supermicro supplied update just forced the Intel NVM firmware back to the shipped value, 3.50. I'm pretty sure it would have issues again if the latest version from Intel, 3.60 was re-applied.
As to why I attempted the firmware update is that Intel recommends keeping the firmware and drivers in sync with each other. If you;re using the latest drivers you should also be using the latest firmware.
https://www.intel.com/content/www/us/en/download/19358/non-volatile-memory-nvm-update-utility-for-intel-ethernet-network-adapter-x550-series.html
"With the newest line of Intel
10 GbE adapters, both the firmware (device NVM
image) and network drivers are field-serviceable, and the NVM image and network
driver are updated as a matched set. Updating the device image and driver together
can increase key features including performance, manageability, media types, physical
port counts, virtualization, offloads, remote boot options, VLAN support, teaming, and
Receive Side Scaling." -
@InstanceExtension said in PSA: Do not attempt Intel NVM Updates on NICs embedded onto Supermicro MBs:
As to why I attempted the firmware update is that Intel recommends keeping the firmware and drivers in sync with each other. If you;re using the latest drivers you should also be using the latest firmware.
I ask as I have some quirks with my Intel NICs with pfSense/BSD that had me looking at doing updates to my embedded NICs. On my Supermicro Xeon-D I have 2 x Intel 810 series NICs (E823-L in this case) running on v1.37.11. The latest v1.38.16 includes specific base-drivers for FreeBSD, rather than the generic earlier versions.
I've never tried to update an embedded NIC before and have little idea how to do it but with the specific changes for BSD it would seem like a good move.
I also have this oddity in sysctl (plus I had to manually add a
ice_ddp_load="YES"entry to my load.conf.local to load the DDP package):ice0: Loading the iflib ice driver ice0: The DDP package was successfully loaded: ICE OS Default Package version 1.3.30.0, track id 0xc0000001. ice0: fw 5.5.17 api 1.7 nvm 2.28 etid 80011e36 netlist 0.1.7000-1.25.0.f083a9d5 oem 1.3200.0 ice0: Using 8 Tx and Rx queues ice0: Reserving 8 MSI-X interrupts for iRDMA ice0: Using MSI-X interrupts with 17 vectors ice0: Using 1024 TX descriptors and 1024 RX descriptors <6>ice0: Ethernet address: 3c:ec:ef:dc:eb:dc ice0: PCI Express Bus: Speed 2.5GT/s Width x1 ice0: PCI-Express bandwidth available for this device may be insufficient for optimal performance. ice0: Please move the device to a different PCI-e link with more lanes and/or higher transfer rate. ice0: Firmware LLDP agent disabled <6>ice0: link state changed to UP <5>ice0: Link is up, 10 Gbps Full Duplex, Requested FEC: None, Negotiated FEC: None, Autoneg: False, Flow Control: None <6>ice0: netmap queues/slots: TX 8/1024, RX 8/1024I have not noted any throughput issue (albeit I only run 10GbE rather than 25GbE) and I don't understand the PCIe bandwidth restriction on an embedded NIC on a mixed PCIe3.0/4.0 motherboard.
Could you share your method of updating the embedded NIC firmware?
️ -
@RobbieTT Its very easy to do. You just need to download the correct NVM firmware for your NIC and run the nvmupdatew64e utility to do the update. The utility has version that can be run in Windows, Linux, FreeBSD and EFI. I recommend running the EFI version to eliminate any possible OS issues/constraints.
But I suspect that the E823-L we have on our boards cannot use the generic Intel firmware just as I found with the X550-T2. You'll need to contact Supermicro and see if they would be willing to provide the updated NVM firmware for the 820 series.
Looks like Intel has not publicly released specific NVM updates for the 820 series yet. I'm only seeing NVM updates for the 810 series, but I believe that the 820 series might be included.
https://www.intel.com/content/www/us/en/download/19624/non-volatile-memory-nvm-update-utility-for-intel-ethernet-network-adapter-e810-series.htmlAs to the bandwidth restriction notification from the driver, this was mentioned in the comments on a review of this board (albeit the micro-itx version).
https://www.servethehome.com/supermicro-x12sdv-4c-sp6f-review-25gbe-and-intel-xeon-d-1718t/3/
jode February 7, 2023 At 9:20 am
What is the measured performance of the 25gb network port? I am asking because I don’t expect much based on the single lane PCIe Gen 2 connection documented by the lspci outputPatrick Kennedy February 7, 2023 At 9:41 am
We run in the 24-25Gbps range on those ports fairly easily. The 25GbE ports are SoC, not PCIe devices like the i210/i350. IIRC the D-1700 series actually has a 100GbE capable network segment so 2x 25GbE is not huge. -
@InstanceExtension said in PSA: Do not attempt Intel NVM Updates on NICs embedded onto Supermicro MBs:
Patrick Kennedy February 7, 2023 At 9:41 am
Ok, understood - these 2 interfaces are hanging off OpenFabric rather than PCIe - so not exactly short of bandwidth then!
️
