XXV710-DA2 Driver Issue - Firmware Newer Log Note

sheen73

Hello all,
Hope everyone is good.

I am currently running 23.05 using a genuine Intel XXV710-DA2 sfp28 dual port card bare metal (ignore sig cant change it). Prior to building my new system I updated the firmware for the card to 9.20. I am seeing the below message in OS log and wonder if there is a way to update the driver? For the most port everything is working fine, however I am having some issues with my intel transceiver 1/2.5/5/10Gbe working at 2.5Gbe when connected to teh 2.5 port of my Arris S33 Modem. The specific issue is I can connect, but only get 500Mbps max when on the 2.5G port and hit nearly 1G on the 1Gbe port. I understand this could be related to flow control, ive tried several settings in loader.conf.local and nothing seemed to work to improve it.

I am also concerned because the supported media only reflects 10 base-sr where the XXV710-DA2 is 1/10/25. So wanted to eliminate the driver issue 1st. Thoughts? Should i downgrade firmware? To which version?

ixl1: <Intel(R) Ethernet Controller XXV710 for 25GbE SFP28 - 2.3.3-k> mem *removed* at device 0.1 on pci1
ixl1: fw 9.120.73026 api 1.15 nvm 9.20 etid 8000d89c oem 1.269.0
ixl1: The driver for the device detected a newer version of the NVM image than expected.
ixl1: Please install the most recent version of the network driver.

ixl0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: WAN
	options=4e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
	capabilities=4f507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
	ether 40:a6:b7:a8:f9:60
	inet6 fe80::42a6:b7ff:fea8:f960%ixl0 prefixlen 64 scopeid 0x1
	inet 45.50.69.52 netmask 0xffffe000 broadcast 255.255.255.255
	media: Ethernet autoselect (10Gbase-SR <full-duplex>)
	status: active
	supported media:
		media autoselect
		media 10Gbase-SR
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

stephenw10

I would hope the NVM version is backwards compatible so that's unlikely to be an issue IMO. That is close to the latest FreeBSD driver anyway so you could only address that by downgrading the NVM.

Does that card actually support 2.5G? What does ifconfig -vvvm ixl0 show? Does it show differently when connected to the 1G or 2.5G ports?

Steve

sheen73

@stephenw10 said in XXV710-DA2 Driver Issue - Firmware Newer Log Note:

ifconfig -vvvm ixl0

Thank you for the response and information. I didn't think it was pfsense or the driver just wanted to rule it out. The card only supports 1/10/25G. I suspect its my transceiver which is connecting at 10, but should translate down to 2.5. To many people seem to have issues with 2.5 transceivers.

To answer your question, this is the modem on 1 Gbe port

ixl0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: WAN
	options=4e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
	capabilities=4f507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
	ether xxxx
	inet6 xxx prefixlen 64 scopeid 0x1
	inet xxx netmask xxxx broadcast 255.255.255.255
	media: Ethernet autoselect (10Gbase-SR <full-duplex>)
	status: active
	supported media:
		media autoselect
		media 10Gbase-SR
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	plugged: SFP/SFP+/SFP28 10G Base-SR (LC)
	vendor: OEM PN: SFP-10G-SR SN: CY2212270640 DATE: 2022-12-26

	SFF8472 DUMP (0xA0 0..127 range):
	 03 04 07 10 00 00 00 00 00 00 00 06 67 00 00 00
	 08 02 00 1e 4f 45 4d 20 20 20 20 20 20 20 20 20
	 20 20 20 20 00 00 90 65 53 46 50 2d 31 30 47 2d
	 53 52 20 20 20 20 20 20 30 32 20 20 03 52 00 70
	 00 1a 00 00 43 59 32 32 31 32 32 37 30 36 34 30
	 20 20 20 20 32 32 31 32 32 36 20 20 00 80 03 22
	 00 00 11 12 5e 86 dd 45 68 13 dd 3f cf d0 f9 12
	 33 af 48 00 00 00 00 00 00 00 00 00 bd 20 a7 e8

and this is connected to 2.5 Gbe port on modem

ixl0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: WAN
	options=4e100bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
	capabilities=4f507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
	ether XXX
	inet6 xxxx prefixlen 64 scopeid 0x1
	inet xxx netmask xxx broadcast 255.255.255.255
	media: Ethernet autoselect (10Gbase-SR <full-duplex>)
	status: active
	supported media:
		media autoselect
		media 10Gbase-SR
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	plugged: SFP/SFP+/SFP28 10G Base-SR (LC)
	vendor: OEM PN: SFP-10G-SR SN: CY2212270640 DATE: 2022-12-26

	SFF8472 DUMP (0xA0 0..127 range):
	 03 04 07 10 00 00 00 00 00 00 00 06 67 00 00 00
	 08 02 00 1e 4f 45 4d 20 20 20 20 20 20 20 20 20
	 20 20 20 20 00 00 90 65 53 46 50 2d 31 30 47 2d
	 53 52 20 20 20 20 20 20 30 32 20 20 03 52 00 70
	 00 1a 00 00 43 59 32 32 31 32 32 37 30 36 34 30
	 20 20 20 20 32 32 31 32 32 36 20 20 00 80 03 22
	 00 00 11 12 5e 86 dd 45 68 13 dd 3f cf d0 f9 12
	 33 af 48 00 00 00 00 00 00 00 00 00 bd 20 a7 e8

stephenw10

Hmm, interesting. Do you see errors on the interface in Status > Interfaces in either situation?

sheen73

@stephenw10

No sir, looks good.
Not sure if this means anything, but speed and duplex options are different between lan and wan. Both have same transceiver in them, WAN is connected to s33 arris modem and lan is to my unifi US8-150 switch port 9 with a 1GbE transceiver on it.

Wan

Lan

Interface Status

sheen73

@sheen73

And now that i tried a DAC cable, LAN options opened up. It dint connect, but out spf+ transceiver back in and it did.

stephenw10

Hmm, curious. I wouldn't expect the reported media types to differ if the module is identical.

sheen73

@stephenw10
Sorry, probably confusing things. On LAN I had same transceiver as WAN, tried a DAC cable that came in yesterday, set connection rate before inserting DAC to 10GBase. Couldnt get it to connect, when I put the transceiver back in it displayed the new speed options. They are gone now after reboot, I assume just carry over from the DAC options.

stephenw10

Hmm, that's also interesting. Usually DAC cables give you far fewer media options. Often only one.
I've not tested any 25G NICs personally though so behaviour there may differ.

sheen73

@stephenw10

Thanks for all your help! (i posted this response on wrong thread hehe.)

I ended up buying a 6100, been using pfsense for years and wanted to contribute. I will mess with my home built router and see if I can get it working the way I want but ill probably use it for something else if the 6100 performs the way I hope it should.

Thanks again.

P.S. how can i get the 6 reputation points so I can edit my signature?

stephenw10

You have the required rep now. I removed those rogue posts from the other thread.

You certainly see 2.5G through the 6100. Let us know how it goes.

Steve

sheen73

@stephenw10
Thank you sir, will do.

Setarcos

@sheen73 said in XXV710-DA2 Driver Issue - Firmware Newer Log Note:

I am also concerned because the supported media only reflects 10 base-sr where the XXV710-DA2 is 1/10/25. So wanted to eliminate the driver issue 1st. Thoughts? Should i downgrade firmware? To which version?
ixl1: <Intel(R) Ethernet Controller XXV710 for 25GbE SFP28 - 2.3.3-k> mem *removed* at device 0.1 on pci1
ixl1: fw 9.120.73026 api 1.15 nvm 9.20 etid 8000d89c oem 1.269.0
ixl1: The driver for the device detected a newer version of the NVM image than expected.
ixl1: Please install the most recent version of the network driver.

I similarly haven't found much in the way of useful info to try to match up the ideal NVM version with the native FreeBSD ixl driver for my X710-based NIC that more or less is handled the same, and dug into the driver source to see what it considers to be too new. The driver itself is looking at the API version to determine this (1.15 in your case for nvm version 9.20)

Per current FreeBSD main (14.0 pre-release), the latest commits would be compatible with this NVM API version 1.5:
https://github.com/freebsd/freebsd-src/blob/e1353dcb63bbdb33be6a907ed6236c120518129e/sys/dev/ixl/if_ixl.c#L592 and https://github.com/freebsd/freebsd-src/blob/main/sys/dev/ixl/i40e_adminq_cmd.h#L45

However, 23.05 is based on 14.0-CURRENT@0c59e0b4e581, which is an older build which is likely only compatible up to API version 1.4 (https://github.com/freebsd/freebsd-src/blame/1d9722de6f90c3edf286b077938bfa696e728d6c/sys/dev/ixl/i40e_adminq_cmd.h). I have yet to find a table that maps the NVM version with the API version, but this must exist somewhere...

Setarcos

A few more web searches, and it looks like the NVM API version changed from 1.14 to 1.15 as of NVM version 8.50. NVM version 8.40 would be the latest compatible, but Intel's download site for the FreeBSD NVM package jumps from version 8.30 to 8.50, so they likely pulled 8.40 for some reason.

sheen73

@Setarcos

Thanks for this! I will have to check it out.