Intel X520-DA2 + DAC

DWAyotte

Howdy all, I was wondering if there was anything that needed to be done "extra" in order to get a link light up on this NIC when using a DAC? I have used 4 various DAC's (Cisco, Aruba, Juniper and Intel). 1, the Intel, is even on the officially supported list for the NIC, but still no link. I just get "no carrier" in the status of the interface on pfsense. I have tried running 'ifconfig ix1 up' with no change. I can see each of the DACs on the switch (Juniper EX2300-24MP) and can get link when going switch to switch, so I know the DACs are all good. The only thing I have had success with is using a 1G copper transceiver, but looking to go 10G.

Details:
pfSense 2.5.1-RELEASE (amd64)
Dell OptiPlex 7010
Intel X520-DA2
Intel X540-T1

Thanks!

stephenw10

Can the NIC 'see' the DAC? Try running: ifconfig -vvvm ix1

The X520 NICs are usually pretty forgiving.

Steve

DWAyotte

@stephenw10 it does appear to!

ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: OPT2
	options=e138bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
	capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6>
	ether xx:xx:xx:xx:xx:xx
	inet6 xxxx::xxxx:xxxx:xxxx:xxxx%ix1 prefixlen 64 scopeid 0x2
	media: Ethernet autoselect
	status: no carrier
	supported media:
		media autoselect
		media 10Gbase-Twinax
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	plugged: SFP/SFP+/SFP28 Unknown (Copper pigtail)
	vendor: MACROREER PN: 10GBase-CU SFP+c SN: MHxxxxxx DATE: 2021-03-03

stephenw10

It probably won't link at 1G using twinax if the switch is set for that. It will likely be 10G or nothing.

DWAyotte

@stephenw10

I am a little confused by your reply. Where does it indicate 1G is configured? The switch is set to auto and a show interface indicates it's trying to negotiate 10G:

Link-level type: Ethernet, MTU: 1514, LAN-PHY mode, Speed: 10Gbps

The switch doesn't allow a manual speed setting of 10G, apparently it's against the standard. pfsense allows me to pick default, autoselect or 10Gbase-Twinax. Neither option renders a different result.

stephenw10

It doesn't show that but you had said you had connected a 1G copper transceiver so I was speculating the switch may have been set to 1G. That could show like this.

Does it link correct to other 10G devices using DAC?

Does it link to that switch using fibre modules?

I would expect that to work. There is something low level preventing it.

Steve

DWAyotte

@stephenw10

Ah, gotcha, no worries.

So today we had some 10G optics arrive from fs and we have the same situation as with the DACs, but now with some more information. When plugging in the optics the switch alarms stating the reciever power is too low. It is certainly sounding like the NIC is bad at this point... Tried a fiber optic loopback test on the switch and power levels are strong on the output and receiver, so that's pretty damning evidence. We will try the NIC in another machine to see if it works, otherwise I think it's time for a replacement... I will do my best to remember to come back and update with any news.

stephenw10

Hmm, well that's disappointing.

Actual fiber modules usually give you more connection speed options so you try setting the pfSense end to 10G or 1G fixed. That may get you a link.

You should also be able to see the Tx and Rx levels the module is reporting in ifconfig -vvvm

Steve

DWAyotte

@stephenw10

   plugged: SFP/SFP+/SFP28 10G Base-SR (LC)
   vendor: FS PN: SFP-10GSR-85 SN: xxxxx DATE: 2020-05-05
   module temperature: 47.77 C Voltage: 3.38 Volts
   RX: 0.00 mW (-40.00 dBm) TX: 0.72 mW (-1.40 dBm)

RX is bad on the pfsense side.

    Laser bias current                        :  6.132 mA
    Laser output power                        :  0.6900 mW / -1.61 dBm
    Module temperature                        :  38 degrees C / 100 degrees F
    Module voltage                            :  3.3680 V
    Laser receiver power                      :  0.0001 mW / -40.00 dBm

RX is bad on the switch side.

I don't know what to make of this...

I now have the option to choose:

supported media:
		media autoselect
		media 10Gbase-SR

They both render no carrier.

stephenw10

Hmm, well to rule out the obvious is it possible the fibers are swapped and they are both transmitting on the same one?

Surprising that module does not offer 1G. I guess it's just not dual speed.

Steve