SFP+ DAC "media: Ethernet Unknown <full-duplex,rxpause,txpause>"
I'm using a Direct Attach Cable between a Chelsio T422-CR adapter and a Ubiquity Unifi US-16-XG switch.
On PfSense, ifconfig shows the Chelsio adapter as unable to determine an appropriate media type and link speed which appears to be causing issues establishing a LAGG connection.
The DAC being used is listed here https://www.provantage.com/accortec-tapcable1m-acc~7ACCBE4X.htm
Can anyone give me guidance here?
You might be better asking here:-
I wonder if the card supports the DAC, does the Ubiquity switch see the DAC ?
What do you see from
ifconfig cxl0 -vv? Assuming cxl0 is the problem interface. cxgbe0 on the T4?
kklouzal last edited by kklouzal
cxgbe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> ether 98:be:94:12:d5:e0 hwaddr 98:be:94:12:d5:e0 inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255 inet6 2600:8800:9d00:17d:9abe:94ff:fe12:d5e0 prefixlen 64 inet6 fe80::1:1%cxgbe0 prefixlen 64 scopeid 0x1 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet Unknown <full-duplex,rxpause,txpause> status: active plugged: SFP/SFP+/SFP28 Unknown (LC) vendor: OEM PN: 81.8800-OC3SFPS SN: ACCT0999 DATE: 2018-11-02 module temperature: 256.00 C Voltage: 6.55 Volts RX: 6.55 mW (8.16 dBm) TX: 6.55 mW (8.16 dBm)
From that output it appears as if the DAC isn't being recognized. "plugged: SFP/SFP+/SFP28 Unknown (LC)"
I'm not sure if it's related but, assuming PCI4 relates to the Chelsio adapter and no other system hardware then these system log entries show some driver issues.
Sep 4 09:20:08 kernel pci4: <network, ethernet> at device 0.7 (no driver attached) Sep 4 09:20:08 kernel pci4: <serial bus, Fibre Channel> at device 0.6 (no driver attached) Sep 4 09:20:08 kernel pci4: <mass storage, SCSI> at device 0.5 (no driver attached) Sep 4 09:20:08 kernel t4nex0: PCIe gen2 x8, 4 ports, 42 MSI-X interrupts, 109 eq, 41 iq Sep 4 09:20:08 kernel cxgbe3: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE) Sep 4 09:20:08 kernel cxgbe3: Ethernet address: 98:be:94:12:d5:f8 Sep 4 09:20:08 kernel cxgbe3: <port 3> on t4nex0 Sep 4 09:20:08 kernel cxgbe2: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE) Sep 4 09:20:08 kernel cxgbe2: Ethernet address: 98:be:94:12:d5:f0 Sep 4 09:20:08 kernel cxgbe2: <port 2> on t4nex0 Sep 4 09:20:08 kernel cxgbe1: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE) Sep 4 09:20:08 kernel cxgbe1: Ethernet address: 98:be:94:12:d5:e8 Sep 4 09:20:08 kernel cxgbe1: <port 1> on t4nex0 Sep 4 09:20:08 kernel cxgbe0: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE) Sep 4 09:20:08 kernel cxgbe0: Ethernet address: 98:be:94:12:d5:e0 Sep 4 09:20:08 kernel cxgbe0: <port 0> on t4nex0 Sep 4 09:20:08 kernel t4nex0: <Chelsio T422-CR> mem 0xdf080000-0xdf0bffff,0xde800000-0xdeffffff,0xdf2c4000-0xdf2c5fff irq 22 at device 0.4 on pci4 Sep 4 09:20:08 kernel pci4: <network, ethernet> at device 0.3 (no driver attached) Sep 4 09:20:08 kernel pci4: <network, ethernet> at device 0.2 (no driver attached) Sep 4 09:20:08 kernel pci4: <ACPI PCI bus> on pcib4 Sep 4 09:20:08 kernel pcib4: <ACPI PCI-PCI bridge> mem 0xde1a0000-0xde1bffff irq 20 at device 3.0 on pci0
Does it recognise other DACs? Or other SFP modules?
You might try a different setting for
I only have a handfull of these specific DACs. I was thinking my next step would be to purchase a pair that Ubiquiti sells in their store https://store.ui.com/collections/accessories/products/unifi-direct-attach-copper-cable-10-gbps-1-meter
Originally I meant to purchase a set of Chelsio DACs https://www.chelsio.com/wp-content/uploads/resources/tapcable-1m-pb.pdf and I thought that's what I did but on closer inspection I don't believe these are actually what I acquired.
I'll go ahead and set
loader.conf.localafter I get home from work in the morning and see if that makes a difference.
On a side note, whenever I try to query a
hw.cxgbe.*sysctl from the command line I always get the output
sysctl: unknown oid. Is there something extra I need to do to query their values? I would like to check that they were set properly after rebooting.
I see output there even on a device that doesn't have a Chelsio card:
[2.5.0-DEVELOPMENT][firstname.lastname@example.org]/root: sysctl hw.cxgbe hw.cxgbe.lazy_tx_credit_flush: 1 hw.cxgbe.starve_fl: 0 hw.cxgbe.nm_holdoff_tmr_idx: 2 hw.cxgbe.nm_rx_nframes: 64 hw.cxgbe.nm_rx_ndesc: 256 hw.cxgbe.nm_black_hole: 0
You won't see those values there are they are loader tunables:
hw.cxgbe.autoneg="1"and rebooting the firewall we are still unable to identify the DAC. I'm thinking this cable is unsupported. Might there be anything else to try?
That seems likely. There is no config option to allow SFP modules I can see for cxgbe. They do have a lot of options though. Updating the card firmware for example might be an option.
Using a different DAC cable is probably a better option though.
dev.t4nex.0.firmware_version: 188.8.131.52I'm not sure if this is the latest available but I have
hw.cxgbe.fw_install="2"so it should force install whatever firmware is present in the driver being used, that being said it is most likely the latest available for the Chelsio driver provided by FreeBSD 11.2-RELEASE-p10.
I'll go ahead and order two of the Ubiquiti DACs and see what happens.
Thank you for all your help I really appreciate it.