SFP+ DAC "media: Ethernet Unknown <full-duplex,rxpause,txpause>"



  • I'm using a Direct Attach Cable between a Chelsio T422-CR adapter and a Ubiquity Unifi US-16-XG switch.

    On PfSense, ifconfig shows the Chelsio adapter as unable to determine an appropriate media type and link speed which appears to be causing issues establishing a LAGG connection.

    The DAC being used is listed here https://www.provantage.com/accortec-tapcable1m-acc~7ACCBE4X.htm

    Can anyone give me guidance here?
    Thank you.


  • Galactic Empire

    You might be better asking here:-

    https://service.chelsio.com/support/support.html

    I wonder if the card supports the DAC, does the Ubiquity switch see the DAC ?


  • Netgate Administrator

    What do you see from ifconfig cxl0 -vv? Assuming cxl0 is the problem interface. cxgbe0 on the T4?

    Steve



  • cxgbe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    	ether 98:be:94:12:d5:e0
    	hwaddr 98:be:94:12:d5:e0
    	inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
    	inet6 2600:8800:9d00:17d:9abe:94ff:fe12:d5e0 prefixlen 64
    	inet6 fe80::1:1%cxgbe0 prefixlen 64 scopeid 0x1
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    	media: Ethernet Unknown <full-duplex,rxpause,txpause>
    	status: active
    	plugged: SFP/SFP+/SFP28 Unknown (LC)
    	vendor: OEM PN: 81.8800-OC3SFPS SN: ACCT0999 DATE: 2018-11-02
    	module temperature: 256.00 C Voltage: 6.55 Volts
    	RX: 6.55 mW (8.16 dBm) TX: 6.55 mW (8.16 dBm)
    

    From that output it appears as if the DAC isn't being recognized. "plugged: SFP/SFP+/SFP28 Unknown (LC)"

    I'm not sure if it's related but, assuming PCI4 relates to the Chelsio adapter and no other system hardware then these system log entries show some driver issues.

    Sep 4 09:20:08	kernel		pci4: <network, ethernet> at device 0.7 (no driver attached)
    Sep 4 09:20:08	kernel		pci4: <serial bus, Fibre Channel> at device 0.6 (no driver attached)
    Sep 4 09:20:08	kernel		pci4: <mass storage, SCSI> at device 0.5 (no driver attached)
    Sep 4 09:20:08	kernel		t4nex0: PCIe gen2 x8, 4 ports, 42 MSI-X interrupts, 109 eq, 41 iq
    Sep 4 09:20:08	kernel		cxgbe3: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE)
    Sep 4 09:20:08	kernel		cxgbe3: Ethernet address: 98:be:94:12:d5:f8
    Sep 4 09:20:08	kernel		cxgbe3: <port 3> on t4nex0
    Sep 4 09:20:08	kernel		cxgbe2: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE)
    Sep 4 09:20:08	kernel		cxgbe2: Ethernet address: 98:be:94:12:d5:f0
    Sep 4 09:20:08	kernel		cxgbe2: <port 2> on t4nex0
    Sep 4 09:20:08	kernel		cxgbe1: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE)
    Sep 4 09:20:08	kernel		cxgbe1: Ethernet address: 98:be:94:12:d5:e8
    Sep 4 09:20:08	kernel		cxgbe1: <port 1> on t4nex0
    Sep 4 09:20:08	kernel		cxgbe0: 8 txq, 8 rxq (NIC); 8 txq, 2 rxq (TOE)
    Sep 4 09:20:08	kernel		cxgbe0: Ethernet address: 98:be:94:12:d5:e0
    Sep 4 09:20:08	kernel		cxgbe0: <port 0> on t4nex0
    Sep 4 09:20:08	kernel		t4nex0: <Chelsio T422-CR> mem 0xdf080000-0xdf0bffff,0xde800000-0xdeffffff,0xdf2c4000-0xdf2c5fff irq 22 at device 0.4 on pci4
    Sep 4 09:20:08	kernel		pci4: <network, ethernet> at device 0.3 (no driver attached)
    Sep 4 09:20:08	kernel		pci4: <network, ethernet> at device 0.2 (no driver attached)
    Sep 4 09:20:08	kernel		pci4: <ACPI PCI bus> on pcib4
    Sep 4 09:20:08	kernel		pcib4: <ACPI PCI-PCI bridge> mem 0xde1a0000-0xde1bffff irq 20 at device 3.0 on pci0
    

  • Netgate Administrator

    Does it recognise other DACs? Or other SFP modules?

    You might try a different setting for hw.cxgbe.autoneg.

    Steve



  • I only have a handfull of these specific DACs. I was thinking my next step would be to purchase a pair that Ubiquiti sells in their store https://store.ui.com/collections/accessories/products/unifi-direct-attach-copper-cable-10-gbps-1-meter

    Originally I meant to purchase a set of Chelsio DACs https://www.chelsio.com/wp-content/uploads/resources/tapcable-1m-pb.pdf and I thought that's what I did but on closer inspection I don't believe these are actually what I acquired.

    I'll go ahead and set hw.cxgbe.autoneg="1" in loader.conf and loader.conf.local after I get home from work in the morning and see if that makes a difference.

    On a side note, whenever I try to query a hw.cxgbe.* sysctl from the command line I always get the output sysctl: unknown oid. Is there something extra I need to do to query their values? I would like to check that they were set properly after rebooting.


  • Netgate Administrator

    I see output there even on a device that doesn't have a Chelsio card:

    [2.5.0-DEVELOPMENT][admin@apu.stevew.lan]/root: sysctl hw.cxgbe
    hw.cxgbe.lazy_tx_credit_flush: 1
    hw.cxgbe.starve_fl: 0
    hw.cxgbe.nm_holdoff_tmr_idx: 2
    hw.cxgbe.nm_rx_nframes: 64
    hw.cxgbe.nm_rx_ndesc: 256
    hw.cxgbe.nm_black_hole: 0
    

    You won't see those values there are they are loader tunables:
    https://www.freebsd.org/cgi/man.cgi?query=cxl&apropos=0&sektion=4&manpath=FreeBSD+11-current&format=html

    Steve



  • After setting hw.cxgbe.autoneg="1" and rebooting the firewall we are still unable to identify the DAC. I'm thinking this cable is unsupported. Might there be anything else to try?


  • Netgate Administrator

    That seems likely. There is no config option to allow SFP modules I can see for cxgbe. They do have a lot of options though. Updating the card firmware for example might be an option.
    Using a different DAC cable is probably a better option though.

    Steve



  • dev.t4nex.0.firmware_version: 1.19.1.0 I'm not sure if this is the latest available but I have hw.cxgbe.fw_install="2" so it should force install whatever firmware is present in the driver being used, that being said it is most likely the latest available for the Chelsio driver provided by FreeBSD 11.2-RELEASE-p10.

    I'll go ahead and order two of the Ubiquiti DACs and see what happens.

    Thank you for all your help I really appreciate it.


Log in to reply