Chelsio 10GB Driver Does Not Recoginize DAC



  • I have VLANs assigned to a Chelsio S310E-CR card. Interfaces are up when generic Cisco receivers are connected with fiber optic cable yet are down when using identical receivers connected with generic twinax cable. It appears that some generic cables have neither specific nor unknown value in the EEPROM. The cxbg driver will report an unsupported module and disable the card. Cables are blocked that do not identify themselves as either SFP or SFP+.

    Each connection was tested on pfSense 2.3.4_p1 and 2.4-RC.

    This issue is similar to Bug #7763.
    https://redmine.pfsense.org/issues/7763


  • Netgate Administrator

    Can we see the exact error the driver generates?

    Does it ever show up like the ix does in that bug report or is it always down with an unrecognised module?

    Steve



  • If you post the command for driver error output, I'll post results.

    Interface down appears similar to the following:
    https://groups.google.com/forum/#!topic/muc.lists.freebsd.stable/waROA12uu4w

    Fiber optic installed:

    ifconfig cxgb0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=6c07bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6>ether 00:07:43:05:cc:9f
    	hwaddr 00:07:43:05:cc:9f
    	inet6 fe80::207:43ff:fe05:cc9f%cxgb0 prefixlen 64 scopeid 0x5
    	nd6 options=21 <performnud,auto_linklocal>media: Ethernet 10Gbase-SR <full-duplex>
    	status: active</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,simplex,multicast>
    

    DAC installed

    ifconfig cxgb0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=6c07bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6>ether 00:07:43:05:cc:9f
    	hwaddr 00:07:43:05:cc:9f
    	inet6 fe80::207:43ff:fe05:cc9f%cxgb0 prefixlen 64 scopeid 0x5
    	nd6 options=21 <performnud,auto_linklocal>media: Ethernet 10Gbase-Twinax <full-duplex>
    	status: no carrier</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,simplex,multicast>
    

  • Netgate Administrator

    Ah, you said 'The cxbg driver will report an unsupported module' I assumed you were actually seeing a report. My bad.

    Check the system logs when you connect the cable.

    Check the sysctls, cxbg has some stats there I believe.

    Steve



  • Output from sysctls

    dev.cxgbc.0.port0.mac_stats.link_faults: 0
    dev.cxgbc.0.port0.mac_stats.num_resets: 0
    dev.cxgbc.0.port0.mac_stats.num_toggled: 0
    dev.cxgbc.0.port0.mac_stats.xaui_pcs_align_change: 1
    dev.cxgbc.0.port0.mac_stats.xaui_pcs_ctc_err: 0
    dev.cxgbc.0.port0.mac_stats.serdes_signal_loss: 1
    dev.cxgbc.0.port0.mac_stats.rx_fifo_ovfl: 0
    dev.cxgbc.0.port0.mac_stats.tx_fifo_urun: 0
    dev.cxgbc.0.port0.mac_stats.rx_fifo_parity_err: 0
    dev.cxgbc.0.port0.mac_stats.tx_fifo_parity_err: 0
    dev.cxgbc.0.port0.mac_stats.rx_frames_1519_max: 0
    dev.cxgbc.0.port0.mac_stats.rx_frames_1024_1518: 0
    dev.cxgbc.0.port0.mac_stats.rx_frames_512_1023: 10
    dev.cxgbc.0.port0.mac_stats.rx_frames_256_511: 1201
    dev.cxgbc.0.port0.mac_stats.rx_frames_128_255: 24956
    dev.cxgbc.0.port0.mac_stats.rx_frames_65_127: 67292
    dev.cxgbc.0.port0.mac_stats.rx_frames_64: 1457072
    dev.cxgbc.0.port0.mac_stats.rx_cong_drops: 0
    dev.cxgbc.0.port0.mac_stats.rx_mac_internal_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_too_long: 0
    dev.cxgbc.0.port0.mac_stats.rx_short: 0
    dev.cxgbc.0.port0.mac_stats.rx_jabber: 0
    dev.cxgbc.0.port0.mac_stats.rx_runt: 0
    dev.cxgbc.0.port0.mac_stats.rx_sequence_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_data_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_symbol_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_align_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_fcs_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_pause: 0
    dev.cxgbc.0.port0.mac_stats.rx_bcast_frames: 1550528
    dev.cxgbc.0.port0.mac_stats.rx_mcast_frames: 3
    dev.cxgbc.0.port0.mac_stats.rx_frames: 1550531
    dev.cxgbc.0.port0.mac_stats.rx_octets_bad: 0
    dev.cxgbc.0.port0.mac_stats.rx_octets: 104065069
    dev.cxgbc.0.port0.mac_stats.tx_frames_1519_max: 0
    dev.cxgbc.0.port0.mac_stats.tx_frames_1024_1518: 0
    dev.cxgbc.0.port0.mac_stats.tx_frames_512_1023: 0
    dev.cxgbc.0.port0.mac_stats.tx_frames_256_511: 0
    dev.cxgbc.0.port0.mac_stats.tx_frames_128_255: 1
    dev.cxgbc.0.port0.mac_stats.tx_frames_65_127: 5
    dev.cxgbc.0.port0.mac_stats.tx_frames_64: 4
    dev.cxgbc.0.port0.mac_stats.tx_fcs_errs: 0
    dev.cxgbc.0.port0.mac_stats.tx_excess_deferral: 0
    dev.cxgbc.0.port0.mac_stats.tx_mac_internal_errs: 0
    dev.cxgbc.0.port0.mac_stats.tx_len_errs: 0
    dev.cxgbc.0.port0.mac_stats.tx_underrun: 0
    dev.cxgbc.0.port0.mac_stats.tx_excess_collisions: 0
    dev.cxgbc.0.port0.mac_stats.tx_total_collisions: 0
    dev.cxgbc.0.port0.mac_stats.tx_late_collisions: 0
    dev.cxgbc.0.port0.mac_stats.tx_deferred: 0
    dev.cxgbc.0.port0.mac_stats.tx_pause: 0
    dev.cxgbc.0.port0.mac_stats.tx_bcast_frames: 4
    dev.cxgbc.0.port0.mac_stats.tx_mcast_frames: 6
    dev.cxgbc.0.port0.mac_stats.tx_frames: 10
    dev.cxgbc.0.port0.mac_stats.tx_octets_bad: 0
    dev.cxgbc.0.port0.mac_stats.tx_octets: 940
    dev.cxgbc.0.port0.qs3.lro_stats.lro_cnt: 8
    dev.cxgbc.0.port0.qs3.lro_stats.lro_bad_csum: 0
    dev.cxgbc.0.port0.qs3.lro_stats.lro_flushed: 0
    dev.cxgbc.0.port0.qs3.lro_stats.lro_queued: 0
    dev.cxgbc.0.port0.qs3.txq_ctrl.qdump:
    dev.cxgbc.0.port0.qs3.txq_ctrl.dump_count: 0
    dev.cxgbc.0.port0.qs3.txq_ctrl.dump_start: 0
    dev.cxgbc.0.port0.qs3.txq_eth.qdump:
    dev.cxgbc.0.port0.qs3.txq_eth.dump_count: 0
    dev.cxgbc.0.port0.qs3.txq_eth.dump_start: 0
    dev.cxgbc.0.port0.qs3.txq_eth.hw_pidx: 0
    dev.cxgbc.0.port0.qs3.txq_eth.hw_cidx: 0
    dev.cxgbc.0.port0.qs3.txq_eth.qgen: 1
    dev.cxgbc.0.port0.qs3.txq_eth.phys_addr: 1752743936
    dev.cxgbc.0.port0.qs3.txq_eth.stopped_flags: 0
    dev.cxgbc.0.port0.qs3.txq_eth.enqueued: 0
    dev.cxgbc.0.port0.qs3.txq_eth.coalesced: 0
    dev.cxgbc.0.port0.qs3.txq_eth.skipped: 0
    dev.cxgbc.0.port0.qs3.txq_eth.frees: 0
    dev.cxgbc.0.port0.qs3.txq_eth.in_use: 0
    dev.cxgbc.0.port0.qs3.txq_eth.cleaned: 0
    dev.cxgbc.0.port0.qs3.txq_eth.processed: 0
    dev.cxgbc.0.port0.qs3.txq_eth.sendqlen: 0
    dev.cxgbc.0.port0.qs3.txq_eth.dropped: 0
    dev.cxgbc.0.port0.qs3.rspq.qdump:
    dev.cxgbc.0.port0.qs3.rspq.dump_count: 0
    dev.cxgbc.0.port0.qs3.rspq.dump_start: 0
    dev.cxgbc.0.port0.qs3.rspq.phys_addr: 1752612864
    dev.cxgbc.0.port0.qs3.rspq.starved: 0
    dev.cxgbc.0.port0.qs3.rspq.credits: 1
    dev.cxgbc.0.port0.qs3.rspq.cidx: 65
    dev.cxgbc.0.port0.qs3.rspq.size: 2048
    dev.cxgbc.0.port0.qs3.fl1_empty: 0
    dev.cxgbc.0.port0.qs3.fl0_empty: 0
    dev.cxgbc.0.port0.qs2.lro_stats.lro_cnt: 8
    dev.cxgbc.0.port0.qs2.lro_stats.lro_bad_csum: 0
    dev.cxgbc.0.port0.qs2.lro_stats.lro_flushed: 0
    dev.cxgbc.0.port0.qs2.lro_stats.lro_queued: 0
    dev.cxgbc.0.port0.qs2.txq_ctrl.qdump:
    dev.cxgbc.0.port0.qs2.txq_ctrl.dump_count: 0
    dev.cxgbc.0.port0.qs2.txq_ctrl.dump_start: 0
    dev.cxgbc.0.port0.qs2.txq_eth.qdump:
    dev.cxgbc.0.port0.qs2.txq_eth.dump_count: 0
    dev.cxgbc.0.port0.qs2.txq_eth.dump_start: 0
    dev.cxgbc.0.port0.qs2.txq_eth.hw_pidx: 0
    dev.cxgbc.0.port0.qs2.txq_eth.hw_cidx: 0
    dev.cxgbc.0.port0.qs2.txq_eth.qgen: 1
    dev.cxgbc.0.port0.qs2.txq_eth.phys_addr: 1724973056
    dev.cxgbc.0.port0.qs2.txq_eth.stopped_flags: 0
    dev.cxgbc.0.port0.qs2.txq_eth.enqueued: 0
    dev.cxgbc.0.port0.qs2.txq_eth.coalesced: 0
    dev.cxgbc.0.port0.qs2.txq_eth.skipped: 0
    dev.cxgbc.0.port0.qs2.txq_eth.frees: 0
    dev.cxgbc.0.port0.qs2.txq_eth.in_use: 0
    dev.cxgbc.0.port0.qs2.txq_eth.cleaned: 0
    dev.cxgbc.0.port0.qs2.txq_eth.processed: 0
    dev.cxgbc.0.port0.qs2.txq_eth.sendqlen: 0
    dev.cxgbc.0.port0.qs2.txq_eth.dropped: 0
    dev.cxgbc.0.port0.qs2.rspq.qdump:
    dev.cxgbc.0.port0.qs2.rspq.dump_count: 0
    dev.cxgbc.0.port0.qs2.rspq.dump_start: 0
    dev.cxgbc.0.port0.qs2.rspq.phys_addr: 1724841984
    dev.cxgbc.0.port0.qs2.rspq.starved: 0
    dev.cxgbc.0.port0.qs2.rspq.credits: 61
    dev.cxgbc.0.port0.qs2.rspq.cidx: 1725
    dev.cxgbc.0.port0.qs2.rspq.size: 2048
    dev.cxgbc.0.port0.qs2.fl1_empty: 0
    dev.cxgbc.0.port0.qs2.fl0_empty: 0
    dev.cxgbc.0.port0.qs1.lro_stats.lro_cnt: 8
    dev.cxgbc.0.port0.qs1.lro_stats.lro_bad_csum: 0
    dev.cxgbc.0.port0.qs1.lro_stats.lro_flushed: 0
    dev.cxgbc.0.port0.qs1.lro_stats.lro_queued: 0
    dev.cxgbc.0.port0.qs1.txq_ctrl.qdump:
    dev.cxgbc.0.port0.qs1.txq_ctrl.dump_count: 0
    dev.cxgbc.0.port0.qs1.txq_ctrl.dump_start: 0
    dev.cxgbc.0.port0.qs1.txq_eth.qdump:
    dev.cxgbc.0.port0.qs1.txq_eth.dump_count: 0
    dev.cxgbc.0.port0.qs1.txq_eth.dump_start: 0
    dev.cxgbc.0.port0.qs1.txq_eth.hw_pidx: 0
    dev.cxgbc.0.port0.qs1.txq_eth.hw_cidx: 0
    dev.cxgbc.0.port0.qs1.txq_eth.qgen: 1
    dev.cxgbc.0.port0.qs1.txq_eth.phys_addr: 1688813568
    dev.cxgbc.0.port0.qs1.txq_eth.stopped_flags: 0
    dev.cxgbc.0.port0.qs1.txq_eth.enqueued: 0
    dev.cxgbc.0.port0.qs1.txq_eth.coalesced: 0
    dev.cxgbc.0.port0.qs1.txq_eth.skipped: 0
    dev.cxgbc.0.port0.qs1.txq_eth.frees: 0
    dev.cxgbc.0.port0.qs1.txq_eth.in_use: 0
    dev.cxgbc.0.port0.qs1.txq_eth.cleaned: 0
    dev.cxgbc.0.port0.qs1.txq_eth.processed: 0
    dev.cxgbc.0.port0.qs1.txq_eth.sendqlen: 0
    dev.cxgbc.0.port0.qs1.txq_eth.dropped: 0
    dev.cxgbc.0.port0.qs1.rspq.qdump:
    dev.cxgbc.0.port0.qs1.rspq.dump_count: 0
    dev.cxgbc.0.port0.qs1.rspq.dump_start: 0
    dev.cxgbc.0.port0.qs1.rspq.phys_addr: 1688682496
    dev.cxgbc.0.port0.qs1.rspq.starved: 0
    dev.cxgbc.0.port0.qs1.rspq.credits: 19
    dev.cxgbc.0.port0.qs1.rspq.cidx: 723
    dev.cxgbc.0.port0.qs1.rspq.size: 2048
    dev.cxgbc.0.port0.qs1.fl1_empty: 0
    dev.cxgbc.0.port0.qs1.fl0_empty: 0
    dev.cxgbc.0.port0.qs0.lro_stats.lro_cnt: 8
    dev.cxgbc.0.port0.qs0.lro_stats.lro_bad_csum: 0
    dev.cxgbc.0.port0.qs0.lro_stats.lro_flushed: 0
    dev.cxgbc.0.port0.qs0.lro_stats.lro_queued: 0
    dev.cxgbc.0.port0.qs0.txq_ctrl.qdump:
    dev.cxgbc.0.port0.qs0.txq_ctrl.dump_count: 0
    dev.cxgbc.0.port0.qs0.txq_ctrl.dump_start: 0
    dev.cxgbc.0.port0.qs0.txq_eth.qdump:
    dev.cxgbc.0.port0.qs0.txq_eth.dump_count: 0
    dev.cxgbc.0.port0.qs0.txq_eth.dump_start: 0
    dev.cxgbc.0.port0.qs0.txq_eth.hw_pidx: 10
    dev.cxgbc.0.port0.qs0.txq_eth.hw_cidx: 6
    dev.cxgbc.0.port0.qs0.txq_eth.qgen: 1
    dev.cxgbc.0.port0.qs0.txq_eth.phys_addr: 1679949824
    dev.cxgbc.0.port0.qs0.txq_eth.stopped_flags: 0
    dev.cxgbc.0.port0.qs0.txq_eth.enqueued: 0
    dev.cxgbc.0.port0.qs0.txq_eth.coalesced: 0
    dev.cxgbc.0.port0.qs0.txq_eth.skipped: 2
    dev.cxgbc.0.port0.qs0.txq_eth.frees: 0
    dev.cxgbc.0.port0.qs0.txq_eth.in_use: 4
    dev.cxgbc.0.port0.qs0.txq_eth.cleaned: 6
    dev.cxgbc.0.port0.qs0.txq_eth.processed: 10
    dev.cxgbc.0.port0.qs0.txq_eth.sendqlen: 0
    dev.cxgbc.0.port0.qs0.txq_eth.dropped: 0
    dev.cxgbc.0.port0.qs0.rspq.qdump:
    dev.cxgbc.0.port0.qs0.rspq.dump_count: 0
    dev.cxgbc.0.port0.qs0.rspq.dump_start: 0
    dev.cxgbc.0.port0.qs0.rspq.phys_addr: 1679818752
    dev.cxgbc.0.port0.qs0.rspq.starved: 0
    dev.cxgbc.0.port0.qs0.rspq.credits: 28
    dev.cxgbc.0.port0.qs0.rspq.cidx: 1820
    dev.cxgbc.0.port0.qs0.rspq.size: 2048
    dev.cxgbc.0.port0.qs0.fl1_empty: 0
    dev.cxgbc.0.port0.qs0.fl0_empty: 0
    dev.cxgbc.0.port0.nqsets: 4
    dev.cxgbc.0.pkt_timestamp: 0
    dev.cxgbc.0.intr_coal: 5
    dev.cxgbc.0.wake: 0
    dev.cxgbc.0.core_clock: 175000
    dev.cxgbc.0.txq_overrun: 0
    dev.cxgbc.0.tunq_coalesce: 0
    dev.cxgbc.0.enable_debug: 0
    dev.cxgbc.0.port_types: 5000
    dev.cxgbc.0.hw_revision: 4
    dev.cxgbc.0.firmware_version: 7.11.0
    dev.cxgbc.0.%parent: pci7
    dev.cxgbc.0.%pnpinfo: vendor=0x1425 device=0x0035 subvendor=0x1425 subdevice=0x0001 class=0x020000
    dev.cxgbc.0.%location: pci0:7:0:0 handle=\_SB_.PCI0.RP09.PXSX
    dev.cxgbc.0.%driver: cxgbc
    dev.cxgbc.0.%desc: Chelsio T310 RNIC	 E/C: 110110730A0      S/N: PT25090230
    dev.cxgbc.%parent:
    
    Sep 6 20:10:09  kernel		                cxgb0: SR PHY module inserted
    Sep 6 20:10:09	check_reload_status		Linkup starting cxgb0
    Sep 6 20:10:09	kernel		                cxgb0: link state changed to UP
    Sep 6 20:10:09	kernel		                cxgb0_vlan20: link state changed to UP
    Sep 6 20:10:09	kernel		                cxgb0_vlan40: link state changed to UP
    Sep 6 20:10:09	kernel		                cxgb0_vlan10: link state changed to UP
    Sep 6 20:10:09	kernel		                cxgb0_vlan30: link state changed to UP
    Sep 6 20:10:09	check_reload_status		Linkup starting cxgb0_vlan20
    Sep 6 20:10:09	check_reload_status		Linkup starting cxgb0_vlan40
    Sep 6 20:10:09	check_reload_status		Linkup starting cxgb0_vlan10
    Sep 6 20:10:09	check_reload_status		Linkup starting cxgb0_vlan30
    

    Twinax cable:

    dev.cxgbc.0.port0.mac_stats.link_faults: 0
    dev.cxgbc.0.port0.mac_stats.num_resets: 0
    dev.cxgbc.0.port0.mac_stats.num_toggled: 0
    dev.cxgbc.0.port0.mac_stats.xaui_pcs_align_change: 2
    dev.cxgbc.0.port0.mac_stats.xaui_pcs_ctc_err: 0
    dev.cxgbc.0.port0.mac_stats.serdes_signal_loss: 2
    dev.cxgbc.0.port0.mac_stats.rx_fifo_ovfl: 0
    dev.cxgbc.0.port0.mac_stats.tx_fifo_urun: 0
    dev.cxgbc.0.port0.mac_stats.rx_fifo_parity_err: 0
    dev.cxgbc.0.port0.mac_stats.tx_fifo_parity_err: 0
    dev.cxgbc.0.port0.mac_stats.rx_frames_1519_max: 0
    dev.cxgbc.0.port0.mac_stats.rx_frames_1024_1518: 0
    dev.cxgbc.0.port0.mac_stats.rx_frames_512_1023: 10
    dev.cxgbc.0.port0.mac_stats.rx_frames_256_511: 1203
    dev.cxgbc.0.port0.mac_stats.rx_frames_128_255: 24999
    dev.cxgbc.0.port0.mac_stats.rx_frames_65_127: 67422
    dev.cxgbc.0.port0.mac_stats.rx_frames_64: 1459457
    dev.cxgbc.0.port0.mac_stats.rx_cong_drops: 0
    dev.cxgbc.0.port0.mac_stats.rx_mac_internal_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_too_long: 0
    dev.cxgbc.0.port0.mac_stats.rx_short: 0
    dev.cxgbc.0.port0.mac_stats.rx_jabber: 0
    dev.cxgbc.0.port0.mac_stats.rx_runt: 0
    dev.cxgbc.0.port0.mac_stats.rx_sequence_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_data_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_symbol_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_align_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_fcs_errs: 0
    dev.cxgbc.0.port0.mac_stats.rx_pause: 0
    dev.cxgbc.0.port0.mac_stats.rx_bcast_frames: 1553088
    dev.cxgbc.0.port0.mac_stats.rx_mcast_frames: 3
    dev.cxgbc.0.port0.mac_stats.rx_frames: 1553091
    dev.cxgbc.0.port0.mac_stats.rx_octets_bad: 0
    dev.cxgbc.0.port0.mac_stats.rx_octets: 104237663
    dev.cxgbc.0.port0.mac_stats.tx_frames_1519_max: 0
    dev.cxgbc.0.port0.mac_stats.tx_frames_1024_1518: 0
    dev.cxgbc.0.port0.mac_stats.tx_frames_512_1023: 0
    dev.cxgbc.0.port0.mac_stats.tx_frames_256_511: 0
    dev.cxgbc.0.port0.mac_stats.tx_frames_128_255: 1
    dev.cxgbc.0.port0.mac_stats.tx_frames_65_127: 5
    dev.cxgbc.0.port0.mac_stats.tx_frames_64: 4
    dev.cxgbc.0.port0.mac_stats.tx_fcs_errs: 0
    dev.cxgbc.0.port0.mac_stats.tx_excess_deferral: 0
    dev.cxgbc.0.port0.mac_stats.tx_mac_internal_errs: 0
    dev.cxgbc.0.port0.mac_stats.tx_len_errs: 0
    dev.cxgbc.0.port0.mac_stats.tx_underrun: 0
    dev.cxgbc.0.port0.mac_stats.tx_excess_collisions: 0
    dev.cxgbc.0.port0.mac_stats.tx_total_collisions: 0
    dev.cxgbc.0.port0.mac_stats.tx_late_collisions: 0
    dev.cxgbc.0.port0.mac_stats.tx_deferred: 0
    dev.cxgbc.0.port0.mac_stats.tx_pause: 0
    dev.cxgbc.0.port0.mac_stats.tx_bcast_frames: 4
    dev.cxgbc.0.port0.mac_stats.tx_mcast_frames: 6
    dev.cxgbc.0.port0.mac_stats.tx_frames: 10
    dev.cxgbc.0.port0.mac_stats.tx_octets_bad: 0
    dev.cxgbc.0.port0.mac_stats.tx_octets: 940
    dev.cxgbc.0.port0.qs3.lro_stats.lro_cnt: 8
    dev.cxgbc.0.port0.qs3.lro_stats.lro_bad_csum: 0
    dev.cxgbc.0.port0.qs3.lro_stats.lro_flushed: 0
    dev.cxgbc.0.port0.qs3.lro_stats.lro_queued: 0
    dev.cxgbc.0.port0.qs3.txq_ctrl.qdump:
    dev.cxgbc.0.port0.qs3.txq_ctrl.dump_count: 0
    dev.cxgbc.0.port0.qs3.txq_ctrl.dump_start: 0
    dev.cxgbc.0.port0.qs3.txq_eth.qdump:
    dev.cxgbc.0.port0.qs3.txq_eth.dump_count: 0
    dev.cxgbc.0.port0.qs3.txq_eth.dump_start: 0
    dev.cxgbc.0.port0.qs3.txq_eth.hw_pidx: 0
    dev.cxgbc.0.port0.qs3.txq_eth.hw_cidx: 0
    dev.cxgbc.0.port0.qs3.txq_eth.qgen: 1
    dev.cxgbc.0.port0.qs3.txq_eth.phys_addr: 1752743936
    dev.cxgbc.0.port0.qs3.txq_eth.stopped_flags: 0
    dev.cxgbc.0.port0.qs3.txq_eth.enqueued: 0
    dev.cxgbc.0.port0.qs3.txq_eth.coalesced: 0
    dev.cxgbc.0.port0.qs3.txq_eth.skipped: 0
    dev.cxgbc.0.port0.qs3.txq_eth.frees: 0
    dev.cxgbc.0.port0.qs3.txq_eth.in_use: 0
    dev.cxgbc.0.port0.qs3.txq_eth.cleaned: 0
    dev.cxgbc.0.port0.qs3.txq_eth.processed: 0
    dev.cxgbc.0.port0.qs3.txq_eth.sendqlen: 0
    dev.cxgbc.0.port0.qs3.txq_eth.dropped: 0
    dev.cxgbc.0.port0.qs3.rspq.qdump:
    dev.cxgbc.0.port0.qs3.rspq.dump_count: 0
    dev.cxgbc.0.port0.qs3.rspq.dump_start: 0
    dev.cxgbc.0.port0.qs3.rspq.phys_addr: 1752612864
    dev.cxgbc.0.port0.qs3.rspq.starved: 0
    dev.cxgbc.0.port0.qs3.rspq.credits: 29
    dev.cxgbc.0.port0.qs3.rspq.cidx: 93
    dev.cxgbc.0.port0.qs3.rspq.size: 2048
    dev.cxgbc.0.port0.qs3.fl1_empty: 0
    dev.cxgbc.0.port0.qs3.fl0_empty: 0
    dev.cxgbc.0.port0.qs2.lro_stats.lro_cnt: 8
    dev.cxgbc.0.port0.qs2.lro_stats.lro_bad_csum: 0
    dev.cxgbc.0.port0.qs2.lro_stats.lro_flushed: 0
    dev.cxgbc.0.port0.qs2.lro_stats.lro_queued: 0
    dev.cxgbc.0.port0.qs2.txq_ctrl.qdump:
    dev.cxgbc.0.port0.qs2.txq_ctrl.dump_count: 0
    dev.cxgbc.0.port0.qs2.txq_ctrl.dump_start: 0
    dev.cxgbc.0.port0.qs2.txq_eth.qdump:
    dev.cxgbc.0.port0.qs2.txq_eth.dump_count: 0
    dev.cxgbc.0.port0.qs2.txq_eth.dump_start: 0
    dev.cxgbc.0.port0.qs2.txq_eth.hw_pidx: 0
    dev.cxgbc.0.port0.qs2.txq_eth.hw_cidx: 0
    dev.cxgbc.0.port0.qs2.txq_eth.qgen: 1
    dev.cxgbc.0.port0.qs2.txq_eth.phys_addr: 1724973056
    dev.cxgbc.0.port0.qs2.txq_eth.stopped_flags: 0
    dev.cxgbc.0.port0.qs2.txq_eth.enqueued: 0
    dev.cxgbc.0.port0.qs2.txq_eth.coalesced: 0
    dev.cxgbc.0.port0.qs2.txq_eth.skipped: 0
    dev.cxgbc.0.port0.qs2.txq_eth.frees: 0
    dev.cxgbc.0.port0.qs2.txq_eth.in_use: 0
    dev.cxgbc.0.port0.qs2.txq_eth.cleaned: 0
    dev.cxgbc.0.port0.qs2.txq_eth.processed: 0
    dev.cxgbc.0.port0.qs2.txq_eth.sendqlen: 0
    dev.cxgbc.0.port0.qs2.txq_eth.dropped: 0
    dev.cxgbc.0.port0.qs2.rspq.qdump:
    dev.cxgbc.0.port0.qs2.rspq.dump_count: 0
    dev.cxgbc.0.port0.qs2.rspq.dump_start: 0
    dev.cxgbc.0.port0.qs2.rspq.phys_addr: 1724841984
    dev.cxgbc.0.port0.qs2.rspq.starved: 0
    dev.cxgbc.0.port0.qs2.rspq.credits: 4
    dev.cxgbc.0.port0.qs2.rspq.cidx: 1732
    dev.cxgbc.0.port0.qs2.rspq.size: 2048
    dev.cxgbc.0.port0.qs2.fl1_empty: 0
    dev.cxgbc.0.port0.qs2.fl0_empty: 0
    dev.cxgbc.0.port0.qs1.lro_stats.lro_cnt: 8
    dev.cxgbc.0.port0.qs1.lro_stats.lro_bad_csum: 0
    dev.cxgbc.0.port0.qs1.lro_stats.lro_flushed: 0
    dev.cxgbc.0.port0.qs1.lro_stats.lro_queued: 0
    dev.cxgbc.0.port0.qs1.txq_ctrl.qdump:
    dev.cxgbc.0.port0.qs1.txq_ctrl.dump_count: 0
    dev.cxgbc.0.port0.qs1.txq_ctrl.dump_start: 0
    dev.cxgbc.0.port0.qs1.txq_eth.qdump:
    dev.cxgbc.0.port0.qs1.txq_eth.dump_count: 0
    dev.cxgbc.0.port0.qs1.txq_eth.dump_start: 0
    dev.cxgbc.0.port0.qs1.txq_eth.hw_pidx: 0
    dev.cxgbc.0.port0.qs1.txq_eth.hw_cidx: 0
    dev.cxgbc.0.port0.qs1.txq_eth.qgen: 1
    dev.cxgbc.0.port0.qs1.txq_eth.phys_addr: 1688813568
    dev.cxgbc.0.port0.qs1.txq_eth.stopped_flags: 0
    dev.cxgbc.0.port0.qs1.txq_eth.enqueued: 0
    dev.cxgbc.0.port0.qs1.txq_eth.coalesced: 0
    dev.cxgbc.0.port0.qs1.txq_eth.skipped: 0
    dev.cxgbc.0.port0.qs1.txq_eth.frees: 0
    dev.cxgbc.0.port0.qs1.txq_eth.in_use: 0
    dev.cxgbc.0.port0.qs1.txq_eth.cleaned: 0
    dev.cxgbc.0.port0.qs1.txq_eth.processed: 0
    dev.cxgbc.0.port0.qs1.txq_eth.sendqlen: 0
    dev.cxgbc.0.port0.qs1.txq_eth.dropped: 0
    dev.cxgbc.0.port0.qs1.rspq.qdump:
    dev.cxgbc.0.port0.qs1.rspq.dump_count: 0
    dev.cxgbc.0.port0.qs1.rspq.dump_start: 0
    dev.cxgbc.0.port0.qs1.rspq.phys_addr: 1688682496
    dev.cxgbc.0.port0.qs1.rspq.starved: 0
    dev.cxgbc.0.port0.qs1.rspq.credits: 54
    dev.cxgbc.0.port0.qs1.rspq.cidx: 758
    dev.cxgbc.0.port0.qs1.rspq.size: 2048
    dev.cxgbc.0.port0.qs1.fl1_empty: 0
    dev.cxgbc.0.port0.qs1.fl0_empty: 0
    dev.cxgbc.0.port0.qs0.lro_stats.lro_cnt: 8
    dev.cxgbc.0.port0.qs0.lro_stats.lro_bad_csum: 0
    dev.cxgbc.0.port0.qs0.lro_stats.lro_flushed: 0
    dev.cxgbc.0.port0.qs0.lro_stats.lro_queued: 0
    dev.cxgbc.0.port0.qs0.txq_ctrl.qdump:
    dev.cxgbc.0.port0.qs0.txq_ctrl.dump_count: 0
    dev.cxgbc.0.port0.qs0.txq_ctrl.dump_start: 0
    dev.cxgbc.0.port0.qs0.txq_eth.qdump:
    dev.cxgbc.0.port0.qs0.txq_eth.dump_count: 0
    dev.cxgbc.0.port0.qs0.txq_eth.dump_start: 0
    dev.cxgbc.0.port0.qs0.txq_eth.hw_pidx: 10
    dev.cxgbc.0.port0.qs0.txq_eth.hw_cidx: 6
    dev.cxgbc.0.port0.qs0.txq_eth.qgen: 1
    dev.cxgbc.0.port0.qs0.txq_eth.phys_addr: 1679949824
    dev.cxgbc.0.port0.qs0.txq_eth.stopped_flags: 0
    dev.cxgbc.0.port0.qs0.txq_eth.enqueued: 0
    dev.cxgbc.0.port0.qs0.txq_eth.coalesced: 0
    dev.cxgbc.0.port0.qs0.txq_eth.skipped: 2
    dev.cxgbc.0.port0.qs0.txq_eth.frees: 0
    dev.cxgbc.0.port0.qs0.txq_eth.in_use: 4
    dev.cxgbc.0.port0.qs0.txq_eth.cleaned: 6
    dev.cxgbc.0.port0.qs0.txq_eth.processed: 10
    dev.cxgbc.0.port0.qs0.txq_eth.sendqlen: 0
    dev.cxgbc.0.port0.qs0.txq_eth.dropped: 0
    dev.cxgbc.0.port0.qs0.rspq.qdump:
    dev.cxgbc.0.port0.qs0.rspq.dump_count: 0
    dev.cxgbc.0.port0.qs0.rspq.dump_start: 0
    dev.cxgbc.0.port0.qs0.rspq.phys_addr: 1679818752
    dev.cxgbc.0.port0.qs0.rspq.starved: 0
    dev.cxgbc.0.port0.qs0.rspq.credits: 22
    dev.cxgbc.0.port0.qs0.rspq.cidx: 214
    dev.cxgbc.0.port0.qs0.rspq.size: 2048
    dev.cxgbc.0.port0.qs0.fl1_empty: 0
    dev.cxgbc.0.port0.qs0.fl0_empty: 0
    dev.cxgbc.0.port0.nqsets: 4
    dev.cxgbc.0.pkt_timestamp: 0
    dev.cxgbc.0.intr_coal: 5
    dev.cxgbc.0.wake: 0
    dev.cxgbc.0.core_clock: 175000
    dev.cxgbc.0.txq_overrun: 0
    dev.cxgbc.0.tunq_coalesce: 0
    dev.cxgbc.0.enable_debug: 0
    dev.cxgbc.0.port_types: 5000
    dev.cxgbc.0.hw_revision: 4
    dev.cxgbc.0.firmware_version: 7.11.0
    dev.cxgbc.0.%parent: pci7
    dev.cxgbc.0.%pnpinfo: vendor=0x1425 device=0x0035 subvendor=0x1425 subdevice=0x0001 class=0x020000
    dev.cxgbc.0.%location: pci0:7:0:0 handle=\_SB_.PCI0.RP09.PXSX
    dev.cxgbc.0.%driver: cxgbc
    dev.cxgbc.0.%desc: Chelsio T310 RNIC	 E/C: 110110730A0      S/N: PT25090230
    dev.cxgbc.%parent:
    
    Sep 6 20:01:27	kernel		cxgb0: TWINAX PHY module inserted
    

  • Netgate Administrator

    Hmmm. Have you seen this with multiple twinax cables?

    You might try setting dev.cxgbc.0.enable_debug to 1 to see if you get any additional output.

    Steve



  • Twinax with Cisco receivers:

    cxgb0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=6c07bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6>ether 00:07:43:05:cc:9f
    	hwaddr 00:07:43:05:cc:9f
    	inet6 fe80::207:43ff:fe05:cc9f%cxgb0 prefixlen 64 scopeid 0x1
    	nd6 options=21 <performnud,auto_linklocal>media: Ethernet 10Gbase-Twinax <full-duplex>
    	status: active</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,simplex,multicast>
    

    Problem could be either receivers or cable.

    I did not try debug command.


  • Netgate Administrator

    Well I'd try the debug command as there's not much to lose. But I wouldn't hold out too much hope there.

    In my experience the Chelsio cards have been pretty forgiving, they work with most things. Clearly not that, but without more logs to go on I'm not sure what we can do.

    Steve



  • I reinstalled the problem Twinax cable:

    cxgb0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    	options=6c07bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6>ether 00:07:43:05:cc:9f
    	hwaddr 00:07:43:05:cc:9f
    	inet6 fe80::207:43ff:fe05:cc9f%cxgb0 prefixlen 64 scopeid 0x1
    	nd6 options=21 <performnud,auto_linklocal>media: Ethernet 10Gbase-Twinax <full-duplex>
    	status: active</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,simplex,multicast>
    

    I'll put it under "User Error" category.


  • Netgate Administrator

    Ah!  ;)

    Good to know anyway, thanks.

    Steve