Chelsio 10GB Driver Does Not Recoginize DAC
-
I have VLANs assigned to a Chelsio S310E-CR card. Interfaces are up when generic Cisco receivers are connected with fiber optic cable yet are down when using identical receivers connected with generic twinax cable. It appears that some generic cables have neither specific nor unknown value in the EEPROM. The cxbg driver will report an unsupported module and disable the card. Cables are blocked that do not identify themselves as either SFP or SFP+.
Each connection was tested on pfSense 2.3.4_p1 and 2.4-RC.
This issue is similar to Bug #7763.
https://redmine.pfsense.org/issues/7763 -
Can we see the exact error the driver generates?
Does it ever show up like the ix does in that bug report or is it always down with an unrecognised module?
Steve
-
If you post the command for driver error output, I'll post results.
Interface down appears similar to the following:
https://groups.google.com/forum/#!topic/muc.lists.freebsd.stable/waROA12uu4wFiber optic installed:
ifconfig cxgb0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 options=6c07bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6>ether 00:07:43:05:cc:9f hwaddr 00:07:43:05:cc:9f inet6 fe80::207:43ff:fe05:cc9f%cxgb0 prefixlen 64 scopeid 0x5 nd6 options=21 <performnud,auto_linklocal>media: Ethernet 10Gbase-SR <full-duplex> status: active</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,simplex,multicast>
DAC installed
ifconfig cxgb0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 options=6c07bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6>ether 00:07:43:05:cc:9f hwaddr 00:07:43:05:cc:9f inet6 fe80::207:43ff:fe05:cc9f%cxgb0 prefixlen 64 scopeid 0x5 nd6 options=21 <performnud,auto_linklocal>media: Ethernet 10Gbase-Twinax <full-duplex> status: no carrier</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,simplex,multicast>
-
Ah, you said 'The cxbg driver will report an unsupported module' I assumed you were actually seeing a report. My bad.
Check the system logs when you connect the cable.
Check the sysctls, cxbg has some stats there I believe.
Steve
-
Output from sysctls
dev.cxgbc.0.port0.mac_stats.link_faults: 0 dev.cxgbc.0.port0.mac_stats.num_resets: 0 dev.cxgbc.0.port0.mac_stats.num_toggled: 0 dev.cxgbc.0.port0.mac_stats.xaui_pcs_align_change: 1 dev.cxgbc.0.port0.mac_stats.xaui_pcs_ctc_err: 0 dev.cxgbc.0.port0.mac_stats.serdes_signal_loss: 1 dev.cxgbc.0.port0.mac_stats.rx_fifo_ovfl: 0 dev.cxgbc.0.port0.mac_stats.tx_fifo_urun: 0 dev.cxgbc.0.port0.mac_stats.rx_fifo_parity_err: 0 dev.cxgbc.0.port0.mac_stats.tx_fifo_parity_err: 0 dev.cxgbc.0.port0.mac_stats.rx_frames_1519_max: 0 dev.cxgbc.0.port0.mac_stats.rx_frames_1024_1518: 0 dev.cxgbc.0.port0.mac_stats.rx_frames_512_1023: 10 dev.cxgbc.0.port0.mac_stats.rx_frames_256_511: 1201 dev.cxgbc.0.port0.mac_stats.rx_frames_128_255: 24956 dev.cxgbc.0.port0.mac_stats.rx_frames_65_127: 67292 dev.cxgbc.0.port0.mac_stats.rx_frames_64: 1457072 dev.cxgbc.0.port0.mac_stats.rx_cong_drops: 0 dev.cxgbc.0.port0.mac_stats.rx_mac_internal_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_too_long: 0 dev.cxgbc.0.port0.mac_stats.rx_short: 0 dev.cxgbc.0.port0.mac_stats.rx_jabber: 0 dev.cxgbc.0.port0.mac_stats.rx_runt: 0 dev.cxgbc.0.port0.mac_stats.rx_sequence_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_data_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_symbol_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_align_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_fcs_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_pause: 0 dev.cxgbc.0.port0.mac_stats.rx_bcast_frames: 1550528 dev.cxgbc.0.port0.mac_stats.rx_mcast_frames: 3 dev.cxgbc.0.port0.mac_stats.rx_frames: 1550531 dev.cxgbc.0.port0.mac_stats.rx_octets_bad: 0 dev.cxgbc.0.port0.mac_stats.rx_octets: 104065069 dev.cxgbc.0.port0.mac_stats.tx_frames_1519_max: 0 dev.cxgbc.0.port0.mac_stats.tx_frames_1024_1518: 0 dev.cxgbc.0.port0.mac_stats.tx_frames_512_1023: 0 dev.cxgbc.0.port0.mac_stats.tx_frames_256_511: 0 dev.cxgbc.0.port0.mac_stats.tx_frames_128_255: 1 dev.cxgbc.0.port0.mac_stats.tx_frames_65_127: 5 dev.cxgbc.0.port0.mac_stats.tx_frames_64: 4 dev.cxgbc.0.port0.mac_stats.tx_fcs_errs: 0 dev.cxgbc.0.port0.mac_stats.tx_excess_deferral: 0 dev.cxgbc.0.port0.mac_stats.tx_mac_internal_errs: 0 dev.cxgbc.0.port0.mac_stats.tx_len_errs: 0 dev.cxgbc.0.port0.mac_stats.tx_underrun: 0 dev.cxgbc.0.port0.mac_stats.tx_excess_collisions: 0 dev.cxgbc.0.port0.mac_stats.tx_total_collisions: 0 dev.cxgbc.0.port0.mac_stats.tx_late_collisions: 0 dev.cxgbc.0.port0.mac_stats.tx_deferred: 0 dev.cxgbc.0.port0.mac_stats.tx_pause: 0 dev.cxgbc.0.port0.mac_stats.tx_bcast_frames: 4 dev.cxgbc.0.port0.mac_stats.tx_mcast_frames: 6 dev.cxgbc.0.port0.mac_stats.tx_frames: 10 dev.cxgbc.0.port0.mac_stats.tx_octets_bad: 0 dev.cxgbc.0.port0.mac_stats.tx_octets: 940 dev.cxgbc.0.port0.qs3.lro_stats.lro_cnt: 8 dev.cxgbc.0.port0.qs3.lro_stats.lro_bad_csum: 0 dev.cxgbc.0.port0.qs3.lro_stats.lro_flushed: 0 dev.cxgbc.0.port0.qs3.lro_stats.lro_queued: 0 dev.cxgbc.0.port0.qs3.txq_ctrl.qdump: dev.cxgbc.0.port0.qs3.txq_ctrl.dump_count: 0 dev.cxgbc.0.port0.qs3.txq_ctrl.dump_start: 0 dev.cxgbc.0.port0.qs3.txq_eth.qdump: dev.cxgbc.0.port0.qs3.txq_eth.dump_count: 0 dev.cxgbc.0.port0.qs3.txq_eth.dump_start: 0 dev.cxgbc.0.port0.qs3.txq_eth.hw_pidx: 0 dev.cxgbc.0.port0.qs3.txq_eth.hw_cidx: 0 dev.cxgbc.0.port0.qs3.txq_eth.qgen: 1 dev.cxgbc.0.port0.qs3.txq_eth.phys_addr: 1752743936 dev.cxgbc.0.port0.qs3.txq_eth.stopped_flags: 0 dev.cxgbc.0.port0.qs3.txq_eth.enqueued: 0 dev.cxgbc.0.port0.qs3.txq_eth.coalesced: 0 dev.cxgbc.0.port0.qs3.txq_eth.skipped: 0 dev.cxgbc.0.port0.qs3.txq_eth.frees: 0 dev.cxgbc.0.port0.qs3.txq_eth.in_use: 0 dev.cxgbc.0.port0.qs3.txq_eth.cleaned: 0 dev.cxgbc.0.port0.qs3.txq_eth.processed: 0 dev.cxgbc.0.port0.qs3.txq_eth.sendqlen: 0 dev.cxgbc.0.port0.qs3.txq_eth.dropped: 0 dev.cxgbc.0.port0.qs3.rspq.qdump: dev.cxgbc.0.port0.qs3.rspq.dump_count: 0 dev.cxgbc.0.port0.qs3.rspq.dump_start: 0 dev.cxgbc.0.port0.qs3.rspq.phys_addr: 1752612864 dev.cxgbc.0.port0.qs3.rspq.starved: 0 dev.cxgbc.0.port0.qs3.rspq.credits: 1 dev.cxgbc.0.port0.qs3.rspq.cidx: 65 dev.cxgbc.0.port0.qs3.rspq.size: 2048 dev.cxgbc.0.port0.qs3.fl1_empty: 0 dev.cxgbc.0.port0.qs3.fl0_empty: 0 dev.cxgbc.0.port0.qs2.lro_stats.lro_cnt: 8 dev.cxgbc.0.port0.qs2.lro_stats.lro_bad_csum: 0 dev.cxgbc.0.port0.qs2.lro_stats.lro_flushed: 0 dev.cxgbc.0.port0.qs2.lro_stats.lro_queued: 0 dev.cxgbc.0.port0.qs2.txq_ctrl.qdump: dev.cxgbc.0.port0.qs2.txq_ctrl.dump_count: 0 dev.cxgbc.0.port0.qs2.txq_ctrl.dump_start: 0 dev.cxgbc.0.port0.qs2.txq_eth.qdump: dev.cxgbc.0.port0.qs2.txq_eth.dump_count: 0 dev.cxgbc.0.port0.qs2.txq_eth.dump_start: 0 dev.cxgbc.0.port0.qs2.txq_eth.hw_pidx: 0 dev.cxgbc.0.port0.qs2.txq_eth.hw_cidx: 0 dev.cxgbc.0.port0.qs2.txq_eth.qgen: 1 dev.cxgbc.0.port0.qs2.txq_eth.phys_addr: 1724973056 dev.cxgbc.0.port0.qs2.txq_eth.stopped_flags: 0 dev.cxgbc.0.port0.qs2.txq_eth.enqueued: 0 dev.cxgbc.0.port0.qs2.txq_eth.coalesced: 0 dev.cxgbc.0.port0.qs2.txq_eth.skipped: 0 dev.cxgbc.0.port0.qs2.txq_eth.frees: 0 dev.cxgbc.0.port0.qs2.txq_eth.in_use: 0 dev.cxgbc.0.port0.qs2.txq_eth.cleaned: 0 dev.cxgbc.0.port0.qs2.txq_eth.processed: 0 dev.cxgbc.0.port0.qs2.txq_eth.sendqlen: 0 dev.cxgbc.0.port0.qs2.txq_eth.dropped: 0 dev.cxgbc.0.port0.qs2.rspq.qdump: dev.cxgbc.0.port0.qs2.rspq.dump_count: 0 dev.cxgbc.0.port0.qs2.rspq.dump_start: 0 dev.cxgbc.0.port0.qs2.rspq.phys_addr: 1724841984 dev.cxgbc.0.port0.qs2.rspq.starved: 0 dev.cxgbc.0.port0.qs2.rspq.credits: 61 dev.cxgbc.0.port0.qs2.rspq.cidx: 1725 dev.cxgbc.0.port0.qs2.rspq.size: 2048 dev.cxgbc.0.port0.qs2.fl1_empty: 0 dev.cxgbc.0.port0.qs2.fl0_empty: 0 dev.cxgbc.0.port0.qs1.lro_stats.lro_cnt: 8 dev.cxgbc.0.port0.qs1.lro_stats.lro_bad_csum: 0 dev.cxgbc.0.port0.qs1.lro_stats.lro_flushed: 0 dev.cxgbc.0.port0.qs1.lro_stats.lro_queued: 0 dev.cxgbc.0.port0.qs1.txq_ctrl.qdump: dev.cxgbc.0.port0.qs1.txq_ctrl.dump_count: 0 dev.cxgbc.0.port0.qs1.txq_ctrl.dump_start: 0 dev.cxgbc.0.port0.qs1.txq_eth.qdump: dev.cxgbc.0.port0.qs1.txq_eth.dump_count: 0 dev.cxgbc.0.port0.qs1.txq_eth.dump_start: 0 dev.cxgbc.0.port0.qs1.txq_eth.hw_pidx: 0 dev.cxgbc.0.port0.qs1.txq_eth.hw_cidx: 0 dev.cxgbc.0.port0.qs1.txq_eth.qgen: 1 dev.cxgbc.0.port0.qs1.txq_eth.phys_addr: 1688813568 dev.cxgbc.0.port0.qs1.txq_eth.stopped_flags: 0 dev.cxgbc.0.port0.qs1.txq_eth.enqueued: 0 dev.cxgbc.0.port0.qs1.txq_eth.coalesced: 0 dev.cxgbc.0.port0.qs1.txq_eth.skipped: 0 dev.cxgbc.0.port0.qs1.txq_eth.frees: 0 dev.cxgbc.0.port0.qs1.txq_eth.in_use: 0 dev.cxgbc.0.port0.qs1.txq_eth.cleaned: 0 dev.cxgbc.0.port0.qs1.txq_eth.processed: 0 dev.cxgbc.0.port0.qs1.txq_eth.sendqlen: 0 dev.cxgbc.0.port0.qs1.txq_eth.dropped: 0 dev.cxgbc.0.port0.qs1.rspq.qdump: dev.cxgbc.0.port0.qs1.rspq.dump_count: 0 dev.cxgbc.0.port0.qs1.rspq.dump_start: 0 dev.cxgbc.0.port0.qs1.rspq.phys_addr: 1688682496 dev.cxgbc.0.port0.qs1.rspq.starved: 0 dev.cxgbc.0.port0.qs1.rspq.credits: 19 dev.cxgbc.0.port0.qs1.rspq.cidx: 723 dev.cxgbc.0.port0.qs1.rspq.size: 2048 dev.cxgbc.0.port0.qs1.fl1_empty: 0 dev.cxgbc.0.port0.qs1.fl0_empty: 0 dev.cxgbc.0.port0.qs0.lro_stats.lro_cnt: 8 dev.cxgbc.0.port0.qs0.lro_stats.lro_bad_csum: 0 dev.cxgbc.0.port0.qs0.lro_stats.lro_flushed: 0 dev.cxgbc.0.port0.qs0.lro_stats.lro_queued: 0 dev.cxgbc.0.port0.qs0.txq_ctrl.qdump: dev.cxgbc.0.port0.qs0.txq_ctrl.dump_count: 0 dev.cxgbc.0.port0.qs0.txq_ctrl.dump_start: 0 dev.cxgbc.0.port0.qs0.txq_eth.qdump: dev.cxgbc.0.port0.qs0.txq_eth.dump_count: 0 dev.cxgbc.0.port0.qs0.txq_eth.dump_start: 0 dev.cxgbc.0.port0.qs0.txq_eth.hw_pidx: 10 dev.cxgbc.0.port0.qs0.txq_eth.hw_cidx: 6 dev.cxgbc.0.port0.qs0.txq_eth.qgen: 1 dev.cxgbc.0.port0.qs0.txq_eth.phys_addr: 1679949824 dev.cxgbc.0.port0.qs0.txq_eth.stopped_flags: 0 dev.cxgbc.0.port0.qs0.txq_eth.enqueued: 0 dev.cxgbc.0.port0.qs0.txq_eth.coalesced: 0 dev.cxgbc.0.port0.qs0.txq_eth.skipped: 2 dev.cxgbc.0.port0.qs0.txq_eth.frees: 0 dev.cxgbc.0.port0.qs0.txq_eth.in_use: 4 dev.cxgbc.0.port0.qs0.txq_eth.cleaned: 6 dev.cxgbc.0.port0.qs0.txq_eth.processed: 10 dev.cxgbc.0.port0.qs0.txq_eth.sendqlen: 0 dev.cxgbc.0.port0.qs0.txq_eth.dropped: 0 dev.cxgbc.0.port0.qs0.rspq.qdump: dev.cxgbc.0.port0.qs0.rspq.dump_count: 0 dev.cxgbc.0.port0.qs0.rspq.dump_start: 0 dev.cxgbc.0.port0.qs0.rspq.phys_addr: 1679818752 dev.cxgbc.0.port0.qs0.rspq.starved: 0 dev.cxgbc.0.port0.qs0.rspq.credits: 28 dev.cxgbc.0.port0.qs0.rspq.cidx: 1820 dev.cxgbc.0.port0.qs0.rspq.size: 2048 dev.cxgbc.0.port0.qs0.fl1_empty: 0 dev.cxgbc.0.port0.qs0.fl0_empty: 0 dev.cxgbc.0.port0.nqsets: 4 dev.cxgbc.0.pkt_timestamp: 0 dev.cxgbc.0.intr_coal: 5 dev.cxgbc.0.wake: 0 dev.cxgbc.0.core_clock: 175000 dev.cxgbc.0.txq_overrun: 0 dev.cxgbc.0.tunq_coalesce: 0 dev.cxgbc.0.enable_debug: 0 dev.cxgbc.0.port_types: 5000 dev.cxgbc.0.hw_revision: 4 dev.cxgbc.0.firmware_version: 7.11.0 dev.cxgbc.0.%parent: pci7 dev.cxgbc.0.%pnpinfo: vendor=0x1425 device=0x0035 subvendor=0x1425 subdevice=0x0001 class=0x020000 dev.cxgbc.0.%location: pci0:7:0:0 handle=\_SB_.PCI0.RP09.PXSX dev.cxgbc.0.%driver: cxgbc dev.cxgbc.0.%desc: Chelsio T310 RNIC E/C: 110110730A0 S/N: PT25090230 dev.cxgbc.%parent:
Sep 6 20:10:09 kernel cxgb0: SR PHY module inserted Sep 6 20:10:09 check_reload_status Linkup starting cxgb0 Sep 6 20:10:09 kernel cxgb0: link state changed to UP Sep 6 20:10:09 kernel cxgb0_vlan20: link state changed to UP Sep 6 20:10:09 kernel cxgb0_vlan40: link state changed to UP Sep 6 20:10:09 kernel cxgb0_vlan10: link state changed to UP Sep 6 20:10:09 kernel cxgb0_vlan30: link state changed to UP Sep 6 20:10:09 check_reload_status Linkup starting cxgb0_vlan20 Sep 6 20:10:09 check_reload_status Linkup starting cxgb0_vlan40 Sep 6 20:10:09 check_reload_status Linkup starting cxgb0_vlan10 Sep 6 20:10:09 check_reload_status Linkup starting cxgb0_vlan30
Twinax cable:
dev.cxgbc.0.port0.mac_stats.link_faults: 0 dev.cxgbc.0.port0.mac_stats.num_resets: 0 dev.cxgbc.0.port0.mac_stats.num_toggled: 0 dev.cxgbc.0.port0.mac_stats.xaui_pcs_align_change: 2 dev.cxgbc.0.port0.mac_stats.xaui_pcs_ctc_err: 0 dev.cxgbc.0.port0.mac_stats.serdes_signal_loss: 2 dev.cxgbc.0.port0.mac_stats.rx_fifo_ovfl: 0 dev.cxgbc.0.port0.mac_stats.tx_fifo_urun: 0 dev.cxgbc.0.port0.mac_stats.rx_fifo_parity_err: 0 dev.cxgbc.0.port0.mac_stats.tx_fifo_parity_err: 0 dev.cxgbc.0.port0.mac_stats.rx_frames_1519_max: 0 dev.cxgbc.0.port0.mac_stats.rx_frames_1024_1518: 0 dev.cxgbc.0.port0.mac_stats.rx_frames_512_1023: 10 dev.cxgbc.0.port0.mac_stats.rx_frames_256_511: 1203 dev.cxgbc.0.port0.mac_stats.rx_frames_128_255: 24999 dev.cxgbc.0.port0.mac_stats.rx_frames_65_127: 67422 dev.cxgbc.0.port0.mac_stats.rx_frames_64: 1459457 dev.cxgbc.0.port0.mac_stats.rx_cong_drops: 0 dev.cxgbc.0.port0.mac_stats.rx_mac_internal_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_too_long: 0 dev.cxgbc.0.port0.mac_stats.rx_short: 0 dev.cxgbc.0.port0.mac_stats.rx_jabber: 0 dev.cxgbc.0.port0.mac_stats.rx_runt: 0 dev.cxgbc.0.port0.mac_stats.rx_sequence_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_data_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_symbol_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_align_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_fcs_errs: 0 dev.cxgbc.0.port0.mac_stats.rx_pause: 0 dev.cxgbc.0.port0.mac_stats.rx_bcast_frames: 1553088 dev.cxgbc.0.port0.mac_stats.rx_mcast_frames: 3 dev.cxgbc.0.port0.mac_stats.rx_frames: 1553091 dev.cxgbc.0.port0.mac_stats.rx_octets_bad: 0 dev.cxgbc.0.port0.mac_stats.rx_octets: 104237663 dev.cxgbc.0.port0.mac_stats.tx_frames_1519_max: 0 dev.cxgbc.0.port0.mac_stats.tx_frames_1024_1518: 0 dev.cxgbc.0.port0.mac_stats.tx_frames_512_1023: 0 dev.cxgbc.0.port0.mac_stats.tx_frames_256_511: 0 dev.cxgbc.0.port0.mac_stats.tx_frames_128_255: 1 dev.cxgbc.0.port0.mac_stats.tx_frames_65_127: 5 dev.cxgbc.0.port0.mac_stats.tx_frames_64: 4 dev.cxgbc.0.port0.mac_stats.tx_fcs_errs: 0 dev.cxgbc.0.port0.mac_stats.tx_excess_deferral: 0 dev.cxgbc.0.port0.mac_stats.tx_mac_internal_errs: 0 dev.cxgbc.0.port0.mac_stats.tx_len_errs: 0 dev.cxgbc.0.port0.mac_stats.tx_underrun: 0 dev.cxgbc.0.port0.mac_stats.tx_excess_collisions: 0 dev.cxgbc.0.port0.mac_stats.tx_total_collisions: 0 dev.cxgbc.0.port0.mac_stats.tx_late_collisions: 0 dev.cxgbc.0.port0.mac_stats.tx_deferred: 0 dev.cxgbc.0.port0.mac_stats.tx_pause: 0 dev.cxgbc.0.port0.mac_stats.tx_bcast_frames: 4 dev.cxgbc.0.port0.mac_stats.tx_mcast_frames: 6 dev.cxgbc.0.port0.mac_stats.tx_frames: 10 dev.cxgbc.0.port0.mac_stats.tx_octets_bad: 0 dev.cxgbc.0.port0.mac_stats.tx_octets: 940 dev.cxgbc.0.port0.qs3.lro_stats.lro_cnt: 8 dev.cxgbc.0.port0.qs3.lro_stats.lro_bad_csum: 0 dev.cxgbc.0.port0.qs3.lro_stats.lro_flushed: 0 dev.cxgbc.0.port0.qs3.lro_stats.lro_queued: 0 dev.cxgbc.0.port0.qs3.txq_ctrl.qdump: dev.cxgbc.0.port0.qs3.txq_ctrl.dump_count: 0 dev.cxgbc.0.port0.qs3.txq_ctrl.dump_start: 0 dev.cxgbc.0.port0.qs3.txq_eth.qdump: dev.cxgbc.0.port0.qs3.txq_eth.dump_count: 0 dev.cxgbc.0.port0.qs3.txq_eth.dump_start: 0 dev.cxgbc.0.port0.qs3.txq_eth.hw_pidx: 0 dev.cxgbc.0.port0.qs3.txq_eth.hw_cidx: 0 dev.cxgbc.0.port0.qs3.txq_eth.qgen: 1 dev.cxgbc.0.port0.qs3.txq_eth.phys_addr: 1752743936 dev.cxgbc.0.port0.qs3.txq_eth.stopped_flags: 0 dev.cxgbc.0.port0.qs3.txq_eth.enqueued: 0 dev.cxgbc.0.port0.qs3.txq_eth.coalesced: 0 dev.cxgbc.0.port0.qs3.txq_eth.skipped: 0 dev.cxgbc.0.port0.qs3.txq_eth.frees: 0 dev.cxgbc.0.port0.qs3.txq_eth.in_use: 0 dev.cxgbc.0.port0.qs3.txq_eth.cleaned: 0 dev.cxgbc.0.port0.qs3.txq_eth.processed: 0 dev.cxgbc.0.port0.qs3.txq_eth.sendqlen: 0 dev.cxgbc.0.port0.qs3.txq_eth.dropped: 0 dev.cxgbc.0.port0.qs3.rspq.qdump: dev.cxgbc.0.port0.qs3.rspq.dump_count: 0 dev.cxgbc.0.port0.qs3.rspq.dump_start: 0 dev.cxgbc.0.port0.qs3.rspq.phys_addr: 1752612864 dev.cxgbc.0.port0.qs3.rspq.starved: 0 dev.cxgbc.0.port0.qs3.rspq.credits: 29 dev.cxgbc.0.port0.qs3.rspq.cidx: 93 dev.cxgbc.0.port0.qs3.rspq.size: 2048 dev.cxgbc.0.port0.qs3.fl1_empty: 0 dev.cxgbc.0.port0.qs3.fl0_empty: 0 dev.cxgbc.0.port0.qs2.lro_stats.lro_cnt: 8 dev.cxgbc.0.port0.qs2.lro_stats.lro_bad_csum: 0 dev.cxgbc.0.port0.qs2.lro_stats.lro_flushed: 0 dev.cxgbc.0.port0.qs2.lro_stats.lro_queued: 0 dev.cxgbc.0.port0.qs2.txq_ctrl.qdump: dev.cxgbc.0.port0.qs2.txq_ctrl.dump_count: 0 dev.cxgbc.0.port0.qs2.txq_ctrl.dump_start: 0 dev.cxgbc.0.port0.qs2.txq_eth.qdump: dev.cxgbc.0.port0.qs2.txq_eth.dump_count: 0 dev.cxgbc.0.port0.qs2.txq_eth.dump_start: 0 dev.cxgbc.0.port0.qs2.txq_eth.hw_pidx: 0 dev.cxgbc.0.port0.qs2.txq_eth.hw_cidx: 0 dev.cxgbc.0.port0.qs2.txq_eth.qgen: 1 dev.cxgbc.0.port0.qs2.txq_eth.phys_addr: 1724973056 dev.cxgbc.0.port0.qs2.txq_eth.stopped_flags: 0 dev.cxgbc.0.port0.qs2.txq_eth.enqueued: 0 dev.cxgbc.0.port0.qs2.txq_eth.coalesced: 0 dev.cxgbc.0.port0.qs2.txq_eth.skipped: 0 dev.cxgbc.0.port0.qs2.txq_eth.frees: 0 dev.cxgbc.0.port0.qs2.txq_eth.in_use: 0 dev.cxgbc.0.port0.qs2.txq_eth.cleaned: 0 dev.cxgbc.0.port0.qs2.txq_eth.processed: 0 dev.cxgbc.0.port0.qs2.txq_eth.sendqlen: 0 dev.cxgbc.0.port0.qs2.txq_eth.dropped: 0 dev.cxgbc.0.port0.qs2.rspq.qdump: dev.cxgbc.0.port0.qs2.rspq.dump_count: 0 dev.cxgbc.0.port0.qs2.rspq.dump_start: 0 dev.cxgbc.0.port0.qs2.rspq.phys_addr: 1724841984 dev.cxgbc.0.port0.qs2.rspq.starved: 0 dev.cxgbc.0.port0.qs2.rspq.credits: 4 dev.cxgbc.0.port0.qs2.rspq.cidx: 1732 dev.cxgbc.0.port0.qs2.rspq.size: 2048 dev.cxgbc.0.port0.qs2.fl1_empty: 0 dev.cxgbc.0.port0.qs2.fl0_empty: 0 dev.cxgbc.0.port0.qs1.lro_stats.lro_cnt: 8 dev.cxgbc.0.port0.qs1.lro_stats.lro_bad_csum: 0 dev.cxgbc.0.port0.qs1.lro_stats.lro_flushed: 0 dev.cxgbc.0.port0.qs1.lro_stats.lro_queued: 0 dev.cxgbc.0.port0.qs1.txq_ctrl.qdump: dev.cxgbc.0.port0.qs1.txq_ctrl.dump_count: 0 dev.cxgbc.0.port0.qs1.txq_ctrl.dump_start: 0 dev.cxgbc.0.port0.qs1.txq_eth.qdump: dev.cxgbc.0.port0.qs1.txq_eth.dump_count: 0 dev.cxgbc.0.port0.qs1.txq_eth.dump_start: 0 dev.cxgbc.0.port0.qs1.txq_eth.hw_pidx: 0 dev.cxgbc.0.port0.qs1.txq_eth.hw_cidx: 0 dev.cxgbc.0.port0.qs1.txq_eth.qgen: 1 dev.cxgbc.0.port0.qs1.txq_eth.phys_addr: 1688813568 dev.cxgbc.0.port0.qs1.txq_eth.stopped_flags: 0 dev.cxgbc.0.port0.qs1.txq_eth.enqueued: 0 dev.cxgbc.0.port0.qs1.txq_eth.coalesced: 0 dev.cxgbc.0.port0.qs1.txq_eth.skipped: 0 dev.cxgbc.0.port0.qs1.txq_eth.frees: 0 dev.cxgbc.0.port0.qs1.txq_eth.in_use: 0 dev.cxgbc.0.port0.qs1.txq_eth.cleaned: 0 dev.cxgbc.0.port0.qs1.txq_eth.processed: 0 dev.cxgbc.0.port0.qs1.txq_eth.sendqlen: 0 dev.cxgbc.0.port0.qs1.txq_eth.dropped: 0 dev.cxgbc.0.port0.qs1.rspq.qdump: dev.cxgbc.0.port0.qs1.rspq.dump_count: 0 dev.cxgbc.0.port0.qs1.rspq.dump_start: 0 dev.cxgbc.0.port0.qs1.rspq.phys_addr: 1688682496 dev.cxgbc.0.port0.qs1.rspq.starved: 0 dev.cxgbc.0.port0.qs1.rspq.credits: 54 dev.cxgbc.0.port0.qs1.rspq.cidx: 758 dev.cxgbc.0.port0.qs1.rspq.size: 2048 dev.cxgbc.0.port0.qs1.fl1_empty: 0 dev.cxgbc.0.port0.qs1.fl0_empty: 0 dev.cxgbc.0.port0.qs0.lro_stats.lro_cnt: 8 dev.cxgbc.0.port0.qs0.lro_stats.lro_bad_csum: 0 dev.cxgbc.0.port0.qs0.lro_stats.lro_flushed: 0 dev.cxgbc.0.port0.qs0.lro_stats.lro_queued: 0 dev.cxgbc.0.port0.qs0.txq_ctrl.qdump: dev.cxgbc.0.port0.qs0.txq_ctrl.dump_count: 0 dev.cxgbc.0.port0.qs0.txq_ctrl.dump_start: 0 dev.cxgbc.0.port0.qs0.txq_eth.qdump: dev.cxgbc.0.port0.qs0.txq_eth.dump_count: 0 dev.cxgbc.0.port0.qs0.txq_eth.dump_start: 0 dev.cxgbc.0.port0.qs0.txq_eth.hw_pidx: 10 dev.cxgbc.0.port0.qs0.txq_eth.hw_cidx: 6 dev.cxgbc.0.port0.qs0.txq_eth.qgen: 1 dev.cxgbc.0.port0.qs0.txq_eth.phys_addr: 1679949824 dev.cxgbc.0.port0.qs0.txq_eth.stopped_flags: 0 dev.cxgbc.0.port0.qs0.txq_eth.enqueued: 0 dev.cxgbc.0.port0.qs0.txq_eth.coalesced: 0 dev.cxgbc.0.port0.qs0.txq_eth.skipped: 2 dev.cxgbc.0.port0.qs0.txq_eth.frees: 0 dev.cxgbc.0.port0.qs0.txq_eth.in_use: 4 dev.cxgbc.0.port0.qs0.txq_eth.cleaned: 6 dev.cxgbc.0.port0.qs0.txq_eth.processed: 10 dev.cxgbc.0.port0.qs0.txq_eth.sendqlen: 0 dev.cxgbc.0.port0.qs0.txq_eth.dropped: 0 dev.cxgbc.0.port0.qs0.rspq.qdump: dev.cxgbc.0.port0.qs0.rspq.dump_count: 0 dev.cxgbc.0.port0.qs0.rspq.dump_start: 0 dev.cxgbc.0.port0.qs0.rspq.phys_addr: 1679818752 dev.cxgbc.0.port0.qs0.rspq.starved: 0 dev.cxgbc.0.port0.qs0.rspq.credits: 22 dev.cxgbc.0.port0.qs0.rspq.cidx: 214 dev.cxgbc.0.port0.qs0.rspq.size: 2048 dev.cxgbc.0.port0.qs0.fl1_empty: 0 dev.cxgbc.0.port0.qs0.fl0_empty: 0 dev.cxgbc.0.port0.nqsets: 4 dev.cxgbc.0.pkt_timestamp: 0 dev.cxgbc.0.intr_coal: 5 dev.cxgbc.0.wake: 0 dev.cxgbc.0.core_clock: 175000 dev.cxgbc.0.txq_overrun: 0 dev.cxgbc.0.tunq_coalesce: 0 dev.cxgbc.0.enable_debug: 0 dev.cxgbc.0.port_types: 5000 dev.cxgbc.0.hw_revision: 4 dev.cxgbc.0.firmware_version: 7.11.0 dev.cxgbc.0.%parent: pci7 dev.cxgbc.0.%pnpinfo: vendor=0x1425 device=0x0035 subvendor=0x1425 subdevice=0x0001 class=0x020000 dev.cxgbc.0.%location: pci0:7:0:0 handle=\_SB_.PCI0.RP09.PXSX dev.cxgbc.0.%driver: cxgbc dev.cxgbc.0.%desc: Chelsio T310 RNIC E/C: 110110730A0 S/N: PT25090230 dev.cxgbc.%parent:
Sep 6 20:01:27 kernel cxgb0: TWINAX PHY module inserted
-
Hmmm. Have you seen this with multiple twinax cables?
You might try setting dev.cxgbc.0.enable_debug to 1 to see if you get any additional output.
Steve
-
Twinax with Cisco receivers:
cxgb0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 options=6c07bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6>ether 00:07:43:05:cc:9f hwaddr 00:07:43:05:cc:9f inet6 fe80::207:43ff:fe05:cc9f%cxgb0 prefixlen 64 scopeid 0x1 nd6 options=21 <performnud,auto_linklocal>media: Ethernet 10Gbase-Twinax <full-duplex> status: active</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,simplex,multicast>
Problem could be either receivers or cable.
I did not try debug command.
-
Well I'd try the debug command as there's not much to lose. But I wouldn't hold out too much hope there.
In my experience the Chelsio cards have been pretty forgiving, they work with most things. Clearly not that, but without more logs to go on I'm not sure what we can do.
Steve
-
I reinstalled the problem Twinax cable:
cxgb0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500 options=6c07bb <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6>ether 00:07:43:05:cc:9f hwaddr 00:07:43:05:cc:9f inet6 fe80::207:43ff:fe05:cc9f%cxgb0 prefixlen 64 scopeid 0x1 nd6 options=21 <performnud,auto_linklocal>media: Ethernet 10Gbase-Twinax <full-duplex> status: active</full-duplex></performnud,auto_linklocal></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,vlan_hwcsum,tso4,tso6,lro,vlan_hwtso,linkstate,rxcsum_ipv6,txcsum_ipv6></up,broadcast,running,simplex,multicast>
I'll put it under "User Error" category.
-
Ah! ;)
Good to know anyway, thanks.
Steve