Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Tunable/sysctl for Mellanox mlx4 driver to disable flow control?

    Scheduled Pinned Locked Moved L2/Switching/VLANs
    12 Posts 2 Posters 2.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • X
      xpxp2002
      last edited by

      Since bringing my NIC interfaces into pfSense as virtual functions with 22.05, they have been showing up with rxpause and txpause enabled by default.

      I found this page below with some tunables that disable flow control on some NICs. However, there doesn't seem to be a known working setting for Mellanox Connect-X 3 Pro NICs, which uses the mlx4 driver.

      If I apply a change to an interface, such as adjusting the speed/duplex setting, it will reapply the settings with flow control disabled. But upon reboot, those settings return.

      I've been searching around, quite literally, for months for a permanent solution to keep flow control disabled, but I haven't been able to find any config settings or tunables that apply for this NIC. Any help would be appreciated.

      https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#general-issues

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by stephenw10

        You tried the loader variable shown for Chelsio on that page? And it didn't work as expected:
        https://docs.netgate.com/pfsense/en/latest/hardware/tune.html#flow-control

        Steve

        X 1 Reply Last reply Reply Quote 0
        • X
          xpxp2002 @stephenw10
          last edited by

          @stephenw10 Correct. I finally had a chance to reboot this instance and try the Chelsio entry.

          I configured the cxgbe in loader.conf.local, but it did not change the bootup state of the interfaces.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Sorry I'm not sure why I read Mellanox as Chelsio there. 🙄

            First run ifconfig -vvvm mlxen0 and see what media types it reports as capable.

            Then look for any sysctls that may set that:
            sysctl hw.mlxen
            sysctl dev.mlxen.0

            Steve

            X 1 Reply Last reply Reply Quote 0
            • X
              xpxp2002 @stephenw10
              last edited by

              @stephenw10 I appreciate your response after I had to wait for an opportunity to test again. In the meantime, I've been perusing forums and searching the web looking for anybody else who has run into this, but so far have come up empty. I feel like there must be a way to do it. I'm just thinking it's not as well documented for the mlx drivers. I know they're not used this way with pfSense as often as some other NIC brands.

              The complete command output is below, only redacting MAC address and the terminal prompt with the pfSense hostname.

              The odd thing is that if I go into an interface in pfSense and make any setting change, even something as benign as setting a static IP or switching back to DHCP, and when the interface is reconfigured, it auto selects "10Gbase-CX4 <full-duplex>" without flow control. It is only being enabled on boot.

              I've also tried statically setting the link state to 10Gbase-CX4 full, which also works at first to remove rxpause and txpause because it reconfigures the interface. But after rebooting, rxpause and txpause come back.

              : ifconfig -vvvm mlxen0
mlxen0: flags=8a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=cc00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>
	capabilities=ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	ether xx:xx:xx:xx:xx:xx
	media: Ethernet autoselect (10Gbase-CX4 <full-duplex,rxpause,txpause>)
	status: active
	supported media:
		media autoselect
		media 40Gbase-CR4 mediaopt full-duplex
		media 10Gbase-CX4 mediaopt full-duplex
		media 10Gbase-SR mediaopt full-duplex
		media 1000baseT mediaopt full-duplex
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

: sysctl hw.mlxen
sysctl: unknown oid 'hw.mlxen'

: sysctl dev.mlxen.0
sysctl: unknown oid 'dev.mlxen.0'
              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by stephenw10

                Hmm, I don't have one of those cards to reference.

                Try: sysctl -a | grep mlx

                That will probably return a lot!

                X 1 Reply Last reply Reply Quote 0
                • X
                  xpxp2002 @stephenw10
                  last edited by

                  Indeed, that's a lot. I searched for the phrases "pause" and "flow" but those didn't turn up anything. "fc" turned up these two entries, which are for priority-based flow control, rather than "global pause," which is what Mellanox apparently calls traditional port-based flow control. And they appear to be disabled, anyway.

                  compat.linuxkpi.mlx4_pfcrx: 0
compat.linuxkpi.mlx4_pfctx: 0

                  https://support.mellanox.com/s/article/network-considerations-for-global-pause--pfc-and-qos-with-mellanox-switches-and-adapters

                  Here's the entire output in case there's another string I should be searching for. I'm hitting a 32768 character limit on posts, so I'll break this up into two posts.

                  : sysctl -a | grep mlx
device	mlx
device	mlx5
device	mlxfw
device	mlx5en
device	mlx4
device	mlx4en
hw.mlxen12.stat.rx_ring7.error: 0
hw.mlxen12.stat.rx_ring7.bytes: 0
hw.mlxen12.stat.rx_ring7.packets: 0
hw.mlxen12.stat.rx_ring6.error: 0
hw.mlxen12.stat.rx_ring6.bytes: 0
hw.mlxen12.stat.rx_ring6.packets: 0
hw.mlxen12.stat.rx_ring5.error: 0
hw.mlxen12.stat.rx_ring5.bytes: 0
hw.mlxen12.stat.rx_ring5.packets: 0
hw.mlxen12.stat.rx_ring4.error: 0
hw.mlxen12.stat.rx_ring4.bytes: 0
hw.mlxen12.stat.rx_ring4.packets: 0
hw.mlxen12.stat.rx_ring3.error: 0
hw.mlxen12.stat.rx_ring3.bytes: 0
hw.mlxen12.stat.rx_ring3.packets: 0
hw.mlxen12.stat.rx_ring2.error: 0
hw.mlxen12.stat.rx_ring2.bytes: 0
hw.mlxen12.stat.rx_ring2.packets: 0
hw.mlxen12.stat.rx_ring1.error: 0
hw.mlxen12.stat.rx_ring1.bytes: 0
hw.mlxen12.stat.rx_ring1.packets: 0
hw.mlxen12.stat.rx_ring0.error: 0
hw.mlxen12.stat.rx_ring0.bytes: 0
hw.mlxen12.stat.rx_ring0.packets: 0
hw.mlxen12.stat.tx_ring7.defrag_attempts: 0
hw.mlxen12.stat.tx_ring7.tso_packets: 0
hw.mlxen12.stat.tx_ring7.bytes: 0
hw.mlxen12.stat.tx_ring7.packets: 0
hw.mlxen12.stat.tx_ring6.defrag_attempts: 0
hw.mlxen12.stat.tx_ring6.tso_packets: 0
hw.mlxen12.stat.tx_ring6.bytes: 30858
hw.mlxen12.stat.tx_ring6.packets: 139
hw.mlxen12.stat.tx_ring5.defrag_attempts: 0
hw.mlxen12.stat.tx_ring5.tso_packets: 0
hw.mlxen12.stat.tx_ring5.bytes: 0
hw.mlxen12.stat.tx_ring5.packets: 0
hw.mlxen12.stat.tx_ring4.defrag_attempts: 0
hw.mlxen12.stat.tx_ring4.tso_packets: 0
hw.mlxen12.stat.tx_ring4.bytes: 0
hw.mlxen12.stat.tx_ring4.packets: 0
hw.mlxen12.stat.tx_ring3.defrag_attempts: 0
hw.mlxen12.stat.tx_ring3.tso_packets: 0
hw.mlxen12.stat.tx_ring3.bytes: 0
hw.mlxen12.stat.tx_ring3.packets: 0
hw.mlxen12.stat.tx_ring2.defrag_attempts: 0
hw.mlxen12.stat.tx_ring2.tso_packets: 0
hw.mlxen12.stat.tx_ring2.bytes: 0
hw.mlxen12.stat.tx_ring2.packets: 0
hw.mlxen12.stat.tx_ring1.defrag_attempts: 0
hw.mlxen12.stat.tx_ring1.tso_packets: 0
hw.mlxen12.stat.tx_ring1.bytes: 260
hw.mlxen12.stat.tx_ring1.packets: 2
hw.mlxen12.stat.tx_ring0.defrag_attempts: 0
hw.mlxen12.stat.tx_ring0.tso_packets: 0
hw.mlxen12.stat.tx_ring0.bytes: 0
hw.mlxen12.stat.tx_ring0.packets: 0
hw.mlxen12.stat.tx_gt_1548_bytes_packets: 0
hw.mlxen12.stat.tx_1548_bytes_packets: 0
hw.mlxen12.stat.tx_1522_bytes_packets: 0
hw.mlxen12.stat.tx_1518_bytes_packets: 0
hw.mlxen12.stat.tx_1023_bytes_packets: 0
hw.mlxen12.stat.tx_511_bytes_packets: 0
hw.mlxen12.stat.tx_255_bytes_packets: 0
hw.mlxen12.stat.tx_127_bytes_packets: 0
hw.mlxen12.stat.tx_lt_64_bytes_packets: 0
hw.mlxen12.stat.tx_dropped: 0
hw.mlxen12.stat.tx_errors: 0
hw.mlxen12.stat.tx_broadcast_packets: 0
hw.mlxen12.stat.tx_multicast_packets: 0
hw.mlxen12.stat.tx_bytes: 0
hw.mlxen12.stat.tx_packets: 0
hw.mlxen12.stat.rx_gt_1548_bytes_packets: 0
hw.mlxen12.stat.rx_1548_bytes_packets: 0
hw.mlxen12.stat.rx_1522_bytes_packets: 0
hw.mlxen12.stat.rx_1518_bytes_packets: 0
hw.mlxen12.stat.rx_1023_bytes_packets: 0
hw.mlxen12.stat.rx_511_bytes_packets: 0
hw.mlxen12.stat.rx_255_bytes_packets: 0
hw.mlxen12.stat.rx_127_bytes_packets: 0
hw.mlxen12.stat.rx_lt_64_bytes_packets: 0
hw.mlxen12.stat.rx_out_range_length_error: 0
hw.mlxen12.stat.rx_in_range_length_error: 0
hw.mlxen12.stat.rx_jabbers: 0
hw.mlxen12.stat.rx_crc_errors: 0
hw.mlxen12.stat.rx_over_errors: 0
hw.mlxen12.stat.rx_length_errors: 0
hw.mlxen12.stat.rx_dropped: 0
hw.mlxen12.stat.rx_errors: 0
hw.mlxen12.stat.rx_broadcast_packets: 0
hw.mlxen12.stat.rx_multicast_packets: 0
hw.mlxen12.stat.rx_packets: 0
hw.mlxen12.stat.rx_bytes: 0
hw.mlxen12.stat.defrag_attempts: 0
hw.mlxen12.stat.tx_chksum_offload: 0
hw.mlxen12.stat.rx_chksum_none: 0
hw.mlxen12.stat.rx_chksum_good: 0
hw.mlxen12.stat.rx_alloc_failed: 0
hw.mlxen12.stat.tx_oversized_packets: 0
hw.mlxen12.stat.tx_timeout: 0
hw.mlxen12.stat.wake_queue: 0
hw.mlxen12.stat.queue_stopped: 0
hw.mlxen12.stat.tso_packets: 0
hw.mlxen12.conf.eeprom_info: 0
hw.mlxen12.conf.coalesce.adaptive_rx_coal: 1
hw.mlxen12.conf.coalesce.sample_interval: 0
hw.mlxen12.conf.coalesce.rx_usecs_high: 128
hw.mlxen12.conf.coalesce.pkt_rate_high: 450000
hw.mlxen12.conf.coalesce.rx_usecs_low: 0
hw.mlxen12.conf.coalesce.pkt_rate_low: 400000
hw.mlxen12.conf.device_name: mlx4_core6
hw.mlxen12.conf.port_num: 1
hw.mlxen12.conf.rx_ppp: 0
hw.mlxen12.conf.tx_ppp: 0
hw.mlxen12.conf.tx_size: 1024
hw.mlxen12.conf.rx_size: 1024
hw.mlxen12.conf.tx_rings: 8
hw.mlxen12.conf.rx_rings: 8
hw.mlxen12.conf.msg_enable: 20
hw.mlxen10.stat.rx_ring7.error: 0
hw.mlxen10.stat.rx_ring7.bytes: 0
hw.mlxen10.stat.rx_ring7.packets: 0
hw.mlxen10.stat.rx_ring6.error: 0
hw.mlxen10.stat.rx_ring6.bytes: 0
hw.mlxen10.stat.rx_ring6.packets: 0
hw.mlxen10.stat.rx_ring5.error: 0
hw.mlxen10.stat.rx_ring5.bytes: 0
hw.mlxen10.stat.rx_ring5.packets: 0
hw.mlxen10.stat.rx_ring4.error: 0
hw.mlxen10.stat.rx_ring4.bytes: 0
hw.mlxen10.stat.rx_ring4.packets: 0
hw.mlxen10.stat.rx_ring3.error: 0
hw.mlxen10.stat.rx_ring3.bytes: 0
hw.mlxen10.stat.rx_ring3.packets: 0
hw.mlxen10.stat.rx_ring2.error: 0
hw.mlxen10.stat.rx_ring2.bytes: 0
hw.mlxen10.stat.rx_ring2.packets: 0
hw.mlxen10.stat.rx_ring1.error: 0
hw.mlxen10.stat.rx_ring1.bytes: 0
hw.mlxen10.stat.rx_ring1.packets: 0
hw.mlxen10.stat.rx_ring0.error: 0
hw.mlxen10.stat.rx_ring0.bytes: 0
hw.mlxen10.stat.rx_ring0.packets: 0
hw.mlxen10.stat.tx_ring7.defrag_attempts: 0
hw.mlxen10.stat.tx_ring7.tso_packets: 0
hw.mlxen10.stat.tx_ring7.bytes: 2290126
hw.mlxen10.stat.tx_ring7.packets: 10937
hw.mlxen10.stat.tx_ring6.defrag_attempts: 0
hw.mlxen10.stat.tx_ring6.tso_packets: 0
hw.mlxen10.stat.tx_ring6.bytes: 349781
hw.mlxen10.stat.tx_ring6.packets: 1190
hw.mlxen10.stat.tx_ring5.defrag_attempts: 0
hw.mlxen10.stat.tx_ring5.tso_packets: 0
hw.mlxen10.stat.tx_ring5.bytes: 62398708
hw.mlxen10.stat.tx_ring5.packets: 45873
hw.mlxen10.stat.tx_ring4.defrag_attempts: 0
hw.mlxen10.stat.tx_ring4.tso_packets: 0
hw.mlxen10.stat.tx_ring4.bytes: 7475064
hw.mlxen10.stat.tx_ring4.packets: 6471
hw.mlxen10.stat.tx_ring3.defrag_attempts: 0
hw.mlxen10.stat.tx_ring3.tso_packets: 0
hw.mlxen10.stat.tx_ring3.bytes: 1979474
hw.mlxen10.stat.tx_ring3.packets: 2827
hw.mlxen10.stat.tx_ring2.defrag_attempts: 0
hw.mlxen10.stat.tx_ring2.tso_packets: 0
hw.mlxen10.stat.tx_ring2.bytes: 165886000
hw.mlxen10.stat.tx_ring2.packets: 130742
hw.mlxen10.stat.tx_ring1.defrag_attempts: 0
hw.mlxen10.stat.tx_ring1.tso_packets: 0
hw.mlxen10.stat.tx_ring1.bytes: 1840429
hw.mlxen10.stat.tx_ring1.packets: 6843
hw.mlxen10.stat.tx_ring0.defrag_attempts: 0
hw.mlxen10.stat.tx_ring0.tso_packets: 0
hw.mlxen10.stat.tx_ring0.bytes: 205740655
hw.mlxen10.stat.tx_ring0.packets: 137831
hw.mlxen10.stat.tx_gt_1548_bytes_packets: 0
hw.mlxen10.stat.tx_1548_bytes_packets: 0
hw.mlxen10.stat.tx_1522_bytes_packets: 0
hw.mlxen10.stat.tx_1518_bytes_packets: 0
hw.mlxen10.stat.tx_1023_bytes_packets: 0
hw.mlxen10.stat.tx_511_bytes_packets: 0
hw.mlxen10.stat.tx_255_bytes_packets: 0
hw.mlxen10.stat.tx_127_bytes_packets: 0
hw.mlxen10.stat.tx_lt_64_bytes_packets: 0
hw.mlxen10.stat.tx_dropped: 0
hw.mlxen10.stat.tx_errors: 0
hw.mlxen10.stat.tx_broadcast_packets: 0
hw.mlxen10.stat.tx_multicast_packets: 0
hw.mlxen10.stat.tx_bytes: 0
hw.mlxen10.stat.tx_packets: 0
hw.mlxen10.stat.rx_gt_1548_bytes_packets: 0
hw.mlxen10.stat.rx_1548_bytes_packets: 0
hw.mlxen10.stat.rx_1522_bytes_packets: 0
hw.mlxen10.stat.rx_1518_bytes_packets: 0
hw.mlxen10.stat.rx_1023_bytes_packets: 0
hw.mlxen10.stat.rx_511_bytes_packets: 0
hw.mlxen10.stat.rx_255_bytes_packets: 0
hw.mlxen10.stat.rx_127_bytes_packets: 0
hw.mlxen10.stat.rx_lt_64_bytes_packets: 0
hw.mlxen10.stat.rx_out_range_length_error: 0
hw.mlxen10.stat.rx_in_range_length_error: 0
hw.mlxen10.stat.rx_jabbers: 0
hw.mlxen10.stat.rx_crc_errors: 0
hw.mlxen10.stat.rx_over_errors: 0
hw.mlxen10.stat.rx_length_errors: 0
hw.mlxen10.stat.rx_dropped: 0
hw.mlxen10.stat.rx_errors: 0
hw.mlxen10.stat.rx_broadcast_packets: 0
hw.mlxen10.stat.rx_multicast_packets: 0
hw.mlxen10.stat.rx_packets: 0
hw.mlxen10.stat.rx_bytes: 0
hw.mlxen10.stat.defrag_attempts: 0
hw.mlxen10.stat.tx_chksum_offload: 165637
hw.mlxen10.stat.rx_chksum_none: 0
hw.mlxen10.stat.rx_chksum_good: 0
hw.mlxen10.stat.rx_alloc_failed: 0
hw.mlxen10.stat.tx_oversized_packets: 0
hw.mlxen10.stat.tx_timeout: 0
hw.mlxen10.stat.wake_queue: 0
hw.mlxen10.stat.queue_stopped: 0
hw.mlxen10.stat.tso_packets: 0
hw.mlxen10.conf.eeprom_info: 0
hw.mlxen10.conf.coalesce.adaptive_rx_coal: 1
hw.mlxen10.conf.coalesce.sample_interval: 0
hw.mlxen10.conf.coalesce.rx_usecs_high: 128
hw.mlxen10.conf.coalesce.pkt_rate_high: 450000
hw.mlxen10.conf.coalesce.rx_usecs_low: 0
hw.mlxen10.conf.coalesce.pkt_rate_low: 400000
hw.mlxen10.conf.device_name: mlx4_core5
hw.mlxen10.conf.port_num: 1
hw.mlxen10.conf.rx_ppp: 0
hw.mlxen10.conf.tx_ppp: 0
hw.mlxen10.conf.tx_size: 1024
hw.mlxen10.conf.rx_size: 1024
hw.mlxen10.conf.tx_rings: 8
hw.mlxen10.conf.rx_rings: 8
hw.mlxen10.conf.msg_enable: 20
hw.mlxen8.stat.rx_ring7.error: 0
hw.mlxen8.stat.rx_ring7.bytes: 3475080
hw.mlxen8.stat.rx_ring7.packets: 5925
hw.mlxen8.stat.rx_ring6.error: 0
hw.mlxen8.stat.rx_ring6.bytes: 21369
hw.mlxen8.stat.rx_ring6.packets: 165
hw.mlxen8.stat.rx_ring5.error: 0
hw.mlxen8.stat.rx_ring5.bytes: 21479143
hw.mlxen8.stat.rx_ring5.packets: 102093
hw.mlxen8.stat.rx_ring4.error: 0
hw.mlxen8.stat.rx_ring4.bytes: 3823944
hw.mlxen8.stat.rx_ring4.packets: 20467
hw.mlxen8.stat.rx_ring3.error: 0
hw.mlxen8.stat.rx_ring3.bytes: 3467641
hw.mlxen8.stat.rx_ring3.packets: 5758
hw.mlxen8.stat.rx_ring2.error: 0
hw.mlxen8.stat.rx_ring2.bytes: 1424715
hw.mlxen8.stat.rx_ring2.packets: 2156
hw.mlxen8.stat.rx_ring1.error: 0
hw.mlxen8.stat.rx_ring1.bytes: 3370284
hw.mlxen8.stat.rx_ring1.packets: 7623
hw.mlxen8.stat.rx_ring0.error: 0
hw.mlxen8.stat.rx_ring0.bytes: 16174450
hw.mlxen8.stat.rx_ring0.packets: 36816
hw.mlxen8.stat.tx_ring7.defrag_attempts: 0
hw.mlxen8.stat.tx_ring7.tso_packets: 0
hw.mlxen8.stat.tx_ring7.bytes: 6725439
hw.mlxen8.stat.tx_ring7.packets: 15620
hw.mlxen8.stat.tx_ring6.defrag_attempts: 0
hw.mlxen8.stat.tx_ring6.tso_packets: 0
hw.mlxen8.stat.tx_ring6.bytes: 10067911
hw.mlxen8.stat.tx_ring6.packets: 43143
hw.mlxen8.stat.tx_ring5.defrag_attempts: 0
hw.mlxen8.stat.tx_ring5.tso_packets: 0
hw.mlxen8.stat.tx_ring5.bytes: 8483056
hw.mlxen8.stat.tx_ring5.packets: 33149
hw.mlxen8.stat.tx_ring4.defrag_attempts: 0
hw.mlxen8.stat.tx_ring4.tso_packets: 0
hw.mlxen8.stat.tx_ring4.bytes: 25766494
hw.mlxen8.stat.tx_ring4.packets: 112939
hw.mlxen8.stat.tx_ring3.defrag_attempts: 0
hw.mlxen8.stat.tx_ring3.tso_packets: 0
hw.mlxen8.stat.tx_ring3.bytes: 8442587
hw.mlxen8.stat.tx_ring3.packets: 24638
hw.mlxen8.stat.tx_ring2.defrag_attempts: 0
hw.mlxen8.stat.tx_ring2.tso_packets: 0
hw.mlxen8.stat.tx_ring2.bytes: 9702331
hw.mlxen8.stat.tx_ring2.packets: 30603
hw.mlxen8.stat.tx_ring1.defrag_attempts: 0
hw.mlxen8.stat.tx_ring1.tso_packets: 0
hw.mlxen8.stat.tx_ring1.bytes: 7559953
hw.mlxen8.stat.tx_ring1.packets: 25326
hw.mlxen8.stat.tx_ring0.defrag_attempts: 0
hw.mlxen8.stat.tx_ring0.tso_packets: 0
hw.mlxen8.stat.tx_ring0.bytes: 14867762
hw.mlxen8.stat.tx_ring0.packets: 58749
hw.mlxen8.stat.tx_gt_1548_bytes_packets: 0
hw.mlxen8.stat.tx_1548_bytes_packets: 0
hw.mlxen8.stat.tx_1522_bytes_packets: 0
hw.mlxen8.stat.tx_1518_bytes_packets: 0
hw.mlxen8.stat.tx_1023_bytes_packets: 0
hw.mlxen8.stat.tx_511_bytes_packets: 0
hw.mlxen8.stat.tx_255_bytes_packets: 0
hw.mlxen8.stat.tx_127_bytes_packets: 0
hw.mlxen8.stat.tx_lt_64_bytes_packets: 0
hw.mlxen8.stat.tx_dropped: 0
hw.mlxen8.stat.tx_errors: 0
hw.mlxen8.stat.tx_broadcast_packets: 0
hw.mlxen8.stat.tx_multicast_packets: 0
hw.mlxen8.stat.tx_bytes: 0
hw.mlxen8.stat.tx_packets: 0
hw.mlxen8.stat.rx_gt_1548_bytes_packets: 0
hw.mlxen8.stat.rx_1548_bytes_packets: 0
hw.mlxen8.stat.rx_1522_bytes_packets: 0
hw.mlxen8.stat.rx_1518_bytes_packets: 0
hw.mlxen8.stat.rx_1023_bytes_packets: 0
hw.mlxen8.stat.rx_511_bytes_packets: 0
hw.mlxen8.stat.rx_255_bytes_packets: 0
hw.mlxen8.stat.rx_127_bytes_packets: 0
hw.mlxen8.stat.rx_lt_64_bytes_packets: 0
hw.mlxen8.stat.rx_out_range_length_error: 0
hw.mlxen8.stat.rx_in_range_length_error: 0
hw.mlxen8.stat.rx_jabbers: 0
hw.mlxen8.stat.rx_crc_errors: 0
hw.mlxen8.stat.rx_over_errors: 0
hw.mlxen8.stat.rx_length_errors: 0
hw.mlxen8.stat.rx_dropped: 0
hw.mlxen8.stat.rx_errors: 0
hw.mlxen8.stat.rx_broadcast_packets: 0
hw.mlxen8.stat.rx_multicast_packets: 0
hw.mlxen8.stat.rx_packets: 0
hw.mlxen8.stat.rx_bytes: 0
hw.mlxen8.stat.defrag_attempts: 0
hw.mlxen8.stat.tx_chksum_offload: 332062
hw.mlxen8.stat.rx_chksum_none: 0
hw.mlxen8.stat.rx_chksum_good: 0
hw.mlxen8.stat.rx_alloc_failed: 0
hw.mlxen8.stat.tx_oversized_packets: 0
hw.mlxen8.stat.tx_timeout: 0
hw.mlxen8.stat.wake_queue: 0
hw.mlxen8.stat.queue_stopped: 0
hw.mlxen8.stat.tso_packets: 0
hw.mlxen8.conf.eeprom_info: 0
hw.mlxen8.conf.coalesce.adaptive_rx_coal: 1
hw.mlxen8.conf.coalesce.sample_interval: 0
hw.mlxen8.conf.coalesce.rx_usecs_high: 128
hw.mlxen8.conf.coalesce.pkt_rate_high: 450000
hw.mlxen8.conf.coalesce.rx_usecs_low: 0
hw.mlxen8.conf.coalesce.pkt_rate_low: 400000
hw.mlxen8.conf.device_name: mlx4_core4
hw.mlxen8.conf.port_num: 1
hw.mlxen8.conf.rx_ppp: 0
hw.mlxen8.conf.tx_ppp: 0
hw.mlxen8.conf.tx_size: 1024
hw.mlxen8.conf.rx_size: 1024
hw.mlxen8.conf.tx_rings: 8
hw.mlxen8.conf.rx_rings: 8
hw.mlxen8.conf.msg_enable: 20
hw.mlxen6.stat.rx_ring7.error: 0
hw.mlxen6.stat.rx_ring7.bytes: 567263910
hw.mlxen6.stat.rx_ring7.packets: 698806
hw.mlxen6.stat.rx_ring6.error: 0
hw.mlxen6.stat.rx_ring6.bytes: 1429296857
hw.mlxen6.stat.rx_ring6.packets: 1085522
hw.mlxen6.stat.rx_ring5.error: 0
hw.mlxen6.stat.rx_ring5.bytes: 729728731
hw.mlxen6.stat.rx_ring5.packets: 772296
hw.mlxen6.stat.rx_ring4.error: 0
hw.mlxen6.stat.rx_ring4.bytes: 291251699
hw.mlxen6.stat.rx_ring4.packets: 428801
hw.mlxen6.stat.rx_ring3.error: 0
hw.mlxen6.stat.rx_ring3.bytes: 693902876
hw.mlxen6.stat.rx_ring3.packets: 719548
hw.mlxen6.stat.rx_ring2.error: 0
hw.mlxen6.stat.rx_ring2.bytes: 353863511
hw.mlxen6.stat.rx_ring2.packets: 418547
hw.mlxen6.stat.rx_ring1.error: 0
hw.mlxen6.stat.rx_ring1.bytes: 299205582
hw.mlxen6.stat.rx_ring1.packets: 484589
hw.mlxen6.stat.rx_ring0.error: 0
hw.mlxen6.stat.rx_ring0.bytes: 2872596377
hw.mlxen6.stat.rx_ring0.packets: 2397744
hw.mlxen6.stat.tx_ring7.defrag_attempts: 0
hw.mlxen6.stat.tx_ring7.tso_packets: 0
hw.mlxen6.stat.tx_ring7.bytes: 555651427
hw.mlxen6.stat.tx_ring7.packets: 563759
hw.mlxen6.stat.tx_ring6.defrag_attempts: 0
hw.mlxen6.stat.tx_ring6.tso_packets: 0
hw.mlxen6.stat.tx_ring6.bytes: 848442097
hw.mlxen6.stat.tx_ring6.packets: 846301
hw.mlxen6.stat.tx_ring5.defrag_attempts: 0
hw.mlxen6.stat.tx_ring5.tso_packets: 0
hw.mlxen6.stat.tx_ring5.bytes: 2456152659
hw.mlxen6.stat.tx_ring5.packets: 2134935
hw.mlxen6.stat.tx_ring4.defrag_attempts: 0
hw.mlxen6.stat.tx_ring4.tso_packets: 0
hw.mlxen6.stat.tx_ring4.bytes: 809461983
hw.mlxen6.stat.tx_ring4.packets: 743335
hw.mlxen6.stat.tx_ring3.defrag_attempts: 0
hw.mlxen6.stat.tx_ring3.tso_packets: 0
hw.mlxen6.stat.tx_ring3.bytes: 1017054919
hw.mlxen6.stat.tx_ring3.packets: 909462
hw.mlxen6.stat.tx_ring2.defrag_attempts: 0
hw.mlxen6.stat.tx_ring2.tso_packets: 0
hw.mlxen6.stat.tx_ring2.bytes: 1953029998
hw.mlxen6.stat.tx_ring2.packets: 1781654
hw.mlxen6.stat.tx_ring1.defrag_attempts: 0
hw.mlxen6.stat.tx_ring1.tso_packets: 0
hw.mlxen6.stat.tx_ring1.bytes: 371273940
hw.mlxen6.stat.tx_ring1.packets: 475349
hw.mlxen6.stat.tx_ring0.defrag_attempts: 0
hw.mlxen6.stat.tx_ring0.tso_packets: 0
hw.mlxen6.stat.tx_ring0.bytes: 631372575
hw.mlxen6.stat.tx_ring0.packets: 643907
hw.mlxen6.stat.tx_gt_1548_bytes_packets: 0
hw.mlxen6.stat.tx_1548_bytes_packets: 0
hw.mlxen6.stat.tx_1522_bytes_packets: 0
hw.mlxen6.stat.tx_1518_bytes_packets: 0
hw.mlxen6.stat.tx_1023_bytes_packets: 0
hw.mlxen6.stat.tx_511_bytes_packets: 0
hw.mlxen6.stat.tx_255_bytes_packets: 0
hw.mlxen6.stat.tx_127_bytes_packets: 0
hw.mlxen6.stat.tx_lt_64_bytes_packets: 0
hw.mlxen6.stat.tx_dropped: 0
hw.mlxen6.stat.tx_errors: 0
hw.mlxen6.stat.tx_broadcast_packets: 0
hw.mlxen6.stat.tx_multicast_packets: 0
hw.mlxen6.stat.tx_bytes: 0
hw.mlxen6.stat.tx_packets: 0
hw.mlxen6.stat.rx_gt_1548_bytes_packets: 0
hw.mlxen6.stat.rx_1548_bytes_packets: 0
hw.mlxen6.stat.rx_1522_bytes_packets: 0
hw.mlxen6.stat.rx_1518_bytes_packets: 0
hw.mlxen6.stat.rx_1023_bytes_packets: 0
hw.mlxen6.stat.rx_511_bytes_packets: 0
hw.mlxen6.stat.rx_255_bytes_packets: 0
hw.mlxen6.stat.rx_127_bytes_packets: 0
hw.mlxen6.stat.rx_lt_64_bytes_packets: 0
hw.mlxen6.stat.rx_out_range_length_error: 0
hw.mlxen6.stat.rx_in_range_length_error: 0
hw.mlxen6.stat.rx_jabbers: 0
hw.mlxen6.stat.rx_crc_errors: 0
hw.mlxen6.stat.rx_over_errors: 0
hw.mlxen6.stat.rx_length_errors: 0
hw.mlxen6.stat.rx_dropped: 0
hw.mlxen6.stat.rx_errors: 0
hw.mlxen6.stat.rx_broadcast_packets: 0
hw.mlxen6.stat.rx_multicast_packets: 0
hw.mlxen6.stat.rx_packets: 0
hw.mlxen6.stat.rx_bytes: 0
hw.mlxen6.stat.defrag_attempts: 0
hw.mlxen6.stat.tx_chksum_offload: 5970963
hw.mlxen6.stat.rx_chksum_none: 0
hw.mlxen6.stat.rx_chksum_good: 98
hw.mlxen6.stat.rx_alloc_failed: 0
hw.mlxen6.stat.tx_oversized_packets: 0
hw.mlxen6.stat.tx_timeout: 0
hw.mlxen6.stat.wake_queue: 0
hw.mlxen6.stat.queue_stopped: 0
hw.mlxen6.stat.tso_packets: 0
hw.mlxen6.conf.eeprom_info: 0
hw.mlxen6.conf.coalesce.adaptive_rx_coal: 1
hw.mlxen6.conf.coalesce.sample_interval: 0
hw.mlxen6.conf.coalesce.rx_usecs_high: 128
hw.mlxen6.conf.coalesce.pkt_rate_high: 450000
hw.mlxen6.conf.coalesce.rx_usecs_low: 0
hw.mlxen6.conf.coalesce.pkt_rate_low: 400000
hw.mlxen6.conf.device_name: mlx4_core3
hw.mlxen6.conf.port_num: 1
hw.mlxen6.conf.rx_ppp: 0
hw.mlxen6.conf.tx_ppp: 0
hw.mlxen6.conf.tx_size: 1024
hw.mlxen6.conf.rx_size: 1024
hw.mlxen6.conf.tx_rings: 8
hw.mlxen6.conf.rx_rings: 8
hw.mlxen6.conf.msg_enable: 20
hw.mlxen4.stat.rx_ring7.error: 0
hw.mlxen4.stat.rx_ring7.bytes: 939697
hw.mlxen4.stat.rx_ring7.packets: 9533
hw.mlxen4.stat.rx_ring6.error: 0
hw.mlxen4.stat.rx_ring6.bytes: 1475523
hw.mlxen4.stat.rx_ring6.packets: 9598
hw.mlxen4.stat.rx_ring5.error: 0
hw.mlxen4.stat.rx_ring5.bytes: 2082252
hw.mlxen4.stat.rx_ring5.packets: 5452
hw.mlxen4.stat.rx_ring4.error: 0
hw.mlxen4.stat.rx_ring4.bytes: 4019873
hw.mlxen4.stat.rx_ring4.packets: 17136
hw.mlxen4.stat.rx_ring3.error: 0
hw.mlxen4.stat.rx_ring3.bytes: 563617
hw.mlxen4.stat.rx_ring3.packets: 2569
hw.mlxen4.stat.rx_ring2.error: 0
hw.mlxen4.stat.rx_ring2.bytes: 7407265
hw.mlxen4.stat.rx_ring2.packets: 15978
hw.mlxen4.stat.rx_ring1.error: 0
hw.mlxen4.stat.rx_ring1.bytes: 123856518
hw.mlxen4.stat.rx_ring1.packets: 363347
hw.mlxen4.stat.rx_ring0.error: 0
hw.mlxen4.stat.rx_ring0.bytes: 47394211
hw.mlxen4.stat.rx_ring0.packets: 148667
hw.mlxen4.stat.tx_ring7.defrag_attempts: 0
hw.mlxen4.stat.tx_ring7.tso_packets: 0
hw.mlxen4.stat.tx_ring7.bytes: 20484321
hw.mlxen4.stat.tx_ring7.packets: 24245
hw.mlxen4.stat.tx_ring6.defrag_attempts: 0
hw.mlxen4.stat.tx_ring6.tso_packets: 0
hw.mlxen4.stat.tx_ring6.bytes: 139657761
hw.mlxen4.stat.tx_ring6.packets: 117583
hw.mlxen4.stat.tx_ring5.defrag_attempts: 0
hw.mlxen4.stat.tx_ring5.tso_packets: 0
hw.mlxen4.stat.tx_ring5.bytes: 48372613
hw.mlxen4.stat.tx_ring5.packets: 40279
hw.mlxen4.stat.tx_ring4.defrag_attempts: 0
hw.mlxen4.stat.tx_ring4.tso_packets: 0
hw.mlxen4.stat.tx_ring4.bytes: 2298528
hw.mlxen4.stat.tx_ring4.packets: 6674
hw.mlxen4.stat.tx_ring3.defrag_attempts: 0
hw.mlxen4.stat.tx_ring3.tso_packets: 0
hw.mlxen4.stat.tx_ring3.bytes: 73017516
hw.mlxen4.stat.tx_ring3.packets: 56715
hw.mlxen4.stat.tx_ring2.defrag_attempts: 0
hw.mlxen4.stat.tx_ring2.tso_packets: 0
hw.mlxen4.stat.tx_ring2.bytes: 352368419
hw.mlxen4.stat.tx_ring2.packets: 605761
hw.mlxen4.stat.tx_ring1.defrag_attempts: 0
hw.mlxen4.stat.tx_ring1.tso_packets: 0
hw.mlxen4.stat.tx_ring1.bytes: 71605813
hw.mlxen4.stat.tx_ring1.packets: 50389
hw.mlxen4.stat.tx_ring0.defrag_attempts: 0
hw.mlxen4.stat.tx_ring0.tso_packets: 0
hw.mlxen4.stat.tx_ring0.bytes: 3969909
hw.mlxen4.stat.tx_ring0.packets: 7420
hw.mlxen4.stat.tx_gt_1548_bytes_packets: 0
hw.mlxen4.stat.tx_1548_bytes_packets: 0
hw.mlxen4.stat.tx_1522_bytes_packets: 0
hw.mlxen4.stat.tx_1518_bytes_packets: 0
hw.mlxen4.stat.tx_1023_bytes_packets: 0
hw.mlxen4.stat.tx_511_bytes_packets: 0
hw.mlxen4.stat.tx_255_bytes_packets: 0
hw.mlxen4.stat.tx_127_bytes_packets: 0
hw.mlxen4.stat.tx_lt_64_bytes_packets: 0
hw.mlxen4.stat.tx_dropped: 0
hw.mlxen4.stat.tx_errors: 0
hw.mlxen4.stat.tx_broadcast_packets: 0
hw.mlxen4.stat.tx_multicast_packets: 0
hw.mlxen4.stat.tx_bytes: 0
hw.mlxen4.stat.tx_packets: 0
hw.mlxen4.stat.rx_gt_1548_bytes_packets: 0
hw.mlxen4.stat.rx_1548_bytes_packets: 0
hw.mlxen4.stat.rx_1522_bytes_packets: 0
hw.mlxen4.stat.rx_1518_bytes_packets: 0
hw.mlxen4.stat.rx_1023_bytes_packets: 0
hw.mlxen4.stat.rx_511_bytes_packets: 0
hw.mlxen4.stat.rx_255_bytes_packets: 0
hw.mlxen4.stat.rx_127_bytes_packets: 0
hw.mlxen4.stat.rx_lt_64_bytes_packets: 0
hw.mlxen4.stat.rx_out_range_length_error: 0
hw.mlxen4.stat.rx_in_range_length_error: 0
hw.mlxen4.stat.rx_jabbers: 0
hw.mlxen4.stat.rx_crc_errors: 0
hw.mlxen4.stat.rx_over_errors: 0
hw.mlxen4.stat.rx_length_errors: 0
hw.mlxen4.stat.rx_dropped: 0
hw.mlxen4.stat.rx_errors: 0
hw.mlxen4.stat.rx_broadcast_packets: 0
hw.mlxen4.stat.rx_multicast_packets: 0
hw.mlxen4.stat.rx_packets: 0
hw.mlxen4.stat.rx_bytes: 0
hw.mlxen4.stat.defrag_attempts: 0
hw.mlxen4.stat.tx_chksum_offload: 896566
hw.mlxen4.stat.rx_chksum_none: 0
hw.mlxen4.stat.rx_chksum_good: 0
hw.mlxen4.stat.rx_alloc_failed: 0
hw.mlxen4.stat.tx_oversized_packets: 0
hw.mlxen4.stat.tx_timeout: 0
hw.mlxen4.stat.wake_queue: 0
hw.mlxen4.stat.queue_stopped: 0
hw.mlxen4.stat.tso_packets: 0
hw.mlxen4.conf.eeprom_info: 0
hw.mlxen4.conf.coalesce.adaptive_rx_coal: 1
hw.mlxen4.conf.coalesce.sample_interval: 0
hw.mlxen4.conf.coalesce.rx_usecs_high: 128
hw.mlxen4.conf.coalesce.pkt_rate_high: 450000
hw.mlxen4.conf.coalesce.rx_usecs_low: 0
hw.mlxen4.conf.coalesce.pkt_rate_low: 400000
hw.mlxen4.conf.device_name: mlx4_core2
hw.mlxen4.conf.port_num: 1
hw.mlxen4.conf.rx_ppp: 0
hw.mlxen4.conf.tx_ppp: 0
hw.mlxen4.conf.tx_size: 1024
hw.mlxen4.conf.rx_size: 1024
hw.mlxen4.conf.tx_rings: 8
hw.mlxen4.conf.rx_rings: 8
hw.mlxen4.conf.msg_enable: 20
hw.mlxen2.stat.rx_ring7.error: 0
hw.mlxen2.stat.rx_ring7.bytes: 3125197569
hw.mlxen2.stat.rx_ring7.packets: 2622273
hw.mlxen2.stat.rx_ring6.error: 0
hw.mlxen2.stat.rx_ring6.bytes: 3749927783
hw.mlxen2.stat.rx_ring6.packets: 3282633
hw.mlxen2.stat.rx_ring5.error: 0
hw.mlxen2.stat.rx_ring5.bytes: 2846390840
hw.mlxen2.stat.rx_ring5.packets: 2839413
hw.mlxen2.stat.rx_ring4.error: 0
hw.mlxen2.stat.rx_ring4.bytes: 2965527263
hw.mlxen2.stat.rx_ring4.packets: 2525415
hw.mlxen2.stat.rx_ring3.error: 0
hw.mlxen2.stat.rx_ring3.bytes: 2963152953
hw.mlxen2.stat.rx_ring3.packets: 2455037
hw.mlxen2.stat.rx_ring2.error: 0
hw.mlxen2.stat.rx_ring2.bytes: 2932629893
hw.mlxen2.stat.rx_ring2.packets: 2605706
hw.mlxen2.stat.rx_ring1.error: 0
hw.mlxen2.stat.rx_ring1.bytes: 2862920951
hw.mlxen2.stat.rx_ring1.packets: 2528860
hw.mlxen2.stat.rx_ring0.error: 0
hw.mlxen2.stat.rx_ring0.bytes: 3499686611
hw.mlxen2.stat.rx_ring0.packets: 2995553
hw.mlxen2.stat.tx_ring7.defrag_attempts: 0
hw.mlxen2.stat.tx_ring7.tso_packets: 0
hw.mlxen2.stat.tx_ring7.bytes: 5162491355
hw.mlxen2.stat.tx_ring7.packets: 4717595
hw.mlxen2.stat.tx_ring6.defrag_attempts: 0
hw.mlxen2.stat.tx_ring6.tso_packets: 0
hw.mlxen2.stat.tx_ring6.bytes: 9969081347
hw.mlxen2.stat.tx_ring6.packets: 8243280
hw.mlxen2.stat.tx_ring5.defrag_attempts: 0
hw.mlxen2.stat.tx_ring5.tso_packets: 0
hw.mlxen2.stat.tx_ring5.bytes: 3686387772
hw.mlxen2.stat.tx_ring5.packets: 3867976
hw.mlxen2.stat.tx_ring4.defrag_attempts: 0
hw.mlxen2.stat.tx_ring4.tso_packets: 0
hw.mlxen2.stat.tx_ring4.bytes: 3116345662
hw.mlxen2.stat.tx_ring4.packets: 3527318
hw.mlxen2.stat.tx_ring3.defrag_attempts: 0
hw.mlxen2.stat.tx_ring3.tso_packets: 0
hw.mlxen2.stat.tx_ring3.bytes: 4260780403
hw.mlxen2.stat.tx_ring3.packets: 4149369
hw.mlxen2.stat.tx_ring2.defrag_attempts: 0
hw.mlxen2.stat.tx_ring2.tso_packets: 0
hw.mlxen2.stat.tx_ring2.bytes: 7606524206
hw.mlxen2.stat.tx_ring2.packets: 6835039
hw.mlxen2.stat.tx_ring1.defrag_attempts: 0
hw.mlxen2.stat.tx_ring1.tso_packets: 0
hw.mlxen2.stat.tx_ring1.bytes: 2865170818
hw.mlxen2.stat.tx_ring1.packets: 3106333
hw.mlxen2.stat.tx_ring0.defrag_attempts: 0
hw.mlxen2.stat.tx_ring0.tso_packets: 0
hw.mlxen2.stat.tx_ring0.bytes: 6651104972
hw.mlxen2.stat.tx_ring0.packets: 6117473
hw.mlxen2.stat.tx_gt_1548_bytes_packets: 0
hw.mlxen2.stat.tx_1548_bytes_packets: 0
hw.mlxen2.stat.tx_1522_bytes_packets: 0
hw.mlxen2.stat.tx_1518_bytes_packets: 0
hw.mlxen2.stat.tx_1023_bytes_packets: 0
hw.mlxen2.stat.tx_511_bytes_packets: 0
hw.mlxen2.stat.tx_255_bytes_packets: 0
hw.mlxen2.stat.tx_127_bytes_packets: 0
hw.mlxen2.stat.tx_lt_64_bytes_packets: 0
hw.mlxen2.stat.tx_dropped: 0
hw.mlxen2.stat.tx_errors: 0
hw.mlxen2.stat.tx_broadcast_packets: 0
hw.mlxen2.stat.tx_multicast_packets: 0
hw.mlxen2.stat.tx_bytes: 0
hw.mlxen2.stat.tx_packets: 0
hw.mlxen2.stat.rx_gt_1548_bytes_packets: 0
hw.mlxen2.stat.rx_1548_bytes_packets: 0
hw.mlxen2.stat.rx_1522_bytes_packets: 0
hw.mlxen2.stat.rx_1518_bytes_packets: 0
hw.mlxen2.stat.rx_1023_bytes_packets: 0
hw.mlxen2.stat.rx_511_bytes_packets: 0
hw.mlxen2.stat.rx_255_bytes_packets: 0
hw.mlxen2.stat.rx_127_bytes_packets: 0
hw.mlxen2.stat.rx_lt_64_bytes_packets: 0
hw.mlxen2.stat.rx_out_range_length_error: 0
hw.mlxen2.stat.rx_in_range_length_error: 0
hw.mlxen2.stat.rx_jabbers: 0
hw.mlxen2.stat.rx_crc_errors: 0
hw.mlxen2.stat.rx_over_errors: 0
hw.mlxen2.stat.rx_length_errors: 0
hw.mlxen2.stat.rx_dropped: 0
hw.mlxen2.stat.rx_errors: 0
hw.mlxen2.stat.rx_broadcast_packets: 0
hw.mlxen2.stat.rx_multicast_packets: 0
hw.mlxen2.stat.rx_packets: 0
hw.mlxen2.stat.rx_bytes: 0
hw.mlxen2.stat.defrag_attempts: 0
hw.mlxen2.stat.tx_chksum_offload: 25119559
hw.mlxen2.stat.rx_chksum_none: 0
hw.mlxen2.stat.rx_chksum_good: 657
hw.mlxen2.stat.rx_alloc_failed: 0
hw.mlxen2.stat.tx_oversized_packets: 0
hw.mlxen2.stat.tx_timeout: 0
hw.mlxen2.stat.wake_queue: 0
hw.mlxen2.stat.queue_stopped: 0
hw.mlxen2.stat.tso_packets: 0
hw.mlxen2.conf.eeprom_info: 0
hw.mlxen2.conf.coalesce.adaptive_rx_coal: 1
hw.mlxen2.conf.coalesce.sample_interval: 0
hw.mlxen2.conf.coalesce.rx_usecs_high: 128
hw.mlxen2.conf.coalesce.pkt_rate_high: 450000
hw.mlxen2.conf.coalesce.rx_usecs_low: 0
hw.mlxen2.conf.coalesce.pkt_rate_low: 400000
hw.mlxen2.conf.device_name: mlx4_core1
hw.mlxen2.conf.port_num: 1
hw.mlxen2.conf.rx_ppp: 0
hw.mlxen2.conf.tx_ppp: 0
hw.mlxen2.conf.tx_size: 1024
hw.mlxen2.conf.rx_size: 1024
hw.mlxen2.conf.tx_rings: 8
hw.mlxen2.conf.rx_rings: 8
hw.mlxen2.conf.msg_enable: 20
                  X 1 Reply Last reply Reply Quote 0
                  • X
                    xpxp2002 @xpxp2002
                    last edited by 

                    hw.mlxen0.stat.rx_ring7.error: 0
hw.mlxen0.stat.rx_ring7.bytes: 5223939670
hw.mlxen0.stat.rx_ring7.packets: 4787387
hw.mlxen0.stat.rx_ring6.error: 0
hw.mlxen0.stat.rx_ring6.bytes: 9623639327
hw.mlxen0.stat.rx_ring6.packets: 8129273
hw.mlxen0.stat.rx_ring5.error: 0
hw.mlxen0.stat.rx_ring5.bytes: 5550493687
hw.mlxen0.stat.rx_ring5.packets: 5155958
hw.mlxen0.stat.rx_ring4.error: 0
hw.mlxen0.stat.rx_ring4.bytes: 3708993683
hw.mlxen0.stat.rx_ring4.packets: 3962707
hw.mlxen0.stat.rx_ring3.error: 0
hw.mlxen0.stat.rx_ring3.bytes: 4705789645
hw.mlxen0.stat.rx_ring3.packets: 4377860
hw.mlxen0.stat.rx_ring2.error: 0
hw.mlxen0.stat.rx_ring2.bytes: 9553913082
hw.mlxen0.stat.rx_ring2.packets: 8342700
hw.mlxen0.stat.rx_ring1.error: 0
hw.mlxen0.stat.rx_ring1.bytes: 3215413264
hw.mlxen0.stat.rx_ring1.packets: 3557237
hw.mlxen0.stat.rx_ring0.error: 0
hw.mlxen0.stat.rx_ring0.bytes: 4663009944
hw.mlxen0.stat.rx_ring0.packets: 4677401
hw.mlxen0.stat.tx_ring7.defrag_attempts: 0
hw.mlxen0.stat.tx_ring7.tso_packets: 0
hw.mlxen0.stat.tx_ring7.bytes: 3173747221
hw.mlxen0.stat.tx_ring7.packets: 2924833
hw.mlxen0.stat.tx_ring6.defrag_attempts: 0
hw.mlxen0.stat.tx_ring6.tso_packets: 0
hw.mlxen0.stat.tx_ring6.bytes: 3844506086
hw.mlxen0.stat.tx_ring6.packets: 3443632
hw.mlxen0.stat.tx_ring5.defrag_attempts: 0
hw.mlxen0.stat.tx_ring5.tso_packets: 0
hw.mlxen0.stat.tx_ring5.bytes: 2884750087
hw.mlxen0.stat.tx_ring5.packets: 2859605
hw.mlxen0.stat.tx_ring4.defrag_attempts: 0
hw.mlxen0.stat.tx_ring4.tso_packets: 0
hw.mlxen0.stat.tx_ring4.bytes: 3034582670
hw.mlxen0.stat.tx_ring4.packets: 2769997
hw.mlxen0.stat.tx_ring3.defrag_attempts: 0
hw.mlxen0.stat.tx_ring3.tso_packets: 0
hw.mlxen0.stat.tx_ring3.bytes: 3066817040
hw.mlxen0.stat.tx_ring3.packets: 2749789
hw.mlxen0.stat.tx_ring2.defrag_attempts: 0
hw.mlxen0.stat.tx_ring2.tso_packets: 0
hw.mlxen0.stat.tx_ring2.bytes: 2974801824
hw.mlxen0.stat.tx_ring2.packets: 2813491
hw.mlxen0.stat.tx_ring1.defrag_attempts: 0
hw.mlxen0.stat.tx_ring1.tso_packets: 0
hw.mlxen0.stat.tx_ring1.bytes: 3058973604
hw.mlxen0.stat.tx_ring1.packets: 3088725
hw.mlxen0.stat.tx_ring0.defrag_attempts: 0
hw.mlxen0.stat.tx_ring0.tso_packets: 0
hw.mlxen0.stat.tx_ring0.bytes: 3729335976
hw.mlxen0.stat.tx_ring0.packets: 3787441
hw.mlxen0.stat.tx_gt_1548_bytes_packets: 0
hw.mlxen0.stat.tx_1548_bytes_packets: 0
hw.mlxen0.stat.tx_1522_bytes_packets: 0
hw.mlxen0.stat.tx_1518_bytes_packets: 0
hw.mlxen0.stat.tx_1023_bytes_packets: 0
hw.mlxen0.stat.tx_511_bytes_packets: 0
hw.mlxen0.stat.tx_255_bytes_packets: 0
hw.mlxen0.stat.tx_127_bytes_packets: 0
hw.mlxen0.stat.tx_lt_64_bytes_packets: 0
hw.mlxen0.stat.tx_dropped: 0
hw.mlxen0.stat.tx_errors: 0
hw.mlxen0.stat.tx_broadcast_packets: 0
hw.mlxen0.stat.tx_multicast_packets: 0
hw.mlxen0.stat.tx_bytes: 0
hw.mlxen0.stat.tx_packets: 0
hw.mlxen0.stat.rx_gt_1548_bytes_packets: 0
hw.mlxen0.stat.rx_1548_bytes_packets: 0
hw.mlxen0.stat.rx_1522_bytes_packets: 0
hw.mlxen0.stat.rx_1518_bytes_packets: 0
hw.mlxen0.stat.rx_1023_bytes_packets: 0
hw.mlxen0.stat.rx_511_bytes_packets: 0
hw.mlxen0.stat.rx_255_bytes_packets: 0
hw.mlxen0.stat.rx_127_bytes_packets: 0
hw.mlxen0.stat.rx_lt_64_bytes_packets: 0
hw.mlxen0.stat.rx_out_range_length_error: 0
hw.mlxen0.stat.rx_in_range_length_error: 0
hw.mlxen0.stat.rx_jabbers: 0
hw.mlxen0.stat.rx_crc_errors: 0
hw.mlxen0.stat.rx_over_errors: 0
hw.mlxen0.stat.rx_length_errors: 0
hw.mlxen0.stat.rx_dropped: 0
hw.mlxen0.stat.rx_errors: 0
hw.mlxen0.stat.rx_broadcast_packets: 0
hw.mlxen0.stat.rx_multicast_packets: 0
hw.mlxen0.stat.rx_packets: 0
hw.mlxen0.stat.rx_bytes: 0
hw.mlxen0.stat.defrag_attempts: 0
hw.mlxen0.stat.tx_chksum_offload: 19783126
hw.mlxen0.stat.rx_chksum_none: 0
hw.mlxen0.stat.rx_chksum_good: 312
hw.mlxen0.stat.rx_alloc_failed: 0
hw.mlxen0.stat.tx_oversized_packets: 0
hw.mlxen0.stat.tx_timeout: 0
hw.mlxen0.stat.wake_queue: 0
hw.mlxen0.stat.queue_stopped: 0
hw.mlxen0.stat.tso_packets: 0
hw.mlxen0.conf.eeprom_info: 0
hw.mlxen0.conf.coalesce.adaptive_rx_coal: 1
hw.mlxen0.conf.coalesce.sample_interval: 0
hw.mlxen0.conf.coalesce.rx_usecs_high: 128
hw.mlxen0.conf.coalesce.pkt_rate_high: 450000
hw.mlxen0.conf.coalesce.rx_usecs_low: 0
hw.mlxen0.conf.coalesce.pkt_rate_low: 400000
hw.mlxen0.conf.device_name: mlx4_core0
hw.mlxen0.conf.port_num: 1
hw.mlxen0.conf.rx_ppp: 0
hw.mlxen0.conf.tx_ppp: 0
hw.mlxen0.conf.tx_size: 1024
hw.mlxen0.conf.rx_size: 1024
hw.mlxen0.conf.tx_rings: 8
hw.mlxen0.conf.rx_rings: 8
hw.mlxen0.conf.msg_enable: 20
irq256: mlx4_core0:63 @cpu0(domain0): 215793
irq257: mlx4_core0:65 @cpu0(domain0): 2409401
irq258: mlx4_core0:67 @cpu1(domain0): 1924013
irq259: mlx4_core0:69 @cpu2(domain0): 3578839
irq260: mlx4_core0:71 @cpu3(domain0): 1894844
irq261: mlx4_core0:73 @cpu4(domain0): 1914869
irq262: mlx4_core0:75 @cpu5(domain0): 2198026
irq263: mlx4_core0:77 @cpu6(domain0): 3091417
irq264: mlx4_core0:79 @cpu7(domain0): 2071627
irq265: mlx4_core1:81 @cpu0(domain0): 215791
irq266: mlx4_core1:83 @cpu0(domain0): 875589
irq267: mlx4_core1:85 @cpu1(domain0): 717777
irq268: mlx4_core1:87 @cpu2(domain0): 746723
irq269: mlx4_core1:89 @cpu3(domain0): 643236
irq270: mlx4_core1:91 @cpu4(domain0): 669631
irq271: mlx4_core1:93 @cpu5(domain0): 1039522
irq272: mlx4_core1:95 @cpu6(domain0): 969079
irq273: mlx4_core1:97 @cpu7(domain0): 739405
irq274: mlx4_core2:99 @cpu0(domain0): 215791
irq275: mlx4_core2:101 @cpu0(domain0): 98904
irq276: mlx4_core2:103 @cpu1(domain0): 288935
irq277: mlx4_core2:105 @cpu2(domain0): 9409
irq278: mlx4_core2:107 @cpu3(domain0): 2262
irq279: mlx4_core2:109 @cpu4(domain0): 11898
irq280: mlx4_core2:111 @cpu5(domain0): 3885
irq281: mlx4_core2:113 @cpu6(domain0): 7049
irq282: mlx4_core2:115 @cpu7(domain0): 6529
irq283: mlx4_core3:117 @cpu0(domain0): 215791
irq284: mlx4_core3:119 @cpu0(domain0): 1878403
irq285: mlx4_core3:121 @cpu1(domain0): 339462
irq286: mlx4_core3:123 @cpu2(domain0): 287315
irq287: mlx4_core3:125 @cpu3(domain0): 521382
irq288: mlx4_core3:127 @cpu4(domain0): 307190
irq289: mlx4_core3:129 @cpu5(domain0): 566305
irq290: mlx4_core3:131 @cpu6(domain0): 774495
irq291: mlx4_core3:133 @cpu7(domain0): 478187
irq292: mlx4_core4:135 @cpu0(domain0): 215988
irq293: mlx4_core4:137 @cpu0(domain0): 36818
irq294: mlx4_core4:139 @cpu1(domain0): 7623
irq295: mlx4_core4:141 @cpu2(domain0): 2158
irq296: mlx4_core4:143 @cpu3(domain0): 5723
irq297: mlx4_core4:145 @cpu4(domain0): 20469
irq298: mlx4_core4:147 @cpu5(domain0): 102067
irq299: mlx4_core4:149 @cpu6(domain0): 163
irq300: mlx4_core4:151 @cpu7(domain0): 5903
irq301: mlx4_core5:153 @cpu0(domain0): 215794
irq302: mlx4_core5:155 @cpu0(domain0): 1
irq303: mlx4_core5:157 @cpu1(domain0): 1
irq304: mlx4_core5:159 @cpu2(domain0): 1
irq305: mlx4_core5:161 @cpu3(domain0): 1
irq306: mlx4_core5:163 @cpu4(domain0): 1
irq307: mlx4_core5:165 @cpu5(domain0): 1
irq308: mlx4_core5:167 @cpu6(domain0): 1
irq309: mlx4_core5:169 @cpu7(domain0): 1
irq310: mlx4_core6:171 @cpu0(domain0): 215994
irq311: mlx4_core6:173 @cpu0(domain0): 0
irq312: mlx4_core6:175 @cpu1(domain0): 1
irq313: mlx4_core6:177 @cpu2(domain0): 1
irq314: mlx4_core6:179 @cpu3(domain0): 1
irq315: mlx4_core6:181 @cpu4(domain0): 1
irq316: mlx4_core6:183 @cpu5(domain0): 1
irq317: mlx4_core6:185 @cpu6(domain0): 1
irq318: mlx4_core6:187 @cpu7(domain0): 0
hw.hn.vfmap: mlxen0:hn0 mlxen2:hn1 mlxen4:hn2 mlxen6:hn3 mlxen8:hn4 mlxen10:hn5 mlxen12:hn6
hw.hn.vflist: mlxen0 mlxen2 mlxen4 mlxen6 mlxen8 mlxen10 mlxen12
hw.mlx5.calibr.normal: 30
hw.mlx5.calibr.fast: 1
hw.mlx5.calibr.duration: 20
hw.mlx5.relaxed_ordering_write: 0
hw.mlx5.auto_fw_update: 0
hw.mlx5.fast_unload_enabled: 1
hw.mlx5.prof_sel: 2
hw.mlx5.debug_mask: 0
hw.mlx5.sw_reset_timeout: 1200
hw.mlx5.fw_reset_enable: 1
sys.device.mlx4_core6.hw.board_id: MT_1200111023
sys.device.mlx4_core6.hw.fw_version: 2.42.5000
sys.device.mlx4_core6.mlx4_port1_mtu: -1
sys.device.mlx4_core6.mlx4_port1: eth
sys.device.mlx4_core5.hw.board_id: MT_1200111023
sys.device.mlx4_core5.hw.fw_version: 2.42.5000
sys.device.mlx4_core5.mlx4_port1_mtu: -1
sys.device.mlx4_core5.mlx4_port1: eth
sys.device.mlx4_core4.hw.board_id: MT_1200111023
sys.device.mlx4_core4.hw.fw_version: 2.42.5000
sys.device.mlx4_core4.mlx4_port1_mtu: -1
sys.device.mlx4_core4.mlx4_port1: eth
sys.device.mlx4_core3.hw.board_id: MT_1200111023
sys.device.mlx4_core3.hw.fw_version: 2.42.5000
sys.device.mlx4_core3.mlx4_port1_mtu: -1
sys.device.mlx4_core3.mlx4_port1: eth
sys.device.mlx4_core2.hw.board_id: MT_1200111023
sys.device.mlx4_core2.hw.fw_version: 2.42.5000
sys.device.mlx4_core2.mlx4_port1_mtu: -1
sys.device.mlx4_core2.mlx4_port1: eth
sys.device.mlx4_core1.hw.board_id: MT_1200111023
sys.device.mlx4_core1.hw.fw_version: 2.42.5000
sys.device.mlx4_core1.mlx4_port1_mtu: -1
sys.device.mlx4_core1.mlx4_port1: eth
sys.device.mlx4_core0.hw.board_id: MT_1200111023
sys.device.mlx4_core0.hw.fw_version: 2.42.5000
sys.device.mlx4_core0.mlx4_port1_mtu: -1
sys.device.mlx4_core0.mlx4_port1: eth
dev.mlx4_core.6.%parent: pci6
dev.mlx4_core.6.%pnpinfo: vendor=0x15b3 device=0x1004 subvendor=0x15b3 subdevice=0x61b0 class=0x020000
dev.mlx4_core.6.%location: slot=2 function=0 dbsf=pci7:0:2:0
dev.mlx4_core.6.%driver: mlx4_core
dev.mlx4_core.6.%desc: Mellanox driver (3.6.0)
dev.mlx4_core.5.%parent: pci5
dev.mlx4_core.5.%pnpinfo: vendor=0x15b3 device=0x1004 subvendor=0x15b3 subdevice=0x61b0 class=0x020000
dev.mlx4_core.5.%location: slot=2 function=0 dbsf=pci6:0:2:0
dev.mlx4_core.5.%driver: mlx4_core
dev.mlx4_core.5.%desc: Mellanox driver (3.6.0)
dev.mlx4_core.4.%parent: pci4
dev.mlx4_core.4.%pnpinfo: vendor=0x15b3 device=0x1004 subvendor=0x15b3 subdevice=0x61b0 class=0x020000
dev.mlx4_core.4.%location: slot=2 function=0 dbsf=pci5:0:2:0
dev.mlx4_core.4.%driver: mlx4_core
dev.mlx4_core.4.%desc: Mellanox driver (3.6.0)
dev.mlx4_core.3.%parent: pci3
dev.mlx4_core.3.%pnpinfo: vendor=0x15b3 device=0x1004 subvendor=0x15b3 subdevice=0x61b0 class=0x020000
dev.mlx4_core.3.%location: slot=2 function=0 dbsf=pci4:0:2:0
dev.mlx4_core.3.%driver: mlx4_core
dev.mlx4_core.3.%desc: Mellanox driver (3.6.0)
dev.mlx4_core.2.%parent: pci2
dev.mlx4_core.2.%pnpinfo: vendor=0x15b3 device=0x1004 subvendor=0x15b3 subdevice=0x61b0 class=0x020000
dev.mlx4_core.2.%location: slot=2 function=0 dbsf=pci3:0:2:0
dev.mlx4_core.2.%driver: mlx4_core
dev.mlx4_core.2.%desc: Mellanox driver (3.6.0)
dev.mlx4_core.1.%parent: pci1
dev.mlx4_core.1.%pnpinfo: vendor=0x15b3 device=0x1004 subvendor=0x15b3 subdevice=0x61b0 class=0x020000
dev.mlx4_core.1.%location: slot=2 function=0 dbsf=pci2:0:2:0
dev.mlx4_core.1.%driver: mlx4_core
dev.mlx4_core.1.%desc: Mellanox driver (3.6.0)
dev.mlx4_core.0.%parent: pci0
dev.mlx4_core.0.%pnpinfo: vendor=0x15b3 device=0x1004 subvendor=0x15b3 subdevice=0x61b0 class=0x020000
dev.mlx4_core.0.%location: slot=2 function=0 dbsf=pci1:0:2:0
dev.mlx4_core.0.%driver: mlx4_core
dev.mlx4_core.0.%desc: Mellanox driver (3.6.0)
dev.mlx4_core.%parent: 
dev.hn.6.vf: mlxen12
dev.hn.5.vf: mlxen10
dev.hn.4.vf: mlxen8
dev.hn.3.vf: mlxen6
dev.hn.2.vf: mlxen4
dev.hn.1.vf: mlxen2
dev.hn.0.vf: mlxen0
compat.linuxkpi.mlx4_inline_thold: 104
compat.linuxkpi.mlx4_pfcrx: 0
compat.linuxkpi.mlx4_pfctx: 0
compat.linuxkpi.mlx4_udp_rss: 1
compat.linuxkpi.mlx4_log_mtts_per_seg: 3
compat.linuxkpi.mlx4_use_prio: 0
compat.linuxkpi.mlx4_log_num_vlan: 0
compat.linuxkpi.mlx4_log_num_mac: 7
compat.linuxkpi.mlx4_enable_4k_uar: 0
compat.linuxkpi.mlx4_enable_64b_cqe_eqe: 1
compat.linuxkpi.mlx4_log_num_mgm_entry_size: 7
compat.linuxkpi.mlx4_msi_x: 1
compat.linuxkpi.mlx4_enable_qos: 0
compat.linuxkpi.mlx4_internal_err_reset: 1
                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Hmm, interesting. Looks like 'priority flow control'. Let me see if I can test something....

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Nope, I've got nothing!

                        I have an older Mellanox card I dusted off but it's always been oddly behaved in pfSense/FreeBSD. The fact it's a converged device that presents as Ethernet seems to hide a lot of values you might otherwise see and be able to set.

                        Steve

                        X 1 Reply Last reply Reply Quote 0
                        • X
                          xpxp2002 @stephenw10
                          last edited by

                          Thanks for checking. It very well could be a consequence of using VFs instead of bare metal/passthrough.

                          I have to believe that it must be configurable at boot since it is being configured when an interface is reconfigured. Would those parameters be in the mlx driver source code? I'm not a programmer, but I'm wondering if a casual search of the source might reveal the strings that need to go into loader.conf.local.

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            There might be. I did search some of the source files for something similar but only found the priority flow-control options.

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.