Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    2.1.1/2.1.2 - Unable to connect LAN from power off without firewall reboot

    Scheduled Pinned Locked Moved General pfSense Questions
    23 Posts 3 Posters 3.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • E
      ESPNSTI
      last edited by

      Thanks for the suggestions.

      I would think that being unable to connect while having a static IP would eliminate any sort of DHCP problem (like you said).
      The two other devices that could have DHCP servers running, my NAS and WAP have them both disabled, and the problem still occurs with them off.
      I have my devices set up with static leases, including the main PC.
      The lease time for the DHCP server on pfSense is left at the defaults, which is 2 hours I think.
      When the PC is connected, the DHCP logs show a DHCPREQUEST once an hour, and a DHCPINFORM and DHCPACK once every 10 minutes.
      I haven't quite figured out the exact threshold is for this problem to occur, but I think it is somewhere past 4 hours after powering off a PC.
      I hadn't kept track of exact times between power off and power on, but I have started doing that now.
      So far, the longest time between power off and power on and still being able to connect has been 3 hours and 40 minutes, but I don't have a lot of data yet.

      An IP conflict/rogue device is an interesting thought, however I have a limited set of devices (about a half dozen or so) and most are set up with static DHCP leases (half of them are usually off).
      The only ones that don't have static IPs are devices that are connecting to the wireless access point, and I almost never have the WAP on.
      And virtual machines on the main PC itself, which is a rare occurrence as well.

      When this first started, I did try a packet capture on the LAN interface on pfSense before, while pinging from the PC.
      It showed no traffic for the relevant address in the capture.
      However, that has been a while ago, I will try that again next time.

      1 Reply Last reply Reply Quote 0
      • E
        ESPNSTI
        last edited by

        More information:

        This time I checked "disable all packet filtering" in "System: Advanced: Firewall and NAT" after being unable to connect.
        It did not make a difference, I still couldn't connect.
        FYI, the time the PC was powered off this time was 5 hours.

        However, I re-ran packet captures (with "disable all packet filtering" back to unchecked I think), and I must have done it incorrectly last time, since it showed some traffic this time:

        • On the LAN/BRIDGE0 bridge interface, it shows ARP broadcast requests from the main PC and ARP replies from pfSense LAN/BRIDGE0 to the main PC

        • On the OPT2/em2 interface that the main PC is connected to, it only shows ARP broadcast requests from the main PC, but no replies.

        So for some reason pfSense is able to direct the OPT2/em2 ARP broadcast to LAN/BRIDGE0, but it doesn't  forward the ARP reply from LAN/BRIDGE0 to OPT2/em2.

        I did run a ping from another PC to pfSense to confirm that ARP replies do normally show up on the specific OPT/em interface, and it shows that normally they do.

        FYI, in case it is relevant for this, in "System: Advanced: System Tunables", I have all the settings set at their defaults, except "net.link.bridge.pfil_member" is set to 0 and "net.link.bridge.pfil_bridge" is set to 1.

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          Hmm. If you run ifconfig at the console do the bridge member interfaces show as 'promiscuous mode'?

          Why are arp replies from LAN not appearing on OPT2? Perhaps because the bridge device thinks it has a route the main PC via some other interface? Maybe try packet captures on other bridge members to see if replies are ending up on the wrong interface. Just speculation.  ;)

          Steve

          1 Reply Last reply Reply Quote 0
          • E
            ESPNSTI
            last edited by

            The first post shows output from ifconfig, it shows PROMISC in the interfaces flags of the bridge member interfaces.
            I will re-run it next time to make sure.

            I will run package captures on the other interfaces next time to see if the replies end up elsewhere.

            Thanks for the suggestions.

            1 Reply Last reply Reply Quote 0
            • E
              ESPNSTI
              last edited by

              Packet captures on other interfaces don't show the missing ARP replies.

              I re-ran ifconfig, and it shows the same info as in the first post.

              However, I had success coaxing the connection back to life without a reboot.

              I ran "ifconfig em2 down" and "ifconfig em2 up" and I think that got it going again.
              However, I am not 100% sure, since I also played around with "ifconfig em2 -promisc" and "ifconfig em2 promisc", but nothing seemed to happen when I ran those.

              Edit:
              Tried it on a different PC, and it's the "ifconfig em3 up" after the "ifconfig em3 down" that does it.

              Here is the log, I ran the ifconfig commands at 07:22:09 and 07:22:13.
              It came back to life shortly after 07:22:13.

              Apr 21 07:22:19 	php: rc.newwanip: Interface does not have an IP address, nothing to do.
              Apr 21 07:22:19 	php: rc.newwanip: rc.newwanip: Informational is starting em3.
              Apr 21 07:22:16 	check_reload_status: rc.newwanip starting em3
              Apr 21 07:22:15 	php: rc.linkup: Hotplug event detected for OPT3(opt3) but ignoring since interface is configured with static IP ( )
              Apr 21 07:22:13 	php: rc.linkup: Hotplug event detected for OPT3(opt3) but ignoring since interface is configured with static IP ( )
              Apr 21 07:22:13 	check_reload_status: Linkup starting em3
              Apr 21 07:22:13 	kernel: em3: link state changed to UP
              Apr 21 07:22:09 	kernel: em3: link state changed to DOWN
              Apr 21 07:22:09 	check_reload_status: Linkup starting em3
              Apr 21 07:20:05 	php: rc.newwanip: Interface does not have an IP address, nothing to do.
              Apr 21 07:20:05 	php: rc.newwanip: rc.newwanip: Informational is starting em3.
              Apr 21 07:20:01 	check_reload_status: rc.newwanip starting em3
              Apr 21 07:20:00 	php: rc.linkup: Hotplug event detected for OPT3(opt3) but ignoring since interface is configured with static IP ( )
              Apr 21 07:19:57 	kernel: em3: link state changed to UP
              Apr 21 07:19:57 	check_reload_status: Linkup starting em3
              
              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Hmm, interesting. I wonder why the interface stops responding yet nothing in the logs.  :-
                Can you bring the interface back by unplugging and re-plugging the cable?

                Steve

                1 Reply Last reply Reply Quote 0
                • E
                  ESPNSTI
                  last edited by

                  When I tried it in the past, unplugging and re-plugging the cable didn't make a difference, neither did switching to a different port.
                  I will try that again though next time.

                  I am wondering if it's not some sort of sleep or power saving mode that is enabled in the 2.1.1 nic driver that isn't in 2.1.0 (if such a thing exists)?
                  I haven't narrowed down the exact time threshold, but it seems consistent somewhere between 4 and 5 hours, so maybe it's some sort of sleep time setting (although I would think that would be much shorter).

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Good question. The em drivers were updated for 2.1.1 so that's a distinct possibility. What are the actual chips in the NICs you have? Anything relevant in hw.em? This looks a likely suspect:

                    [2.1.2-RELEASE][root@pfsense.fire.box]/root(3): sysctl hw.em.smart_pwr_down
                    hw.em.smart_pwr_down: 0
                    

                    Anything suspect in dev.em?

                    Steve

                    1 Reply Last reply Reply Quote 0
                    • E
                      ESPNSTI
                      last edited by

                      The nics are all Intel 82574L.

                      $ sysctl hw.em
                      hw.em.num_queues: 0
                      hw.em.eee_setting: 1
                      hw.em.rx_process_limit: 100
                      hw.em.enable_msix: 1
                      hw.em.sbp: 0
                      hw.em.smart_pwr_down: 0
                      hw.em.txd: 1024
                      hw.em.rxd: 1024
                      hw.em.rx_abs_int_delay: 66
                      hw.em.tx_abs_int_delay: 66
                      hw.em.rx_int_delay: 0
                      hw.em.tx_int_delay: 66
                      
                      $ sysctl dev.em
                      dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                      dev.em.0.%driver: em
                      dev.em.0.%location: slot=0 function=0
                      dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                      dev.em.0.%parent: pci5
                      dev.em.0.nvm: -1
                      dev.em.0.debug: -1
                      dev.em.0.fc: 3
                      dev.em.0.rx_int_delay: 0
                      dev.em.0.tx_int_delay: 66
                      dev.em.0.rx_abs_int_delay: 66
                      dev.em.0.tx_abs_int_delay: 66
                      dev.em.0.itr: 488
                      dev.em.0.rx_processing_limit: 100
                      dev.em.0.eee_control: 1
                      dev.em.0.link_irq: 3
                      dev.em.0.mbuf_alloc_fail: 0
                      dev.em.0.cluster_alloc_fail: 0
                      dev.em.0.dropped: 0
                      dev.em.0.tx_dma_fail: 0
                      dev.em.0.rx_overruns: 0
                      dev.em.0.watchdog_timeouts: 0
                      dev.em.0.device_control: 1074790984
                      dev.em.0.rx_control: 67141658
                      dev.em.0.fc_high_water: 18432
                      dev.em.0.fc_low_water: 16932
                      dev.em.0.queue0.txd_head: 66
                      dev.em.0.queue0.txd_tail: 66
                      dev.em.0.queue0.tx_irq: 1332941
                      dev.em.0.queue0.no_desc_avail: 0
                      dev.em.0.queue0.rxd_head: 543
                      dev.em.0.queue0.rxd_tail: 542
                      dev.em.0.queue0.rx_irq: 2110549
                      dev.em.0.mac_stats.excess_coll: 0
                      dev.em.0.mac_stats.single_coll: 0
                      dev.em.0.mac_stats.multiple_coll: 0
                      dev.em.0.mac_stats.late_coll: 0
                      dev.em.0.mac_stats.collision_count: 0
                      dev.em.0.mac_stats.symbol_errors: 0
                      dev.em.0.mac_stats.sequence_errors: 0
                      dev.em.0.mac_stats.defer_count: 0
                      dev.em.0.mac_stats.missed_packets: 0
                      dev.em.0.mac_stats.recv_no_buff: 0
                      dev.em.0.mac_stats.recv_undersize: 0
                      dev.em.0.mac_stats.recv_fragmented: 0
                      dev.em.0.mac_stats.recv_oversize: 14
                      dev.em.0.mac_stats.recv_jabber: 0
                      dev.em.0.mac_stats.recv_errs: 1
                      dev.em.0.mac_stats.crc_errs: 1616
                      dev.em.0.mac_stats.alignment_errs: 0
                      dev.em.0.mac_stats.coll_ext_errs: 0
                      dev.em.0.mac_stats.xon_recvd: 0
                      dev.em.0.mac_stats.xon_txd: 0
                      dev.em.0.mac_stats.xoff_recvd: 0
                      dev.em.0.mac_stats.xoff_txd: 0
                      dev.em.0.mac_stats.total_pkts_recvd: 2601085
                      dev.em.0.mac_stats.good_pkts_recvd: 2599455
                      dev.em.0.mac_stats.bcast_pkts_recvd: 86896
                      dev.em.0.mac_stats.mcast_pkts_recvd: 144132
                      dev.em.0.mac_stats.rx_frames_64: 207244
                      dev.em.0.mac_stats.rx_frames_65_127: 839440
                      dev.em.0.mac_stats.rx_frames_128_255: 225214
                      dev.em.0.mac_stats.rx_frames_256_511: 121557
                      dev.em.0.mac_stats.rx_frames_512_1023: 43046
                      dev.em.0.mac_stats.rx_frames_1024_1522: 1162954
                      dev.em.0.mac_stats.good_octets_recvd: 1925347696
                      dev.em.0.mac_stats.good_octets_txd: 265656113
                      dev.em.0.mac_stats.total_pkts_txd: 2281444
                      dev.em.0.mac_stats.good_pkts_txd: 2281444
                      dev.em.0.mac_stats.bcast_pkts_txd: 19
                      dev.em.0.mac_stats.mcast_pkts_txd: 5
                      dev.em.0.mac_stats.tx_frames_64: 401078
                      dev.em.0.mac_stats.tx_frames_65_127: 1682199
                      dev.em.0.mac_stats.tx_frames_128_255: 59450
                      dev.em.0.mac_stats.tx_frames_256_511: 57767
                      dev.em.0.mac_stats.tx_frames_512_1023: 37172
                      dev.em.0.mac_stats.tx_frames_1024_1522: 43778
                      dev.em.0.mac_stats.tso_txd: 0
                      dev.em.0.mac_stats.tso_ctx_fail: 0
                      dev.em.0.interrupts.asserts: 5
                      dev.em.0.interrupts.rx_pkt_timer: 0
                      dev.em.0.interrupts.rx_abs_timer: 0
                      dev.em.0.interrupts.tx_pkt_timer: 0
                      dev.em.0.interrupts.tx_abs_timer: 0
                      dev.em.0.interrupts.tx_queue_empty: 0
                      dev.em.0.interrupts.tx_queue_min_thresh: 0
                      dev.em.0.interrupts.rx_desc_min_thresh: 0
                      dev.em.0.interrupts.rx_overrun: 0
                      dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                      dev.em.1.%driver: em
                      dev.em.1.%location: slot=0 function=0
                      dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                      dev.em.1.%parent: pci6
                      dev.em.1.nvm: -1
                      dev.em.1.debug: -1
                      dev.em.1.fc: 3
                      dev.em.1.rx_int_delay: 0
                      dev.em.1.tx_int_delay: 66
                      dev.em.1.rx_abs_int_delay: 66
                      dev.em.1.tx_abs_int_delay: 66
                      dev.em.1.itr: 488
                      dev.em.1.rx_processing_limit: 100
                      dev.em.1.eee_control: 1
                      dev.em.1.link_irq: 13
                      dev.em.1.mbuf_alloc_fail: 0
                      dev.em.1.cluster_alloc_fail: 0
                      dev.em.1.dropped: 0
                      dev.em.1.tx_dma_fail: 0
                      dev.em.1.rx_overruns: 0
                      dev.em.1.watchdog_timeouts: 0
                      dev.em.1.device_control: 1477444168
                      dev.em.1.rx_control: 67141658
                      dev.em.1.fc_high_water: 18432
                      dev.em.1.fc_low_water: 16932
                      dev.em.1.queue0.txd_head: 829
                      dev.em.1.queue0.txd_tail: 829
                      dev.em.1.queue0.tx_irq: 1680542
                      dev.em.1.queue0.no_desc_avail: 0
                      dev.em.1.queue0.rxd_head: 18
                      dev.em.1.queue0.rxd_tail: 17
                      dev.em.1.queue0.rx_irq: 1377586
                      dev.em.1.mac_stats.excess_coll: 0
                      dev.em.1.mac_stats.single_coll: 0
                      dev.em.1.mac_stats.multiple_coll: 0
                      dev.em.1.mac_stats.late_coll: 0
                      dev.em.1.mac_stats.collision_count: 0
                      dev.em.1.mac_stats.symbol_errors: 0
                      dev.em.1.mac_stats.sequence_errors: 0
                      dev.em.1.mac_stats.defer_count: 0
                      dev.em.1.mac_stats.missed_packets: 0
                      dev.em.1.mac_stats.recv_no_buff: 0
                      dev.em.1.mac_stats.recv_undersize: 0
                      dev.em.1.mac_stats.recv_fragmented: 0
                      dev.em.1.mac_stats.recv_oversize: 0
                      dev.em.1.mac_stats.recv_jabber: 0
                      dev.em.1.mac_stats.recv_errs: 0
                      dev.em.1.mac_stats.crc_errs: 0
                      dev.em.1.mac_stats.alignment_errs: 0
                      dev.em.1.mac_stats.coll_ext_errs: 0
                      dev.em.1.mac_stats.xon_recvd: 0
                      dev.em.1.mac_stats.xon_txd: 0
                      dev.em.1.mac_stats.xoff_recvd: 0
                      dev.em.1.mac_stats.xoff_txd: 0
                      dev.em.1.mac_stats.total_pkts_recvd: 6054930
                      dev.em.1.mac_stats.good_pkts_recvd: 6054930
                      dev.em.1.mac_stats.bcast_pkts_recvd: 814
                      dev.em.1.mac_stats.mcast_pkts_recvd: 18
                      dev.em.1.mac_stats.rx_frames_64: 3394973
                      dev.em.1.mac_stats.rx_frames_65_127: 1581197
                      dev.em.1.mac_stats.rx_frames_128_255: 145932
                      dev.em.1.mac_stats.rx_frames_256_511: 68688
                      dev.em.1.mac_stats.rx_frames_512_1023: 57308
                      dev.em.1.mac_stats.rx_frames_1024_1522: 806832
                      dev.em.1.mac_stats.good_octets_recvd: 1643667828
                      dev.em.1.mac_stats.good_octets_txd: 10924986586
                      dev.em.1.mac_stats.total_pkts_txd: 8607743
                      dev.em.1.mac_stats.good_pkts_txd: 8607743
                      dev.em.1.mac_stats.bcast_pkts_txd: 16764
                      dev.em.1.mac_stats.mcast_pkts_txd: 8128
                      dev.em.1.mac_stats.tx_frames_64: 435053
                      dev.em.1.mac_stats.tx_frames_65_127: 621579
                      dev.em.1.mac_stats.tx_frames_128_255: 326492
                      dev.em.1.mac_stats.tx_frames_256_511: 104439
                      dev.em.1.mac_stats.tx_frames_512_1023: 44603
                      dev.em.1.mac_stats.tx_frames_1024_1522: 7075577
                      dev.em.1.mac_stats.tso_txd: 0
                      dev.em.1.mac_stats.tso_ctx_fail: 0
                      dev.em.1.interrupts.asserts: 7
                      dev.em.1.interrupts.rx_pkt_timer: 0
                      dev.em.1.interrupts.rx_abs_timer: 0
                      dev.em.1.interrupts.tx_pkt_timer: 0
                      dev.em.1.interrupts.tx_abs_timer: 0
                      dev.em.1.interrupts.tx_queue_empty: 0
                      dev.em.1.interrupts.tx_queue_min_thresh: 0
                      dev.em.1.interrupts.rx_desc_min_thresh: 0
                      dev.em.1.interrupts.rx_overrun: 0
                      dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                      dev.em.2.%driver: em
                      dev.em.2.%location: slot=0 function=0
                      dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                      dev.em.2.%parent: pci10
                      dev.em.2.nvm: -1
                      dev.em.2.debug: -1
                      dev.em.2.fc: 3
                      dev.em.2.rx_int_delay: 0
                      dev.em.2.tx_int_delay: 66
                      dev.em.2.rx_abs_int_delay: 66
                      dev.em.2.tx_abs_int_delay: 66
                      dev.em.2.itr: 488
                      dev.em.2.rx_processing_limit: 100
                      dev.em.2.eee_control: 1
                      dev.em.2.link_irq: 57
                      dev.em.2.mbuf_alloc_fail: 0
                      dev.em.2.cluster_alloc_fail: 0
                      dev.em.2.dropped: 0
                      dev.em.2.tx_dma_fail: 0
                      dev.em.2.rx_overruns: 0
                      dev.em.2.watchdog_timeouts: 0
                      dev.em.2.device_control: 1074790984
                      dev.em.2.rx_control: 67141658
                      dev.em.2.fc_high_water: 18432
                      dev.em.2.fc_low_water: 16932
                      dev.em.2.queue0.txd_head: 475
                      dev.em.2.queue0.txd_tail: 475
                      dev.em.2.queue0.tx_irq: 1314386
                      dev.em.2.queue0.no_desc_avail: 4
                      dev.em.2.queue0.rxd_head: 565
                      dev.em.2.queue0.rxd_tail: 564
                      dev.em.2.queue0.rx_irq: 1025355
                      dev.em.2.mac_stats.excess_coll: 0
                      dev.em.2.mac_stats.single_coll: 0
                      dev.em.2.mac_stats.multiple_coll: 0
                      dev.em.2.mac_stats.late_coll: 0
                      dev.em.2.mac_stats.collision_count: 0
                      dev.em.2.mac_stats.symbol_errors: 0
                      dev.em.2.mac_stats.sequence_errors: 0
                      dev.em.2.mac_stats.defer_count: 0
                      dev.em.2.mac_stats.missed_packets: 0
                      dev.em.2.mac_stats.recv_no_buff: 0
                      dev.em.2.mac_stats.recv_undersize: 0
                      dev.em.2.mac_stats.recv_fragmented: 0
                      dev.em.2.mac_stats.recv_oversize: 0
                      dev.em.2.mac_stats.recv_jabber: 0
                      dev.em.2.mac_stats.recv_errs: 0
                      dev.em.2.mac_stats.crc_errs: 0
                      dev.em.2.mac_stats.alignment_errs: 0
                      dev.em.2.mac_stats.coll_ext_errs: 0
                      dev.em.2.mac_stats.xon_recvd: 0
                      dev.em.2.mac_stats.xon_txd: 67
                      dev.em.2.mac_stats.xoff_recvd: 0
                      dev.em.2.mac_stats.xoff_txd: 67
                      dev.em.2.mac_stats.total_pkts_recvd: 7579284
                      dev.em.2.mac_stats.good_pkts_recvd: 7579284
                      dev.em.2.mac_stats.bcast_pkts_recvd: 4056
                      dev.em.2.mac_stats.mcast_pkts_recvd: 1894
                      dev.em.2.mac_stats.rx_frames_64: 732927
                      dev.em.2.mac_stats.rx_frames_65_127: 127308
                      dev.em.2.mac_stats.rx_frames_128_255: 236522
                      dev.em.2.mac_stats.rx_frames_256_511: 32858
                      dev.em.2.mac_stats.rx_frames_512_1023: 70604
                      dev.em.2.mac_stats.rx_frames_1024_1522: 6379065
                      dev.em.2.mac_stats.good_octets_recvd: 9847640417
                      dev.em.2.mac_stats.good_octets_txd: 2161810275
                      dev.em.2.mac_stats.total_pkts_txd: 5162048
                      dev.em.2.mac_stats.good_pkts_txd: 5161914
                      dev.em.2.mac_stats.bcast_pkts_txd: 5818
                      dev.em.2.mac_stats.mcast_pkts_txd: 3845
                      dev.em.2.mac_stats.tx_frames_64: 3311268
                      dev.em.2.mac_stats.tx_frames_65_127: 402284
                      dev.em.2.mac_stats.tx_frames_128_255: 129308
                      dev.em.2.mac_stats.tx_frames_256_511: 25539
                      dev.em.2.mac_stats.tx_frames_512_1023: 92406
                      dev.em.2.mac_stats.tx_frames_1024_1522: 1201109
                      dev.em.2.mac_stats.tso_txd: 0
                      dev.em.2.mac_stats.tso_ctx_fail: 0
                      dev.em.2.interrupts.asserts: 36
                      dev.em.2.interrupts.rx_pkt_timer: 0
                      dev.em.2.interrupts.rx_abs_timer: 0
                      dev.em.2.interrupts.tx_pkt_timer: 0
                      dev.em.2.interrupts.tx_abs_timer: 0
                      dev.em.2.interrupts.tx_queue_empty: 0
                      dev.em.2.interrupts.tx_queue_min_thresh: 0
                      dev.em.2.interrupts.rx_desc_min_thresh: 0
                      dev.em.2.interrupts.rx_overrun: 0
                      dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                      dev.em.3.%driver: em
                      dev.em.3.%location: slot=0 function=0
                      dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                      dev.em.3.%parent: pci11
                      dev.em.3.nvm: -1
                      dev.em.3.debug: -1
                      dev.em.3.fc: 3
                      dev.em.3.rx_int_delay: 0
                      dev.em.3.tx_int_delay: 66
                      dev.em.3.rx_abs_int_delay: 66
                      dev.em.3.tx_abs_int_delay: 66
                      dev.em.3.itr: 488
                      dev.em.3.rx_processing_limit: 100
                      dev.em.3.eee_control: 1
                      dev.em.3.link_irq: 13
                      dev.em.3.mbuf_alloc_fail: 0
                      dev.em.3.cluster_alloc_fail: 0
                      dev.em.3.dropped: 0
                      dev.em.3.tx_dma_fail: 0
                      dev.em.3.rx_overruns: 0
                      dev.em.3.watchdog_timeouts: 0
                      dev.em.3.device_control: 1074790984
                      dev.em.3.rx_control: 67141658
                      dev.em.3.fc_high_water: 18432
                      dev.em.3.fc_low_water: 16932
                      dev.em.3.queue0.txd_head: 386
                      dev.em.3.queue0.txd_tail: 386
                      dev.em.3.queue0.tx_irq: 276
                      dev.em.3.queue0.no_desc_avail: 0
                      dev.em.3.queue0.rxd_head: 218
                      dev.em.3.queue0.rxd_tail: 217
                      dev.em.3.queue0.rx_irq: 978
                      dev.em.3.mac_stats.excess_coll: 0
                      dev.em.3.mac_stats.single_coll: 0
                      dev.em.3.mac_stats.multiple_coll: 0
                      dev.em.3.mac_stats.late_coll: 0
                      dev.em.3.mac_stats.collision_count: 0
                      dev.em.3.mac_stats.symbol_errors: 0
                      dev.em.3.mac_stats.sequence_errors: 0
                      dev.em.3.mac_stats.defer_count: 0
                      dev.em.3.mac_stats.missed_packets: 0
                      dev.em.3.mac_stats.recv_no_buff: 0
                      dev.em.3.mac_stats.recv_undersize: 0
                      dev.em.3.mac_stats.recv_fragmented: 0
                      dev.em.3.mac_stats.recv_oversize: 0
                      dev.em.3.mac_stats.recv_jabber: 0
                      dev.em.3.mac_stats.recv_errs: 0
                      dev.em.3.mac_stats.crc_errs: 0
                      dev.em.3.mac_stats.alignment_errs: 0
                      dev.em.3.mac_stats.coll_ext_errs: 0
                      dev.em.3.mac_stats.xon_recvd: 0
                      dev.em.3.mac_stats.xon_txd: 0
                      dev.em.3.mac_stats.xoff_recvd: 0
                      dev.em.3.mac_stats.xoff_txd: 0
                      dev.em.3.mac_stats.total_pkts_recvd: 979
                      dev.em.3.mac_stats.good_pkts_recvd: 979
                      dev.em.3.mac_stats.bcast_pkts_recvd: 616
                      dev.em.3.mac_stats.mcast_pkts_recvd: 158
                      dev.em.3.mac_stats.rx_frames_64: 63
                      dev.em.3.mac_stats.rx_frames_65_127: 163
                      dev.em.3.mac_stats.rx_frames_128_255: 112
                      dev.em.3.mac_stats.rx_frames_256_511: 641
                      dev.em.3.mac_stats.rx_frames_512_1023: 0
                      dev.em.3.mac_stats.rx_frames_1024_1522: 0
                      dev.em.3.mac_stats.good_octets_recvd: 255346
                      dev.em.3.mac_stats.good_octets_txd: 55615
                      dev.em.3.mac_stats.total_pkts_txd: 303
                      dev.em.3.mac_stats.good_pkts_txd: 303
                      dev.em.3.mac_stats.bcast_pkts_txd: 79
                      dev.em.3.mac_stats.mcast_pkts_txd: 32
                      dev.em.3.mac_stats.tx_frames_64: 32
                      dev.em.3.mac_stats.tx_frames_65_127: 229
                      dev.em.3.mac_stats.tx_frames_128_255: 9
                      dev.em.3.mac_stats.tx_frames_256_511: 14
                      dev.em.3.mac_stats.tx_frames_512_1023: 1
                      dev.em.3.mac_stats.tx_frames_1024_1522: 18
                      dev.em.3.mac_stats.tso_txd: 0
                      dev.em.3.mac_stats.tso_ctx_fail: 0
                      dev.em.3.interrupts.asserts: 9
                      dev.em.3.interrupts.rx_pkt_timer: 0
                      dev.em.3.interrupts.rx_abs_timer: 0
                      dev.em.3.interrupts.tx_pkt_timer: 0
                      dev.em.3.interrupts.tx_abs_timer: 0
                      dev.em.3.interrupts.tx_queue_empty: 0
                      dev.em.3.interrupts.tx_queue_min_thresh: 0
                      dev.em.3.interrupts.rx_desc_min_thresh: 0
                      dev.em.3.interrupts.rx_overrun: 0
                      dev.em.4.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                      dev.em.4.%driver: em
                      dev.em.4.%location: slot=0 function=0
                      dev.em.4.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                      dev.em.4.%parent: pci15
                      dev.em.4.nvm: -1
                      dev.em.4.debug: -1
                      dev.em.4.fc: 3
                      dev.em.4.rx_int_delay: 0
                      dev.em.4.tx_int_delay: 66
                      dev.em.4.rx_abs_int_delay: 66
                      dev.em.4.tx_abs_int_delay: 66
                      dev.em.4.itr: 488
                      dev.em.4.rx_processing_limit: 100
                      dev.em.4.eee_control: 1
                      dev.em.4.link_irq: 0
                      dev.em.4.mbuf_alloc_fail: 0
                      dev.em.4.cluster_alloc_fail: 0
                      dev.em.4.dropped: 0
                      dev.em.4.tx_dma_fail: 0
                      dev.em.4.rx_overruns: 0
                      dev.em.4.watchdog_timeouts: 0
                      dev.em.4.device_control: 1074790984
                      dev.em.4.rx_control: 67141658
                      dev.em.4.fc_high_water: 18432
                      dev.em.4.fc_low_water: 16932
                      dev.em.4.queue0.txd_head: 0
                      dev.em.4.queue0.txd_tail: 0
                      dev.em.4.queue0.tx_irq: 0
                      dev.em.4.queue0.no_desc_avail: 0
                      dev.em.4.queue0.rxd_head: 0
                      dev.em.4.queue0.rxd_tail: 1023
                      dev.em.4.queue0.rx_irq: 0
                      dev.em.4.mac_stats.excess_coll: 0
                      dev.em.4.mac_stats.single_coll: 0
                      dev.em.4.mac_stats.multiple_coll: 0
                      dev.em.4.mac_stats.late_coll: 0
                      dev.em.4.mac_stats.collision_count: 0
                      dev.em.4.mac_stats.symbol_errors: 0
                      dev.em.4.mac_stats.sequence_errors: 0
                      dev.em.4.mac_stats.defer_count: 0
                      dev.em.4.mac_stats.missed_packets: 0
                      dev.em.4.mac_stats.recv_no_buff: 0
                      dev.em.4.mac_stats.recv_undersize: 0
                      dev.em.4.mac_stats.recv_fragmented: 0
                      dev.em.4.mac_stats.recv_oversize: 0
                      dev.em.4.mac_stats.recv_jabber: 0
                      dev.em.4.mac_stats.recv_errs: 0
                      dev.em.4.mac_stats.crc_errs: 0
                      dev.em.4.mac_stats.alignment_errs: 0
                      dev.em.4.mac_stats.coll_ext_errs: 0
                      dev.em.4.mac_stats.xon_recvd: 0
                      dev.em.4.mac_stats.xon_txd: 0
                      dev.em.4.mac_stats.xoff_recvd: 0
                      dev.em.4.mac_stats.xoff_txd: 0
                      dev.em.4.mac_stats.total_pkts_recvd: 0
                      dev.em.4.mac_stats.good_pkts_recvd: 0
                      dev.em.4.mac_stats.bcast_pkts_recvd: 0
                      dev.em.4.mac_stats.mcast_pkts_recvd: 0
                      dev.em.4.mac_stats.rx_frames_64: 0
                      dev.em.4.mac_stats.rx_frames_65_127: 0
                      dev.em.4.mac_stats.rx_frames_128_255: 0
                      dev.em.4.mac_stats.rx_frames_256_511: 0
                      dev.em.4.mac_stats.rx_frames_512_1023: 0
                      dev.em.4.mac_stats.rx_frames_1024_1522: 0
                      dev.em.4.mac_stats.good_octets_recvd: 0
                      dev.em.4.mac_stats.good_octets_txd: 0
                      dev.em.4.mac_stats.total_pkts_txd: 0
                      dev.em.4.mac_stats.good_pkts_txd: 0
                      dev.em.4.mac_stats.bcast_pkts_txd: 0
                      dev.em.4.mac_stats.mcast_pkts_txd: 0
                      dev.em.4.mac_stats.tx_frames_64: 0
                      dev.em.4.mac_stats.tx_frames_65_127: 0
                      dev.em.4.mac_stats.tx_frames_128_255: 0
                      dev.em.4.mac_stats.tx_frames_256_511: 0
                      dev.em.4.mac_stats.tx_frames_512_1023: 0
                      dev.em.4.mac_stats.tx_frames_1024_1522: 0
                      dev.em.4.mac_stats.tso_txd: 0
                      dev.em.4.mac_stats.tso_ctx_fail: 0
                      dev.em.4.interrupts.asserts: 0
                      dev.em.4.interrupts.rx_pkt_timer: 0
                      dev.em.4.interrupts.rx_abs_timer: 0
                      dev.em.4.interrupts.tx_pkt_timer: 0
                      dev.em.4.interrupts.tx_abs_timer: 0
                      dev.em.4.interrupts.tx_queue_empty: 0
                      dev.em.4.interrupts.tx_queue_min_thresh: 0
                      dev.em.4.interrupts.rx_desc_min_thresh: 0
                      dev.em.4.interrupts.rx_overrun: 0
                      dev.em.5.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                      dev.em.5.%driver: em
                      dev.em.5.%location: slot=0 function=0
                      dev.em.5.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                      dev.em.5.%parent: pci16
                      dev.em.5.nvm: -1
                      dev.em.5.debug: -1
                      dev.em.5.fc: 3
                      dev.em.5.rx_int_delay: 0
                      dev.em.5.tx_int_delay: 66
                      dev.em.5.rx_abs_int_delay: 66
                      dev.em.5.tx_abs_int_delay: 66
                      dev.em.5.itr: 488
                      dev.em.5.rx_processing_limit: 100
                      dev.em.5.eee_control: 1
                      dev.em.5.link_irq: 14
                      dev.em.5.mbuf_alloc_fail: 0
                      dev.em.5.cluster_alloc_fail: 0
                      dev.em.5.dropped: 0
                      dev.em.5.tx_dma_fail: 0
                      dev.em.5.rx_overruns: 0
                      dev.em.5.watchdog_timeouts: 0
                      dev.em.5.device_control: 1477444168
                      dev.em.5.rx_control: 67141658
                      dev.em.5.fc_high_water: 18432
                      dev.em.5.fc_low_water: 16932
                      dev.em.5.queue0.txd_head: 1015
                      dev.em.5.queue0.txd_tail: 1015
                      dev.em.5.queue0.tx_irq: 31244
                      dev.em.5.queue0.no_desc_avail: 0
                      dev.em.5.queue0.rxd_head: 771
                      dev.em.5.queue0.rxd_tail: 770
                      dev.em.5.queue0.rx_irq: 29384
                      dev.em.5.mac_stats.excess_coll: 0
                      dev.em.5.mac_stats.single_coll: 0
                      dev.em.5.mac_stats.multiple_coll: 0
                      dev.em.5.mac_stats.late_coll: 0
                      dev.em.5.mac_stats.collision_count: 0
                      dev.em.5.mac_stats.symbol_errors: 0
                      dev.em.5.mac_stats.sequence_errors: 0
                      dev.em.5.mac_stats.defer_count: 0
                      dev.em.5.mac_stats.missed_packets: 0
                      dev.em.5.mac_stats.recv_no_buff: 0
                      dev.em.5.mac_stats.recv_undersize: 0
                      dev.em.5.mac_stats.recv_fragmented: 0
                      dev.em.5.mac_stats.recv_oversize: 0
                      dev.em.5.mac_stats.recv_jabber: 0
                      dev.em.5.mac_stats.recv_errs: 0
                      dev.em.5.mac_stats.crc_errs: 0
                      dev.em.5.mac_stats.alignment_errs: 0
                      dev.em.5.mac_stats.coll_ext_errs: 0
                      dev.em.5.mac_stats.xon_recvd: 0
                      dev.em.5.mac_stats.xon_txd: 0
                      dev.em.5.mac_stats.xoff_recvd: 0
                      dev.em.5.mac_stats.xoff_txd: 0
                      dev.em.5.mac_stats.total_pkts_recvd: 29443
                      dev.em.5.mac_stats.good_pkts_recvd: 29443
                      dev.em.5.mac_stats.bcast_pkts_recvd: 10091
                      dev.em.5.mac_stats.mcast_pkts_recvd: 6055
                      dev.em.5.mac_stats.rx_frames_64: 10733
                      dev.em.5.mac_stats.rx_frames_65_127: 10674
                      dev.em.5.mac_stats.rx_frames_128_255: 408
                      dev.em.5.mac_stats.rx_frames_256_511: 6155
                      dev.em.5.mac_stats.rx_frames_512_1023: 409
                      dev.em.5.mac_stats.rx_frames_1024_1522: 1064
                      dev.em.5.mac_stats.good_octets_recvd: 5941407
                      dev.em.5.mac_stats.good_octets_txd: 5567151
                      dev.em.5.mac_stats.total_pkts_txd: 31914
                      dev.em.5.mac_stats.good_pkts_txd: 31914
                      dev.em.5.mac_stats.bcast_pkts_txd: 7475
                      dev.em.5.mac_stats.mcast_pkts_txd: 2087
                      dev.em.5.mac_stats.tx_frames_64: 14358
                      dev.em.5.mac_stats.tx_frames_65_127: 13797
                      dev.em.5.mac_stats.tx_frames_128_255: 790
                      dev.em.5.mac_stats.tx_frames_256_511: 681
                      dev.em.5.mac_stats.tx_frames_512_1023: 274
                      dev.em.5.mac_stats.tx_frames_1024_1522: 2014
                      dev.em.5.mac_stats.tso_txd: 0
                      dev.em.5.mac_stats.tso_ctx_fail: 0
                      dev.em.5.interrupts.asserts: 9
                      dev.em.5.interrupts.rx_pkt_timer: 0
                      dev.em.5.interrupts.rx_abs_timer: 0
                      dev.em.5.interrupts.tx_pkt_timer: 0
                      dev.em.5.interrupts.tx_abs_timer: 0
                      dev.em.5.interrupts.tx_queue_empty: 0
                      dev.em.5.interrupts.tx_queue_min_thresh: 0
                      dev.em.5.interrupts.rx_desc_min_thresh: 0
                      dev.em.5.interrupts.rx_overrun: 0
                      dev.em.6.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                      dev.em.6.%driver: em
                      dev.em.6.%location: slot=0 function=0
                      dev.em.6.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                      dev.em.6.%parent: pci17
                      dev.em.6.nvm: -1
                      dev.em.6.debug: -1
                      dev.em.6.fc: 3
                      dev.em.6.rx_int_delay: 0
                      dev.em.6.tx_int_delay: 66
                      dev.em.6.rx_abs_int_delay: 66
                      dev.em.6.tx_abs_int_delay: 66
                      dev.em.6.itr: 488
                      dev.em.6.rx_processing_limit: 100
                      dev.em.6.eee_control: 1
                      dev.em.6.link_irq: 17
                      dev.em.6.mbuf_alloc_fail: 0
                      dev.em.6.cluster_alloc_fail: 0
                      dev.em.6.dropped: 0
                      dev.em.6.tx_dma_fail: 0
                      dev.em.6.rx_overruns: 0
                      dev.em.6.watchdog_timeouts: 0
                      dev.em.6.device_control: 1477444168
                      dev.em.6.rx_control: 67141658
                      dev.em.6.fc_high_water: 18432
                      dev.em.6.fc_low_water: 16932
                      dev.em.6.queue0.txd_head: 817
                      dev.em.6.queue0.txd_tail: 817
                      dev.em.6.queue0.tx_irq: 81780
                      dev.em.6.queue0.no_desc_avail: 0
                      dev.em.6.queue0.rxd_head: 348
                      dev.em.6.queue0.rxd_tail: 347
                      dev.em.6.queue0.rx_irq: 36669
                      dev.em.6.mac_stats.excess_coll: 0
                      dev.em.6.mac_stats.single_coll: 0
                      dev.em.6.mac_stats.multiple_coll: 0
                      dev.em.6.mac_stats.late_coll: 0
                      dev.em.6.mac_stats.collision_count: 0
                      dev.em.6.mac_stats.symbol_errors: 0
                      dev.em.6.mac_stats.sequence_errors: 0
                      dev.em.6.mac_stats.defer_count: 0
                      dev.em.6.mac_stats.missed_packets: 0
                      dev.em.6.mac_stats.recv_no_buff: 0
                      dev.em.6.mac_stats.recv_undersize: 0
                      dev.em.6.mac_stats.recv_fragmented: 0
                      dev.em.6.mac_stats.recv_oversize: 0
                      dev.em.6.mac_stats.recv_jabber: 0
                      dev.em.6.mac_stats.recv_errs: 0
                      dev.em.6.mac_stats.crc_errs: 0
                      dev.em.6.mac_stats.alignment_errs: 0
                      dev.em.6.mac_stats.coll_ext_errs: 0
                      dev.em.6.mac_stats.xon_recvd: 0
                      dev.em.6.mac_stats.xon_txd: 0
                      dev.em.6.mac_stats.xoff_recvd: 0
                      dev.em.6.mac_stats.xoff_txd: 0
                      dev.em.6.mac_stats.total_pkts_recvd: 40284
                      dev.em.6.mac_stats.good_pkts_recvd: 40284
                      dev.em.6.mac_stats.bcast_pkts_recvd: 420
                      dev.em.6.mac_stats.mcast_pkts_recvd: 376
                      dev.em.6.mac_stats.rx_frames_64: 28074
                      dev.em.6.mac_stats.rx_frames_65_127: 8076
                      dev.em.6.mac_stats.rx_frames_128_255: 927
                      dev.em.6.mac_stats.rx_frames_256_511: 800
                      dev.em.6.mac_stats.rx_frames_512_1023: 843
                      dev.em.6.mac_stats.rx_frames_1024_1522: 1564
                      dev.em.6.mac_stats.good_octets_recvd: 5925684
                      dev.em.6.mac_stats.good_octets_txd: 87073020
                      dev.em.6.mac_stats.total_pkts_txd: 88589
                      dev.em.6.mac_stats.good_pkts_txd: 88589
                      dev.em.6.mac_stats.bcast_pkts_txd: 17191
                      dev.em.6.mac_stats.mcast_pkts_txd: 7828
                      dev.em.6.mac_stats.tx_frames_64: 15737
                      dev.em.6.mac_stats.tx_frames_65_127: 8968
                      dev.em.6.mac_stats.tx_frames_128_255: 2483
                      dev.em.6.mac_stats.tx_frames_256_511: 6989
                      dev.em.6.mac_stats.tx_frames_512_1023: 704
                      dev.em.6.mac_stats.tx_frames_1024_1522: 53708
                      dev.em.6.mac_stats.tso_txd: 0
                      dev.em.6.mac_stats.tso_ctx_fail: 0
                      dev.em.6.interrupts.asserts: 11
                      dev.em.6.interrupts.rx_pkt_timer: 0
                      dev.em.6.interrupts.rx_abs_timer: 0
                      dev.em.6.interrupts.tx_pkt_timer: 0
                      dev.em.6.interrupts.tx_abs_timer: 0
                      dev.em.6.interrupts.tx_queue_empty: 0
                      dev.em.6.interrupts.tx_queue_min_thresh: 0
                      dev.em.6.interrupts.rx_desc_min_thresh: 0
                      dev.em.6.interrupts.rx_overrun: 0
                      dev.em.7.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                      dev.em.7.%driver: em
                      dev.em.7.%location: slot=0 function=0
                      dev.em.7.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                      dev.em.7.%parent: pci18
                      dev.em.7.nvm: -1
                      dev.em.7.debug: -1
                      dev.em.7.fc: 3
                      dev.em.7.rx_int_delay: 0
                      dev.em.7.tx_int_delay: 66
                      dev.em.7.rx_abs_int_delay: 66
                      dev.em.7.tx_abs_int_delay: 66
                      dev.em.7.itr: 488
                      dev.em.7.rx_processing_limit: 100
                      dev.em.7.eee_control: 1
                      dev.em.7.link_irq: 0
                      dev.em.7.mbuf_alloc_fail: 0
                      dev.em.7.cluster_alloc_fail: 0
                      dev.em.7.dropped: 0
                      dev.em.7.tx_dma_fail: 0
                      dev.em.7.rx_overruns: 0
                      dev.em.7.watchdog_timeouts: 0
                      dev.em.7.device_control: 1074790984
                      dev.em.7.rx_control: 67141658
                      dev.em.7.fc_high_water: 18432
                      dev.em.7.fc_low_water: 16932
                      dev.em.7.queue0.txd_head: 0
                      dev.em.7.queue0.txd_tail: 0
                      dev.em.7.queue0.tx_irq: 0
                      dev.em.7.queue0.no_desc_avail: 0
                      dev.em.7.queue0.rxd_head: 0
                      dev.em.7.queue0.rxd_tail: 1023
                      dev.em.7.queue0.rx_irq: 0
                      dev.em.7.mac_stats.excess_coll: 0
                      dev.em.7.mac_stats.single_coll: 0
                      dev.em.7.mac_stats.multiple_coll: 0
                      dev.em.7.mac_stats.late_coll: 0
                      dev.em.7.mac_stats.collision_count: 0
                      dev.em.7.mac_stats.symbol_errors: 0
                      dev.em.7.mac_stats.sequence_errors: 0
                      dev.em.7.mac_stats.defer_count: 0
                      dev.em.7.mac_stats.missed_packets: 0
                      dev.em.7.mac_stats.recv_no_buff: 0
                      dev.em.7.mac_stats.recv_undersize: 0
                      dev.em.7.mac_stats.recv_fragmented: 0
                      dev.em.7.mac_stats.recv_oversize: 0
                      dev.em.7.mac_stats.recv_jabber: 0
                      dev.em.7.mac_stats.recv_errs: 0
                      dev.em.7.mac_stats.crc_errs: 0
                      dev.em.7.mac_stats.alignment_errs: 0
                      dev.em.7.mac_stats.coll_ext_errs: 0
                      dev.em.7.mac_stats.xon_recvd: 0
                      dev.em.7.mac_stats.xon_txd: 0
                      dev.em.7.mac_stats.xoff_recvd: 0
                      dev.em.7.mac_stats.xoff_txd: 0
                      dev.em.7.mac_stats.total_pkts_recvd: 0
                      dev.em.7.mac_stats.good_pkts_recvd: 0
                      dev.em.7.mac_stats.bcast_pkts_recvd: 0
                      dev.em.7.mac_stats.mcast_pkts_recvd: 0
                      dev.em.7.mac_stats.rx_frames_64: 0
                      dev.em.7.mac_stats.rx_frames_65_127: 0
                      dev.em.7.mac_stats.rx_frames_128_255: 0
                      dev.em.7.mac_stats.rx_frames_256_511: 0
                      dev.em.7.mac_stats.rx_frames_512_1023: 0
                      dev.em.7.mac_stats.rx_frames_1024_1522: 0
                      dev.em.7.mac_stats.good_octets_recvd: 0
                      dev.em.7.mac_stats.good_octets_txd: 0
                      dev.em.7.mac_stats.total_pkts_txd: 0
                      dev.em.7.mac_stats.good_pkts_txd: 0
                      dev.em.7.mac_stats.bcast_pkts_txd: 0
                      dev.em.7.mac_stats.mcast_pkts_txd: 0
                      dev.em.7.mac_stats.tx_frames_64: 0
                      dev.em.7.mac_stats.tx_frames_65_127: 0
                      dev.em.7.mac_stats.tx_frames_128_255: 0
                      dev.em.7.mac_stats.tx_frames_256_511: 0
                      dev.em.7.mac_stats.tx_frames_512_1023: 0
                      dev.em.7.mac_stats.tx_frames_1024_1522: 0
                      dev.em.7.mac_stats.tso_txd: 0
                      dev.em.7.mac_stats.tso_ctx_fail: 0
                      dev.em.7.interrupts.asserts: 0
                      dev.em.7.interrupts.rx_pkt_timer: 0
                      dev.em.7.interrupts.rx_abs_timer: 0
                      dev.em.7.interrupts.tx_pkt_timer: 0
                      dev.em.7.interrupts.tx_abs_timer: 0
                      dev.em.7.interrupts.tx_queue_empty: 0
                      dev.em.7.interrupts.tx_queue_min_thresh: 0
                      dev.em.7.interrupts.rx_desc_min_thresh: 0
                      dev.em.7.interrupts.rx_overrun: 0
                      

                      I'm not sure what to watch for.
                      FYI, this was not taken while the problem was occurring.

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Hmm, very common NIC so should be reported if it was a driver issue.
                        You have a lot of CRC errors on em0. Which interface is giving trouble?
                        Try to get a set of data when it's stopped responding.

                        Steve

                        1 Reply Last reply Reply Quote 0
                        • E
                          ESPNSTI
                          last edited by

                          The nic may be common, but perhaps not a lot of people use pfsense the way I do?
                          Meaning, directly connecting PCs to the pfSense box rather than having a switch in the middle, in combination with completely unplugging the PC rather than leaving it powered off, but not unplugged.

                          My main PC is connected to em2, but I have reproduced the problem with it connected to other interfaces and with other PCs on other interfaces.

                          My WAN interface is em0, which is connected using WiMAX, don't  know if that explains the number of CRC errors or not.

                          I will try to get dev.em data the next time the problem happens.

                          Edit:

                          I tried unplugging and re-plugging the connection, that does not seem to solve the problem, but ifconfig em2 down followed by ifconfig em2 up does.

                          Here is the dev.em data while the problem happens:

                          $ sysctl dev.em
                          dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.0.%driver: em
                          dev.em.0.%location: slot=0 function=0
                          dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.0.%parent: pci5
                          dev.em.0.nvm: -1
                          dev.em.0.debug: -1
                          dev.em.0.fc: 3
                          dev.em.0.rx_int_delay: 0
                          dev.em.0.tx_int_delay: 66
                          dev.em.0.rx_abs_int_delay: 66
                          dev.em.0.tx_abs_int_delay: 66
                          dev.em.0.itr: 488
                          dev.em.0.rx_processing_limit: 100
                          dev.em.0.eee_control: 1
                          dev.em.0.link_irq: 3
                          dev.em.0.mbuf_alloc_fail: 0
                          dev.em.0.cluster_alloc_fail: 0
                          dev.em.0.dropped: 0
                          dev.em.0.tx_dma_fail: 0
                          dev.em.0.rx_overruns: 0
                          dev.em.0.watchdog_timeouts: 0
                          dev.em.0.device_control: 1074790984
                          dev.em.0.rx_control: 67141658
                          dev.em.0.fc_high_water: 18432
                          dev.em.0.fc_low_water: 16932
                          dev.em.0.queue0.txd_head: 955
                          dev.em.0.queue0.txd_tail: 955
                          dev.em.0.queue0.tx_irq: 1353023
                          dev.em.0.queue0.no_desc_avail: 0
                          dev.em.0.queue0.rxd_head: 646
                          dev.em.0.queue0.rxd_tail: 645
                          dev.em.0.queue0.rx_irq: 2194821
                          dev.em.0.mac_stats.excess_coll: 0
                          dev.em.0.mac_stats.single_coll: 0
                          dev.em.0.mac_stats.multiple_coll: 0
                          dev.em.0.mac_stats.late_coll: 0
                          dev.em.0.mac_stats.collision_count: 0
                          dev.em.0.mac_stats.symbol_errors: 0
                          dev.em.0.mac_stats.sequence_errors: 0
                          dev.em.0.mac_stats.defer_count: 0
                          dev.em.0.mac_stats.missed_packets: 0
                          dev.em.0.mac_stats.recv_no_buff: 0
                          dev.em.0.mac_stats.recv_undersize: 0
                          dev.em.0.mac_stats.recv_fragmented: 0
                          dev.em.0.mac_stats.recv_oversize: 14
                          dev.em.0.mac_stats.recv_jabber: 0
                          dev.em.0.mac_stats.recv_errs: 1
                          dev.em.0.mac_stats.crc_errs: 1858
                          dev.em.0.mac_stats.alignment_errs: 0
                          dev.em.0.mac_stats.coll_ext_errs: 0
                          dev.em.0.mac_stats.xon_recvd: 0
                          dev.em.0.mac_stats.xon_txd: 0
                          dev.em.0.mac_stats.xoff_recvd: 0
                          dev.em.0.mac_stats.xoff_txd: 0
                          dev.em.0.mac_stats.total_pkts_recvd: 2693590
                          dev.em.0.mac_stats.good_pkts_recvd: 2691718
                          dev.em.0.mac_stats.bcast_pkts_recvd: 114676
                          dev.em.0.mac_stats.mcast_pkts_recvd: 169968
                          dev.em.0.mac_stats.rx_frames_64: 230759
                          dev.em.0.mac_stats.rx_frames_65_127: 881015
                          dev.em.0.mac_stats.rx_frames_128_255: 240095
                          dev.em.0.mac_stats.rx_frames_256_511: 127210
                          dev.em.0.mac_stats.rx_frames_512_1023: 43897
                          dev.em.0.mac_stats.rx_frames_1024_1522: 1168742
                          dev.em.0.mac_stats.good_octets_recvd: 1944245208
                          dev.em.0.mac_stats.good_octets_txd: 267430581
                          dev.em.0.mac_stats.total_pkts_txd: 2301786
                          dev.em.0.mac_stats.good_pkts_txd: 2301786
                          dev.em.0.mac_stats.bcast_pkts_txd: 21
                          dev.em.0.mac_stats.mcast_pkts_txd: 5
                          dev.em.0.mac_stats.tx_frames_64: 401303
                          dev.em.0.mac_stats.tx_frames_65_127: 1701968
                          dev.em.0.mac_stats.tx_frames_128_255: 59555
                          dev.em.0.mac_stats.tx_frames_256_511: 57933
                          dev.em.0.mac_stats.tx_frames_512_1023: 37200
                          dev.em.0.mac_stats.tx_frames_1024_1522: 43827
                          dev.em.0.mac_stats.tso_txd: 0
                          dev.em.0.mac_stats.tso_ctx_fail: 0
                          dev.em.0.interrupts.asserts: 5
                          dev.em.0.interrupts.rx_pkt_timer: 0
                          dev.em.0.interrupts.rx_abs_timer: 0
                          dev.em.0.interrupts.tx_pkt_timer: 0
                          dev.em.0.interrupts.tx_abs_timer: 0
                          dev.em.0.interrupts.tx_queue_empty: 0
                          dev.em.0.interrupts.tx_queue_min_thresh: 0
                          dev.em.0.interrupts.rx_desc_min_thresh: 0
                          dev.em.0.interrupts.rx_overrun: 0
                          dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.1.%driver: em
                          dev.em.1.%location: slot=0 function=0
                          dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.1.%parent: pci6
                          dev.em.1.nvm: -1
                          dev.em.1.debug: -1
                          dev.em.1.fc: 3
                          dev.em.1.rx_int_delay: 0
                          dev.em.1.tx_int_delay: 66
                          dev.em.1.rx_abs_int_delay: 66
                          dev.em.1.tx_abs_int_delay: 66
                          dev.em.1.itr: 488
                          dev.em.1.rx_processing_limit: 100
                          dev.em.1.eee_control: 1
                          dev.em.1.link_irq: 22
                          dev.em.1.mbuf_alloc_fail: 0
                          dev.em.1.cluster_alloc_fail: 0
                          dev.em.1.dropped: 0
                          dev.em.1.tx_dma_fail: 0
                          dev.em.1.rx_overruns: 0
                          dev.em.1.watchdog_timeouts: 0
                          dev.em.1.device_control: 1477444168
                          dev.em.1.rx_control: 67141658
                          dev.em.1.fc_high_water: 18432
                          dev.em.1.fc_low_water: 16932
                          dev.em.1.queue0.txd_head: 112
                          dev.em.1.queue0.txd_tail: 112
                          dev.em.1.queue0.tx_irq: 1690961
                          dev.em.1.queue0.no_desc_avail: 0
                          dev.em.1.queue0.rxd_head: 37
                          dev.em.1.queue0.rxd_tail: 36
                          dev.em.1.queue0.rx_irq: 1381555
                          dev.em.1.mac_stats.excess_coll: 0
                          dev.em.1.mac_stats.single_coll: 0
                          dev.em.1.mac_stats.multiple_coll: 0
                          dev.em.1.mac_stats.late_coll: 0
                          dev.em.1.mac_stats.collision_count: 0
                          dev.em.1.mac_stats.symbol_errors: 0
                          dev.em.1.mac_stats.sequence_errors: 0
                          dev.em.1.mac_stats.defer_count: 0
                          dev.em.1.mac_stats.missed_packets: 0
                          dev.em.1.mac_stats.recv_no_buff: 0
                          dev.em.1.mac_stats.recv_undersize: 0
                          dev.em.1.mac_stats.recv_fragmented: 0
                          dev.em.1.mac_stats.recv_oversize: 0
                          dev.em.1.mac_stats.recv_jabber: 0
                          dev.em.1.mac_stats.recv_errs: 0
                          dev.em.1.mac_stats.crc_errs: 0
                          dev.em.1.mac_stats.alignment_errs: 0
                          dev.em.1.mac_stats.coll_ext_errs: 0
                          dev.em.1.mac_stats.xon_recvd: 0
                          dev.em.1.mac_stats.xon_txd: 0
                          dev.em.1.mac_stats.xoff_recvd: 0
                          dev.em.1.mac_stats.xoff_txd: 0
                          dev.em.1.mac_stats.total_pkts_recvd: 6059045
                          dev.em.1.mac_stats.good_pkts_recvd: 6059045
                          dev.em.1.mac_stats.bcast_pkts_recvd: 1001
                          dev.em.1.mac_stats.mcast_pkts_recvd: 36
                          dev.em.1.mac_stats.rx_frames_64: 3395197
                          dev.em.1.mac_stats.rx_frames_65_127: 1584918
                          dev.em.1.mac_stats.rx_frames_128_255: 146033
                          dev.em.1.mac_stats.rx_frames_256_511: 68732
                          dev.em.1.mac_stats.rx_frames_512_1023: 57333
                          dev.em.1.mac_stats.rx_frames_1024_1522: 806832
                          dev.em.1.mac_stats.good_octets_recvd: 1644008445
                          dev.em.1.mac_stats.good_octets_txd: 10934359364
                          dev.em.1.mac_stats.total_pkts_txd: 8618288
                          dev.em.1.mac_stats.good_pkts_txd: 8618288
                          dev.em.1.mac_stats.bcast_pkts_txd: 20285
                          dev.em.1.mac_stats.mcast_pkts_txd: 9084
                          dev.em.1.mac_stats.tx_frames_64: 438597
                          dev.em.1.mac_stats.tx_frames_65_127: 621818
                          dev.em.1.mac_stats.tx_frames_128_255: 326533
                          dev.em.1.mac_stats.tx_frames_256_511: 105392
                          dev.em.1.mac_stats.tx_frames_512_1023: 44645
                          dev.em.1.mac_stats.tx_frames_1024_1522: 7081303
                          dev.em.1.mac_stats.tso_txd: 0
                          dev.em.1.mac_stats.tso_ctx_fail: 0
                          dev.em.1.interrupts.asserts: 13
                          dev.em.1.interrupts.rx_pkt_timer: 0
                          dev.em.1.interrupts.rx_abs_timer: 0
                          dev.em.1.interrupts.tx_pkt_timer: 0
                          dev.em.1.interrupts.tx_abs_timer: 0
                          dev.em.1.interrupts.tx_queue_empty: 0
                          dev.em.1.interrupts.tx_queue_min_thresh: 0
                          dev.em.1.interrupts.rx_desc_min_thresh: 0
                          dev.em.1.interrupts.rx_overrun: 0
                          dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.2.%driver: em
                          dev.em.2.%location: slot=0 function=0
                          dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.2.%parent: pci10
                          dev.em.2.nvm: -1
                          dev.em.2.debug: -1
                          dev.em.2.fc: 3
                          dev.em.2.rx_int_delay: 0
                          dev.em.2.tx_int_delay: 66
                          dev.em.2.rx_abs_int_delay: 66
                          dev.em.2.tx_abs_int_delay: 66
                          dev.em.2.itr: 488
                          dev.em.2.rx_processing_limit: 100
                          dev.em.2.eee_control: 1
                          dev.em.2.link_irq: 65
                          dev.em.2.mbuf_alloc_fail: 0
                          dev.em.2.cluster_alloc_fail: 0
                          dev.em.2.dropped: 0
                          dev.em.2.tx_dma_fail: 0
                          dev.em.2.rx_overruns: 0
                          dev.em.2.watchdog_timeouts: 0
                          dev.em.2.device_control: 1477444168
                          dev.em.2.rx_control: 67141658
                          dev.em.2.fc_high_water: 18432
                          dev.em.2.fc_low_water: 16932
                          dev.em.2.queue0.txd_head: 475
                          dev.em.2.queue0.txd_tail: 475
                          dev.em.2.queue0.tx_irq: 1314386
                          dev.em.2.queue0.no_desc_avail: 4
                          dev.em.2.queue0.rxd_head: 737
                          dev.em.2.queue0.rxd_tail: 736
                          dev.em.2.queue0.rx_irq: 1025527
                          dev.em.2.mac_stats.excess_coll: 0
                          dev.em.2.mac_stats.single_coll: 0
                          dev.em.2.mac_stats.multiple_coll: 0
                          dev.em.2.mac_stats.late_coll: 0
                          dev.em.2.mac_stats.collision_count: 0
                          dev.em.2.mac_stats.symbol_errors: 0
                          dev.em.2.mac_stats.sequence_errors: 0
                          dev.em.2.mac_stats.defer_count: 0
                          dev.em.2.mac_stats.missed_packets: 0
                          dev.em.2.mac_stats.recv_no_buff: 0
                          dev.em.2.mac_stats.recv_undersize: 0
                          dev.em.2.mac_stats.recv_fragmented: 0
                          dev.em.2.mac_stats.recv_oversize: 0
                          dev.em.2.mac_stats.recv_jabber: 0
                          dev.em.2.mac_stats.recv_errs: 0
                          dev.em.2.mac_stats.crc_errs: 0
                          dev.em.2.mac_stats.alignment_errs: 0
                          dev.em.2.mac_stats.coll_ext_errs: 0
                          dev.em.2.mac_stats.xon_recvd: 0
                          dev.em.2.mac_stats.xon_txd: 67
                          dev.em.2.mac_stats.xoff_recvd: 0
                          dev.em.2.mac_stats.xoff_txd: 67
                          dev.em.2.mac_stats.total_pkts_recvd: 7579455
                          dev.em.2.mac_stats.good_pkts_recvd: 7579455
                          dev.em.2.mac_stats.bcast_pkts_recvd: 4221
                          dev.em.2.mac_stats.mcast_pkts_recvd: 1900
                          dev.em.2.mac_stats.rx_frames_64: 733094
                          dev.em.2.mac_stats.rx_frames_65_127: 127312
                          dev.em.2.mac_stats.rx_frames_128_255: 236522
                          dev.em.2.mac_stats.rx_frames_256_511: 32858
                          dev.em.2.mac_stats.rx_frames_512_1023: 70604
                          dev.em.2.mac_stats.rx_frames_1024_1522: 6379065
                          dev.em.2.mac_stats.good_octets_recvd: 9847651473
                          dev.em.2.mac_stats.good_octets_txd: 2161810275
                          dev.em.2.mac_stats.total_pkts_txd: 5162048
                          dev.em.2.mac_stats.good_pkts_txd: 5161914
                          dev.em.2.mac_stats.bcast_pkts_txd: 5818
                          dev.em.2.mac_stats.mcast_pkts_txd: 3845
                          dev.em.2.mac_stats.tx_frames_64: 3311268
                          dev.em.2.mac_stats.tx_frames_65_127: 402284
                          dev.em.2.mac_stats.tx_frames_128_255: 129308
                          dev.em.2.mac_stats.tx_frames_256_511: 25539
                          dev.em.2.mac_stats.tx_frames_512_1023: 92406
                          dev.em.2.mac_stats.tx_frames_1024_1522: 1201109
                          dev.em.2.mac_stats.tso_txd: 0
                          dev.em.2.mac_stats.tso_ctx_fail: 0
                          dev.em.2.interrupts.asserts: 41
                          dev.em.2.interrupts.rx_pkt_timer: 0
                          dev.em.2.interrupts.rx_abs_timer: 0
                          dev.em.2.interrupts.tx_pkt_timer: 0
                          dev.em.2.interrupts.tx_abs_timer: 0
                          dev.em.2.interrupts.tx_queue_empty: 0
                          dev.em.2.interrupts.tx_queue_min_thresh: 0
                          dev.em.2.interrupts.rx_desc_min_thresh: 0
                          dev.em.2.interrupts.rx_overrun: 0
                          dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.3.%driver: em
                          dev.em.3.%location: slot=0 function=0
                          dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.3.%parent: pci11
                          dev.em.3.nvm: -1
                          dev.em.3.debug: -1
                          dev.em.3.fc: 3
                          dev.em.3.rx_int_delay: 0
                          dev.em.3.tx_int_delay: 66
                          dev.em.3.rx_abs_int_delay: 66
                          dev.em.3.tx_abs_int_delay: 66
                          dev.em.3.itr: 488
                          dev.em.3.rx_processing_limit: 100
                          dev.em.3.eee_control: 1
                          dev.em.3.link_irq: 13
                          dev.em.3.mbuf_alloc_fail: 0
                          dev.em.3.cluster_alloc_fail: 0
                          dev.em.3.dropped: 0
                          dev.em.3.tx_dma_fail: 0
                          dev.em.3.rx_overruns: 0
                          dev.em.3.watchdog_timeouts: 0
                          dev.em.3.device_control: 1074790984
                          dev.em.3.rx_control: 67141658
                          dev.em.3.fc_high_water: 18432
                          dev.em.3.fc_low_water: 16932
                          dev.em.3.queue0.txd_head: 386
                          dev.em.3.queue0.txd_tail: 386
                          dev.em.3.queue0.tx_irq: 276
                          dev.em.3.queue0.no_desc_avail: 0
                          dev.em.3.queue0.rxd_head: 218
                          dev.em.3.queue0.rxd_tail: 217
                          dev.em.3.queue0.rx_irq: 978
                          dev.em.3.mac_stats.excess_coll: 0
                          dev.em.3.mac_stats.single_coll: 0
                          dev.em.3.mac_stats.multiple_coll: 0
                          dev.em.3.mac_stats.late_coll: 0
                          dev.em.3.mac_stats.collision_count: 0
                          dev.em.3.mac_stats.symbol_errors: 0
                          dev.em.3.mac_stats.sequence_errors: 0
                          dev.em.3.mac_stats.defer_count: 0
                          dev.em.3.mac_stats.missed_packets: 0
                          dev.em.3.mac_stats.recv_no_buff: 0
                          dev.em.3.mac_stats.recv_undersize: 0
                          dev.em.3.mac_stats.recv_fragmented: 0
                          dev.em.3.mac_stats.recv_oversize: 0
                          dev.em.3.mac_stats.recv_jabber: 0
                          dev.em.3.mac_stats.recv_errs: 0
                          dev.em.3.mac_stats.crc_errs: 0
                          dev.em.3.mac_stats.alignment_errs: 0
                          dev.em.3.mac_stats.coll_ext_errs: 0
                          dev.em.3.mac_stats.xon_recvd: 0
                          dev.em.3.mac_stats.xon_txd: 0
                          dev.em.3.mac_stats.xoff_recvd: 0
                          dev.em.3.mac_stats.xoff_txd: 0
                          dev.em.3.mac_stats.total_pkts_recvd: 979
                          dev.em.3.mac_stats.good_pkts_recvd: 979
                          dev.em.3.mac_stats.bcast_pkts_recvd: 616
                          dev.em.3.mac_stats.mcast_pkts_recvd: 158
                          dev.em.3.mac_stats.rx_frames_64: 63
                          dev.em.3.mac_stats.rx_frames_65_127: 163
                          dev.em.3.mac_stats.rx_frames_128_255: 112
                          dev.em.3.mac_stats.rx_frames_256_511: 641
                          dev.em.3.mac_stats.rx_frames_512_1023: 0
                          dev.em.3.mac_stats.rx_frames_1024_1522: 0
                          dev.em.3.mac_stats.good_octets_recvd: 255346
                          dev.em.3.mac_stats.good_octets_txd: 55615
                          dev.em.3.mac_stats.total_pkts_txd: 303
                          dev.em.3.mac_stats.good_pkts_txd: 303
                          dev.em.3.mac_stats.bcast_pkts_txd: 79
                          dev.em.3.mac_stats.mcast_pkts_txd: 32
                          dev.em.3.mac_stats.tx_frames_64: 32
                          dev.em.3.mac_stats.tx_frames_65_127: 229
                          dev.em.3.mac_stats.tx_frames_128_255: 9
                          dev.em.3.mac_stats.tx_frames_256_511: 14
                          dev.em.3.mac_stats.tx_frames_512_1023: 1
                          dev.em.3.mac_stats.tx_frames_1024_1522: 18
                          dev.em.3.mac_stats.tso_txd: 0
                          dev.em.3.mac_stats.tso_ctx_fail: 0
                          dev.em.3.interrupts.asserts: 9
                          dev.em.3.interrupts.rx_pkt_timer: 0
                          dev.em.3.interrupts.rx_abs_timer: 0
                          dev.em.3.interrupts.tx_pkt_timer: 0
                          dev.em.3.interrupts.tx_abs_timer: 0
                          dev.em.3.interrupts.tx_queue_empty: 0
                          dev.em.3.interrupts.tx_queue_min_thresh: 0
                          dev.em.3.interrupts.rx_desc_min_thresh: 0
                          dev.em.3.interrupts.rx_overrun: 0
                          dev.em.4.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.4.%driver: em
                          dev.em.4.%location: slot=0 function=0
                          dev.em.4.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.4.%parent: pci15
                          dev.em.4.nvm: -1
                          dev.em.4.debug: -1
                          dev.em.4.fc: 3
                          dev.em.4.rx_int_delay: 0
                          dev.em.4.tx_int_delay: 66
                          dev.em.4.rx_abs_int_delay: 66
                          dev.em.4.tx_abs_int_delay: 66
                          dev.em.4.itr: 488
                          dev.em.4.rx_processing_limit: 100
                          dev.em.4.eee_control: 1
                          dev.em.4.link_irq: 0
                          dev.em.4.mbuf_alloc_fail: 0
                          dev.em.4.cluster_alloc_fail: 0
                          dev.em.4.dropped: 0
                          dev.em.4.tx_dma_fail: 0
                          dev.em.4.rx_overruns: 0
                          dev.em.4.watchdog_timeouts: 0
                          dev.em.4.device_control: 1074790984
                          dev.em.4.rx_control: 67141658
                          dev.em.4.fc_high_water: 18432
                          dev.em.4.fc_low_water: 16932
                          dev.em.4.queue0.txd_head: 0
                          dev.em.4.queue0.txd_tail: 0
                          dev.em.4.queue0.tx_irq: 0
                          dev.em.4.queue0.no_desc_avail: 0
                          dev.em.4.queue0.rxd_head: 0
                          dev.em.4.queue0.rxd_tail: 1023
                          dev.em.4.queue0.rx_irq: 0
                          dev.em.4.mac_stats.excess_coll: 0
                          dev.em.4.mac_stats.single_coll: 0
                          dev.em.4.mac_stats.multiple_coll: 0
                          dev.em.4.mac_stats.late_coll: 0
                          dev.em.4.mac_stats.collision_count: 0
                          dev.em.4.mac_stats.symbol_errors: 0
                          dev.em.4.mac_stats.sequence_errors: 0
                          dev.em.4.mac_stats.defer_count: 0
                          dev.em.4.mac_stats.missed_packets: 0
                          dev.em.4.mac_stats.recv_no_buff: 0
                          dev.em.4.mac_stats.recv_undersize: 0
                          dev.em.4.mac_stats.recv_fragmented: 0
                          dev.em.4.mac_stats.recv_oversize: 0
                          dev.em.4.mac_stats.recv_jabber: 0
                          dev.em.4.mac_stats.recv_errs: 0
                          dev.em.4.mac_stats.crc_errs: 0
                          dev.em.4.mac_stats.alignment_errs: 0
                          dev.em.4.mac_stats.coll_ext_errs: 0
                          dev.em.4.mac_stats.xon_recvd: 0
                          dev.em.4.mac_stats.xon_txd: 0
                          dev.em.4.mac_stats.xoff_recvd: 0
                          dev.em.4.mac_stats.xoff_txd: 0
                          dev.em.4.mac_stats.total_pkts_recvd: 0
                          dev.em.4.mac_stats.good_pkts_recvd: 0
                          dev.em.4.mac_stats.bcast_pkts_recvd: 0
                          dev.em.4.mac_stats.mcast_pkts_recvd: 0
                          dev.em.4.mac_stats.rx_frames_64: 0
                          dev.em.4.mac_stats.rx_frames_65_127: 0
                          dev.em.4.mac_stats.rx_frames_128_255: 0
                          dev.em.4.mac_stats.rx_frames_256_511: 0
                          dev.em.4.mac_stats.rx_frames_512_1023: 0
                          dev.em.4.mac_stats.rx_frames_1024_1522: 0
                          dev.em.4.mac_stats.good_octets_recvd: 0
                          dev.em.4.mac_stats.good_octets_txd: 0
                          dev.em.4.mac_stats.total_pkts_txd: 0
                          dev.em.4.mac_stats.good_pkts_txd: 0
                          dev.em.4.mac_stats.bcast_pkts_txd: 0
                          dev.em.4.mac_stats.mcast_pkts_txd: 0
                          dev.em.4.mac_stats.tx_frames_64: 0
                          dev.em.4.mac_stats.tx_frames_65_127: 0
                          dev.em.4.mac_stats.tx_frames_128_255: 0
                          dev.em.4.mac_stats.tx_frames_256_511: 0
                          dev.em.4.mac_stats.tx_frames_512_1023: 0
                          dev.em.4.mac_stats.tx_frames_1024_1522: 0
                          dev.em.4.mac_stats.tso_txd: 0
                          dev.em.4.mac_stats.tso_ctx_fail: 0
                          dev.em.4.interrupts.asserts: 0
                          dev.em.4.interrupts.rx_pkt_timer: 0
                          dev.em.4.interrupts.rx_abs_timer: 0
                          dev.em.4.interrupts.tx_pkt_timer: 0
                          dev.em.4.interrupts.tx_abs_timer: 0
                          dev.em.4.interrupts.tx_queue_empty: 0
                          dev.em.4.interrupts.tx_queue_min_thresh: 0
                          dev.em.4.interrupts.rx_desc_min_thresh: 0
                          dev.em.4.interrupts.rx_overrun: 0
                          dev.em.5.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.5.%driver: em
                          dev.em.5.%location: slot=0 function=0
                          dev.em.5.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.5.%parent: pci16
                          dev.em.5.nvm: -1
                          dev.em.5.debug: -1
                          dev.em.5.fc: 3
                          dev.em.5.rx_int_delay: 0
                          dev.em.5.tx_int_delay: 66
                          dev.em.5.rx_abs_int_delay: 66
                          dev.em.5.tx_abs_int_delay: 66
                          dev.em.5.itr: 488
                          dev.em.5.rx_processing_limit: 100
                          dev.em.5.eee_control: 1
                          dev.em.5.link_irq: 14
                          dev.em.5.mbuf_alloc_fail: 0
                          dev.em.5.cluster_alloc_fail: 0
                          dev.em.5.dropped: 0
                          dev.em.5.tx_dma_fail: 0
                          dev.em.5.rx_overruns: 0
                          dev.em.5.watchdog_timeouts: 0
                          dev.em.5.device_control: 1477444168
                          dev.em.5.rx_control: 67141658
                          dev.em.5.fc_high_water: 18432
                          dev.em.5.fc_low_water: 16932
                          dev.em.5.queue0.txd_head: 118
                          dev.em.5.queue0.txd_tail: 118
                          dev.em.5.queue0.tx_irq: 35021
                          dev.em.5.queue0.no_desc_avail: 0
                          dev.em.5.queue0.rxd_head: 813
                          dev.em.5.queue0.rxd_tail: 812
                          dev.em.5.queue0.rx_irq: 33522
                          dev.em.5.mac_stats.excess_coll: 0
                          dev.em.5.mac_stats.single_coll: 0
                          dev.em.5.mac_stats.multiple_coll: 0
                          dev.em.5.mac_stats.late_coll: 0
                          dev.em.5.mac_stats.collision_count: 0
                          dev.em.5.mac_stats.symbol_errors: 0
                          dev.em.5.mac_stats.sequence_errors: 0
                          dev.em.5.mac_stats.defer_count: 0
                          dev.em.5.mac_stats.missed_packets: 0
                          dev.em.5.mac_stats.recv_no_buff: 0
                          dev.em.5.mac_stats.recv_undersize: 0
                          dev.em.5.mac_stats.recv_fragmented: 0
                          dev.em.5.mac_stats.recv_oversize: 0
                          dev.em.5.mac_stats.recv_jabber: 0
                          dev.em.5.mac_stats.recv_errs: 0
                          dev.em.5.mac_stats.crc_errs: 0
                          dev.em.5.mac_stats.alignment_errs: 0
                          dev.em.5.mac_stats.coll_ext_errs: 0
                          dev.em.5.mac_stats.xon_recvd: 0
                          dev.em.5.mac_stats.xon_txd: 0
                          dev.em.5.mac_stats.xoff_recvd: 0
                          dev.em.5.mac_stats.xoff_txd: 0
                          dev.em.5.mac_stats.total_pkts_recvd: 33581
                          dev.em.5.mac_stats.good_pkts_recvd: 33581
                          dev.em.5.mac_stats.bcast_pkts_recvd: 13190
                          dev.em.5.mac_stats.mcast_pkts_recvd: 6958
                          dev.em.5.mac_stats.rx_frames_64: 13832
                          dev.em.5.mac_stats.rx_frames_65_127: 10806
                          dev.em.5.mac_stats.rx_frames_128_255: 408
                          dev.em.5.mac_stats.rx_frames_256_511: 7058
                          dev.em.5.mac_stats.rx_frames_512_1023: 413
                          dev.em.5.mac_stats.rx_frames_1024_1522: 1064
                          dev.em.5.mac_stats.good_octets_recvd: 6533199
                          dev.em.5.mac_stats.good_octets_txd: 5861382
                          dev.em.5.mac_stats.total_pkts_txd: 35811
                          dev.em.5.mac_stats.good_pkts_txd: 35811
                          dev.em.5.mac_stats.bcast_pkts_txd: 8074
                          dev.em.5.mac_stats.mcast_pkts_txd: 2154
                          dev.em.5.mac_stats.tx_frames_64: 17797
                          dev.em.5.mac_stats.tx_frames_65_127: 14125
                          dev.em.5.mac_stats.tx_frames_128_255: 859
                          dev.em.5.mac_stats.tx_frames_256_511: 716
                          dev.em.5.mac_stats.tx_frames_512_1023: 294
                          dev.em.5.mac_stats.tx_frames_1024_1522: 2020
                          dev.em.5.mac_stats.tso_txd: 0
                          dev.em.5.mac_stats.tso_ctx_fail: 0
                          dev.em.5.interrupts.asserts: 9
                          dev.em.5.interrupts.rx_pkt_timer: 0
                          dev.em.5.interrupts.rx_abs_timer: 0
                          dev.em.5.interrupts.tx_pkt_timer: 0
                          dev.em.5.interrupts.tx_abs_timer: 0
                          dev.em.5.interrupts.tx_queue_empty: 0
                          dev.em.5.interrupts.tx_queue_min_thresh: 0
                          dev.em.5.interrupts.rx_desc_min_thresh: 0
                          dev.em.5.interrupts.rx_overrun: 0
                          dev.em.6.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.6.%driver: em
                          dev.em.6.%location: slot=0 function=0
                          dev.em.6.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.6.%parent: pci17
                          dev.em.6.nvm: -1
                          dev.em.6.debug: -1
                          dev.em.6.fc: 3
                          dev.em.6.rx_int_delay: 0
                          dev.em.6.tx_int_delay: 66
                          dev.em.6.rx_abs_int_delay: 66
                          dev.em.6.tx_abs_int_delay: 66
                          dev.em.6.itr: 488
                          dev.em.6.rx_processing_limit: 100
                          dev.em.6.eee_control: 1
                          dev.em.6.link_irq: 22
                          dev.em.6.mbuf_alloc_fail: 0
                          dev.em.6.cluster_alloc_fail: 0
                          dev.em.6.dropped: 0
                          dev.em.6.tx_dma_fail: 0
                          dev.em.6.rx_overruns: 0
                          dev.em.6.watchdog_timeouts: 0
                          dev.em.6.device_control: 1477444168
                          dev.em.6.rx_control: 67141658
                          dev.em.6.fc_high_water: 18432
                          dev.em.6.fc_low_water: 16932
                          dev.em.6.queue0.txd_head: 172
                          dev.em.6.queue0.txd_tail: 172
                          dev.em.6.queue0.tx_irq: 86612
                          dev.em.6.queue0.no_desc_avail: 0
                          dev.em.6.queue0.rxd_head: 886
                          dev.em.6.queue0.rxd_tail: 885
                          dev.em.6.queue0.rx_irq: 37129
                          dev.em.6.mac_stats.excess_coll: 0
                          dev.em.6.mac_stats.single_coll: 0
                          dev.em.6.mac_stats.multiple_coll: 0
                          dev.em.6.mac_stats.late_coll: 0
                          dev.em.6.mac_stats.collision_count: 0
                          dev.em.6.mac_stats.symbol_errors: 0
                          dev.em.6.mac_stats.sequence_errors: 0
                          dev.em.6.mac_stats.defer_count: 0
                          dev.em.6.mac_stats.missed_packets: 0
                          dev.em.6.mac_stats.recv_no_buff: 0
                          dev.em.6.mac_stats.recv_undersize: 0
                          dev.em.6.mac_stats.recv_fragmented: 0
                          dev.em.6.mac_stats.recv_oversize: 0
                          dev.em.6.mac_stats.recv_jabber: 0
                          dev.em.6.mac_stats.recv_errs: 0
                          dev.em.6.mac_stats.crc_errs: 0
                          dev.em.6.mac_stats.alignment_errs: 0
                          dev.em.6.mac_stats.coll_ext_errs: 0
                          dev.em.6.mac_stats.xon_recvd: 0
                          dev.em.6.mac_stats.xon_txd: 0
                          dev.em.6.mac_stats.xoff_recvd: 0
                          dev.em.6.mac_stats.xoff_txd: 0
                          dev.em.6.mac_stats.total_pkts_recvd: 40822
                          dev.em.6.mac_stats.good_pkts_recvd: 40822
                          dev.em.6.mac_stats.bcast_pkts_recvd: 538
                          dev.em.6.mac_stats.mcast_pkts_recvd: 445
                          dev.em.6.mac_stats.rx_frames_64: 28322
                          dev.em.6.mac_stats.rx_frames_65_127: 8188
                          dev.em.6.mac_stats.rx_frames_128_255: 966
                          dev.em.6.mac_stats.rx_frames_256_511: 818
                          dev.em.6.mac_stats.rx_frames_512_1023: 957
                          dev.em.6.mac_stats.rx_frames_1024_1522: 1571
                          dev.em.6.mac_stats.good_octets_recvd: 6040146
                          dev.em.6.mac_stats.good_octets_txd: 88048854
                          dev.em.6.mac_stats.total_pkts_txd: 93690
                          dev.em.6.mac_stats.good_pkts_txd: 93690
                          dev.em.6.mac_stats.bcast_pkts_txd: 20785
                          dev.em.6.mac_stats.mcast_pkts_txd: 8747
                          dev.em.6.mac_stats.tx_frames_64: 19296
                          dev.em.6.mac_stats.tx_frames_65_127: 9220
                          dev.em.6.mac_stats.tx_frames_128_255: 2549
                          dev.em.6.mac_stats.tx_frames_256_511: 7998
                          dev.em.6.mac_stats.tx_frames_512_1023: 746
                          dev.em.6.mac_stats.tx_frames_1024_1522: 53881
                          dev.em.6.mac_stats.tso_txd: 0
                          dev.em.6.mac_stats.tso_ctx_fail: 0
                          dev.em.6.interrupts.asserts: 14
                          dev.em.6.interrupts.rx_pkt_timer: 0
                          dev.em.6.interrupts.rx_abs_timer: 0
                          dev.em.6.interrupts.tx_pkt_timer: 0
                          dev.em.6.interrupts.tx_abs_timer: 0
                          dev.em.6.interrupts.tx_queue_empty: 0
                          dev.em.6.interrupts.tx_queue_min_thresh: 0
                          dev.em.6.interrupts.rx_desc_min_thresh: 0
                          dev.em.6.interrupts.rx_overrun: 0
                          dev.em.7.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.7.%driver: em
                          dev.em.7.%location: slot=0 function=0
                          dev.em.7.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.7.%parent: pci18
                          dev.em.7.nvm: -1
                          dev.em.7.debug: -1
                          dev.em.7.fc: 3
                          dev.em.7.rx_int_delay: 0
                          dev.em.7.tx_int_delay: 66
                          dev.em.7.rx_abs_int_delay: 66
                          dev.em.7.tx_abs_int_delay: 66
                          dev.em.7.itr: 488
                          dev.em.7.rx_processing_limit: 100
                          dev.em.7.eee_control: 1
                          dev.em.7.link_irq: 0
                          dev.em.7.mbuf_alloc_fail: 0
                          dev.em.7.cluster_alloc_fail: 0
                          dev.em.7.dropped: 0
                          dev.em.7.tx_dma_fail: 0
                          dev.em.7.rx_overruns: 0
                          dev.em.7.watchdog_timeouts: 0
                          dev.em.7.device_control: 1074790984
                          dev.em.7.rx_control: 67141658
                          dev.em.7.fc_high_water: 18432
                          dev.em.7.fc_low_water: 16932
                          dev.em.7.queue0.txd_head: 0
                          dev.em.7.queue0.txd_tail: 0
                          dev.em.7.queue0.tx_irq: 0
                          dev.em.7.queue0.no_desc_avail: 0
                          dev.em.7.queue0.rxd_head: 0
                          dev.em.7.queue0.rxd_tail: 1023
                          dev.em.7.queue0.rx_irq: 0
                          dev.em.7.mac_stats.excess_coll: 0
                          dev.em.7.mac_stats.single_coll: 0
                          dev.em.7.mac_stats.multiple_coll: 0
                          dev.em.7.mac_stats.late_coll: 0
                          dev.em.7.mac_stats.collision_count: 0
                          dev.em.7.mac_stats.symbol_errors: 0
                          dev.em.7.mac_stats.sequence_errors: 0
                          dev.em.7.mac_stats.defer_count: 0
                          dev.em.7.mac_stats.missed_packets: 0
                          dev.em.7.mac_stats.recv_no_buff: 0
                          dev.em.7.mac_stats.recv_undersize: 0
                          dev.em.7.mac_stats.recv_fragmented: 0
                          dev.em.7.mac_stats.recv_oversize: 0
                          dev.em.7.mac_stats.recv_jabber: 0
                          dev.em.7.mac_stats.recv_errs: 0
                          dev.em.7.mac_stats.crc_errs: 0
                          dev.em.7.mac_stats.alignment_errs: 0
                          dev.em.7.mac_stats.coll_ext_errs: 0
                          dev.em.7.mac_stats.xon_recvd: 0
                          dev.em.7.mac_stats.xon_txd: 0
                          dev.em.7.mac_stats.xoff_recvd: 0
                          dev.em.7.mac_stats.xoff_txd: 0
                          dev.em.7.mac_stats.total_pkts_recvd: 0
                          dev.em.7.mac_stats.good_pkts_recvd: 0
                          dev.em.7.mac_stats.bcast_pkts_recvd: 0
                          dev.em.7.mac_stats.mcast_pkts_recvd: 0
                          dev.em.7.mac_stats.rx_frames_64: 0
                          dev.em.7.mac_stats.rx_frames_65_127: 0
                          dev.em.7.mac_stats.rx_frames_128_255: 0
                          dev.em.7.mac_stats.rx_frames_256_511: 0
                          dev.em.7.mac_stats.rx_frames_512_1023: 0
                          dev.em.7.mac_stats.rx_frames_1024_1522: 0
                          dev.em.7.mac_stats.good_octets_recvd: 0
                          dev.em.7.mac_stats.good_octets_txd: 0
                          dev.em.7.mac_stats.total_pkts_txd: 0
                          dev.em.7.mac_stats.good_pkts_txd: 0
                          dev.em.7.mac_stats.bcast_pkts_txd: 0
                          dev.em.7.mac_stats.mcast_pkts_txd: 0
                          dev.em.7.mac_stats.tx_frames_64: 0
                          dev.em.7.mac_stats.tx_frames_65_127: 0
                          dev.em.7.mac_stats.tx_frames_128_255: 0
                          dev.em.7.mac_stats.tx_frames_256_511: 0
                          dev.em.7.mac_stats.tx_frames_512_1023: 0
                          dev.em.7.mac_stats.tx_frames_1024_1522: 0
                          dev.em.7.mac_stats.tso_txd: 0
                          dev.em.7.mac_stats.tso_ctx_fail: 0
                          dev.em.7.interrupts.asserts: 0
                          dev.em.7.interrupts.rx_pkt_timer: 0
                          dev.em.7.interrupts.rx_abs_timer: 0
                          dev.em.7.interrupts.tx_pkt_timer: 0
                          dev.em.7.interrupts.tx_abs_timer: 0
                          dev.em.7.interrupts.tx_queue_empty: 0
                          dev.em.7.interrupts.tx_queue_min_thresh: 0
                          dev.em.7.interrupts.rx_desc_min_thresh: 0
                          dev.em.7.interrupts.rx_overrun: 0
                          

                          And this is shortly after, having revived the connection using ifconfig:

                          $ sysctl dev.em
                          dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.0.%driver: em
                          dev.em.0.%location: slot=0 function=0
                          dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.0.%parent: pci5
                          dev.em.0.nvm: -1
                          dev.em.0.debug: -1
                          dev.em.0.fc: 3
                          dev.em.0.rx_int_delay: 0
                          dev.em.0.tx_int_delay: 66
                          dev.em.0.rx_abs_int_delay: 66
                          dev.em.0.tx_abs_int_delay: 66
                          dev.em.0.itr: 488
                          dev.em.0.rx_processing_limit: 100
                          dev.em.0.eee_control: 1
                          dev.em.0.link_irq: 3
                          dev.em.0.mbuf_alloc_fail: 0
                          dev.em.0.cluster_alloc_fail: 0
                          dev.em.0.dropped: 0
                          dev.em.0.tx_dma_fail: 0
                          dev.em.0.rx_overruns: 0
                          dev.em.0.watchdog_timeouts: 0
                          dev.em.0.device_control: 1074790984
                          dev.em.0.rx_control: 67141658
                          dev.em.0.fc_high_water: 18432
                          dev.em.0.fc_low_water: 16932
                          dev.em.0.queue0.txd_head: 685
                          dev.em.0.queue0.txd_tail: 685
                          dev.em.0.queue0.tx_irq: 1353736
                          dev.em.0.queue0.no_desc_avail: 0
                          dev.em.0.queue0.rxd_head: 676
                          dev.em.0.queue0.rxd_tail: 675
                          dev.em.0.queue0.rx_irq: 2196823
                          dev.em.0.mac_stats.excess_coll: 0
                          dev.em.0.mac_stats.single_coll: 0
                          dev.em.0.mac_stats.multiple_coll: 0
                          dev.em.0.mac_stats.late_coll: 0
                          dev.em.0.mac_stats.collision_count: 0
                          dev.em.0.mac_stats.symbol_errors: 0
                          dev.em.0.mac_stats.sequence_errors: 0
                          dev.em.0.mac_stats.defer_count: 0
                          dev.em.0.mac_stats.missed_packets: 0
                          dev.em.0.mac_stats.recv_no_buff: 0
                          dev.em.0.mac_stats.recv_undersize: 0
                          dev.em.0.mac_stats.recv_fragmented: 0
                          dev.em.0.mac_stats.recv_oversize: 14
                          dev.em.0.mac_stats.recv_jabber: 0
                          dev.em.0.mac_stats.recv_errs: 1
                          dev.em.0.mac_stats.crc_errs: 1864
                          dev.em.0.mac_stats.alignment_errs: 0
                          dev.em.0.mac_stats.coll_ext_errs: 0
                          dev.em.0.mac_stats.xon_recvd: 0
                          dev.em.0.mac_stats.xon_txd: 0
                          dev.em.0.mac_stats.xoff_recvd: 0
                          dev.em.0.mac_stats.xoff_txd: 0
                          dev.em.0.mac_stats.total_pkts_recvd: 2695671
                          dev.em.0.mac_stats.good_pkts_recvd: 2693793
                          dev.em.0.mac_stats.bcast_pkts_recvd: 115211
                          dev.em.0.mac_stats.mcast_pkts_recvd: 170573
                          dev.em.0.mac_stats.rx_frames_64: 231428
                          dev.em.0.mac_stats.rx_frames_65_127: 881736
                          dev.em.0.mac_stats.rx_frames_128_255: 240428
                          dev.em.0.mac_stats.rx_frames_256_511: 127402
                          dev.em.0.mac_stats.rx_frames_512_1023: 43918
                          dev.em.0.mac_stats.rx_frames_1024_1522: 1168881
                          dev.em.0.mac_stats.good_octets_recvd: 1944697083
                          dev.em.0.mac_stats.good_octets_txd: 267511513
                          dev.em.0.mac_stats.total_pkts_txd: 2302538
                          dev.em.0.mac_stats.good_pkts_txd: 2302538
                          dev.em.0.mac_stats.bcast_pkts_txd: 21
                          dev.em.0.mac_stats.mcast_pkts_txd: 5
                          dev.em.0.mac_stats.tx_frames_64: 401423
                          dev.em.0.mac_stats.tx_frames_65_127: 1702538
                          dev.em.0.mac_stats.tx_frames_128_255: 59586
                          dev.em.0.mac_stats.tx_frames_256_511: 57937
                          dev.em.0.mac_stats.tx_frames_512_1023: 37227
                          dev.em.0.mac_stats.tx_frames_1024_1522: 43827
                          dev.em.0.mac_stats.tso_txd: 0
                          dev.em.0.mac_stats.tso_ctx_fail: 0
                          dev.em.0.interrupts.asserts: 5
                          dev.em.0.interrupts.rx_pkt_timer: 0
                          dev.em.0.interrupts.rx_abs_timer: 0
                          dev.em.0.interrupts.tx_pkt_timer: 0
                          dev.em.0.interrupts.tx_abs_timer: 0
                          dev.em.0.interrupts.tx_queue_empty: 0
                          dev.em.0.interrupts.tx_queue_min_thresh: 0
                          dev.em.0.interrupts.rx_desc_min_thresh: 0
                          dev.em.0.interrupts.rx_overrun: 0
                          dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.1.%driver: em
                          dev.em.1.%location: slot=0 function=0
                          dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.1.%parent: pci6
                          dev.em.1.nvm: -1
                          dev.em.1.debug: -1
                          dev.em.1.fc: 3
                          dev.em.1.rx_int_delay: 0
                          dev.em.1.tx_int_delay: 66
                          dev.em.1.rx_abs_int_delay: 66
                          dev.em.1.tx_abs_int_delay: 66
                          dev.em.1.itr: 488
                          dev.em.1.rx_processing_limit: 100
                          dev.em.1.eee_control: 1
                          dev.em.1.link_irq: 22
                          dev.em.1.mbuf_alloc_fail: 0
                          dev.em.1.cluster_alloc_fail: 0
                          dev.em.1.dropped: 0
                          dev.em.1.tx_dma_fail: 0
                          dev.em.1.rx_overruns: 0
                          dev.em.1.watchdog_timeouts: 0
                          dev.em.1.device_control: 1477444168
                          dev.em.1.rx_control: 67141658
                          dev.em.1.fc_high_water: 18432
                          dev.em.1.fc_low_water: 16932
                          dev.em.1.queue0.txd_head: 839
                          dev.em.1.queue0.txd_tail: 839
                          dev.em.1.queue0.tx_irq: 1691646
                          dev.em.1.queue0.no_desc_avail: 0
                          dev.em.1.queue0.rxd_head: 100
                          dev.em.1.queue0.rxd_tail: 99
                          dev.em.1.queue0.rx_irq: 1381613
                          dev.em.1.mac_stats.excess_coll: 0
                          dev.em.1.mac_stats.single_coll: 0
                          dev.em.1.mac_stats.multiple_coll: 0
                          dev.em.1.mac_stats.late_coll: 0
                          dev.em.1.mac_stats.collision_count: 0
                          dev.em.1.mac_stats.symbol_errors: 0
                          dev.em.1.mac_stats.sequence_errors: 0
                          dev.em.1.mac_stats.defer_count: 0
                          dev.em.1.mac_stats.missed_packets: 0
                          dev.em.1.mac_stats.recv_no_buff: 0
                          dev.em.1.mac_stats.recv_undersize: 0
                          dev.em.1.mac_stats.recv_fragmented: 0
                          dev.em.1.mac_stats.recv_oversize: 0
                          dev.em.1.mac_stats.recv_jabber: 0
                          dev.em.1.mac_stats.recv_errs: 0
                          dev.em.1.mac_stats.crc_errs: 0
                          dev.em.1.mac_stats.alignment_errs: 0
                          dev.em.1.mac_stats.coll_ext_errs: 0
                          dev.em.1.mac_stats.xon_recvd: 0
                          dev.em.1.mac_stats.xon_txd: 0
                          dev.em.1.mac_stats.xoff_recvd: 0
                          dev.em.1.mac_stats.xoff_txd: 0
                          dev.em.1.mac_stats.total_pkts_recvd: 6059107
                          dev.em.1.mac_stats.good_pkts_recvd: 6059107
                          dev.em.1.mac_stats.bcast_pkts_recvd: 1003
                          dev.em.1.mac_stats.mcast_pkts_recvd: 36
                          dev.em.1.mac_stats.rx_frames_64: 3395204
                          dev.em.1.mac_stats.rx_frames_65_127: 1584944
                          dev.em.1.mac_stats.rx_frames_128_255: 146061
                          dev.em.1.mac_stats.rx_frames_256_511: 68733
                          dev.em.1.mac_stats.rx_frames_512_1023: 57333
                          dev.em.1.mac_stats.rx_frames_1024_1522: 806832
                          dev.em.1.mac_stats.good_octets_recvd: 1644015882
                          dev.em.1.mac_stats.good_octets_txd: 10934440772
                          dev.em.1.mac_stats.total_pkts_txd: 8619016
                          dev.em.1.mac_stats.good_pkts_txd: 8619016
                          dev.em.1.mac_stats.bcast_pkts_txd: 20889
                          dev.em.1.mac_stats.mcast_pkts_txd: 9162
                          dev.em.1.mac_stats.tx_frames_64: 439143
                          dev.em.1.mac_stats.tx_frames_65_127: 621925
                          dev.em.1.mac_stats.tx_frames_128_255: 326546
                          dev.em.1.mac_stats.tx_frames_256_511: 105432
                          dev.em.1.mac_stats.tx_frames_512_1023: 44657
                          dev.em.1.mac_stats.tx_frames_1024_1522: 7081313
                          dev.em.1.mac_stats.tso_txd: 0
                          dev.em.1.mac_stats.tso_ctx_fail: 0
                          dev.em.1.interrupts.asserts: 13
                          dev.em.1.interrupts.rx_pkt_timer: 0
                          dev.em.1.interrupts.rx_abs_timer: 0
                          dev.em.1.interrupts.tx_pkt_timer: 0
                          dev.em.1.interrupts.tx_abs_timer: 0
                          dev.em.1.interrupts.tx_queue_empty: 0
                          dev.em.1.interrupts.tx_queue_min_thresh: 0
                          dev.em.1.interrupts.rx_desc_min_thresh: 0
                          dev.em.1.interrupts.rx_overrun: 0
                          dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.2.%driver: em
                          dev.em.2.%location: slot=0 function=0
                          dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.2.%parent: pci10
                          dev.em.2.nvm: -1
                          dev.em.2.debug: -1
                          dev.em.2.fc: 3
                          dev.em.2.rx_int_delay: 0
                          dev.em.2.tx_int_delay: 66
                          dev.em.2.rx_abs_int_delay: 66
                          dev.em.2.tx_abs_int_delay: 66
                          dev.em.2.itr: 488
                          dev.em.2.rx_processing_limit: 100
                          dev.em.2.eee_control: 1
                          dev.em.2.link_irq: 69
                          dev.em.2.mbuf_alloc_fail: 0
                          dev.em.2.cluster_alloc_fail: 0
                          dev.em.2.dropped: 0
                          dev.em.2.tx_dma_fail: 0
                          dev.em.2.rx_overruns: 0
                          dev.em.2.watchdog_timeouts: 0
                          dev.em.2.device_control: 1477444168
                          dev.em.2.rx_control: 67141658
                          dev.em.2.fc_high_water: 18432
                          dev.em.2.fc_low_water: 16932
                          dev.em.2.queue0.txd_head: 240
                          dev.em.2.queue0.txd_tail: 240
                          dev.em.2.queue0.tx_irq: 1314599
                          dev.em.2.queue0.no_desc_avail: 4
                          dev.em.2.queue0.rxd_head: 293
                          dev.em.2.queue0.rxd_tail: 292
                          dev.em.2.queue0.rx_irq: 1026038
                          dev.em.2.mac_stats.excess_coll: 0
                          dev.em.2.mac_stats.single_coll: 0
                          dev.em.2.mac_stats.multiple_coll: 0
                          dev.em.2.mac_stats.late_coll: 0
                          dev.em.2.mac_stats.collision_count: 0
                          dev.em.2.mac_stats.symbol_errors: 0
                          dev.em.2.mac_stats.sequence_errors: 0
                          dev.em.2.mac_stats.defer_count: 0
                          dev.em.2.mac_stats.missed_packets: 0
                          dev.em.2.mac_stats.recv_no_buff: 0
                          dev.em.2.mac_stats.recv_undersize: 0
                          dev.em.2.mac_stats.recv_fragmented: 0
                          dev.em.2.mac_stats.recv_oversize: 0
                          dev.em.2.mac_stats.recv_jabber: 0
                          dev.em.2.mac_stats.recv_errs: 0
                          dev.em.2.mac_stats.crc_errs: 0
                          dev.em.2.mac_stats.alignment_errs: 0
                          dev.em.2.mac_stats.coll_ext_errs: 0
                          dev.em.2.mac_stats.xon_recvd: 0
                          dev.em.2.mac_stats.xon_txd: 67
                          dev.em.2.mac_stats.xoff_recvd: 0
                          dev.em.2.mac_stats.xoff_txd: 67
                          dev.em.2.mac_stats.total_pkts_recvd: 7579992
                          dev.em.2.mac_stats.good_pkts_recvd: 7579992
                          dev.em.2.mac_stats.bcast_pkts_recvd: 4530
                          dev.em.2.mac_stats.mcast_pkts_recvd: 1960
                          dev.em.2.mac_stats.rx_frames_64: 733376
                          dev.em.2.mac_stats.rx_frames_65_127: 127545
                          dev.em.2.mac_stats.rx_frames_128_255: 236524
                          dev.em.2.mac_stats.rx_frames_256_511: 32859
                          dev.em.2.mac_stats.rx_frames_512_1023: 70613
                          dev.em.2.mac_stats.rx_frames_1024_1522: 6379075
                          dev.em.2.mac_stats.good_octets_recvd: 9847706051
                          dev.em.2.mac_stats.good_octets_txd: 2161839705
                          dev.em.2.mac_stats.total_pkts_txd: 5162265
                          dev.em.2.mac_stats.good_pkts_txd: 5162131
                          dev.em.2.mac_stats.bcast_pkts_txd: 5846
                          dev.em.2.mac_stats.mcast_pkts_txd: 3858
                          dev.em.2.mac_stats.tx_frames_64: 3311306
                          dev.em.2.mac_stats.tx_frames_65_127: 402438
                          dev.em.2.mac_stats.tx_frames_128_255: 129314
                          dev.em.2.mac_stats.tx_frames_256_511: 25549
                          dev.em.2.mac_stats.tx_frames_512_1023: 92410
                          dev.em.2.mac_stats.tx_frames_1024_1522: 1201114
                          dev.em.2.mac_stats.tso_txd: 0
                          dev.em.2.mac_stats.tso_ctx_fail: 0
                          dev.em.2.interrupts.asserts: 44
                          dev.em.2.interrupts.rx_pkt_timer: 0
                          dev.em.2.interrupts.rx_abs_timer: 0
                          dev.em.2.interrupts.tx_pkt_timer: 0
                          dev.em.2.interrupts.tx_abs_timer: 0
                          dev.em.2.interrupts.tx_queue_empty: 0
                          dev.em.2.interrupts.tx_queue_min_thresh: 0
                          dev.em.2.interrupts.rx_desc_min_thresh: 0
                          dev.em.2.interrupts.rx_overrun: 0
                          dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.3.%driver: em
                          dev.em.3.%location: slot=0 function=0
                          dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.3.%parent: pci11
                          dev.em.3.nvm: -1
                          dev.em.3.debug: -1
                          dev.em.3.fc: 3
                          dev.em.3.rx_int_delay: 0
                          dev.em.3.tx_int_delay: 66
                          dev.em.3.rx_abs_int_delay: 66
                          dev.em.3.tx_abs_int_delay: 66
                          dev.em.3.itr: 488
                          dev.em.3.rx_processing_limit: 100
                          dev.em.3.eee_control: 1
                          dev.em.3.link_irq: 13
                          dev.em.3.mbuf_alloc_fail: 0
                          dev.em.3.cluster_alloc_fail: 0
                          dev.em.3.dropped: 0
                          dev.em.3.tx_dma_fail: 0
                          dev.em.3.rx_overruns: 0
                          dev.em.3.watchdog_timeouts: 0
                          dev.em.3.device_control: 1074790984
                          dev.em.3.rx_control: 67141658
                          dev.em.3.fc_high_water: 18432
                          dev.em.3.fc_low_water: 16932
                          dev.em.3.queue0.txd_head: 386
                          dev.em.3.queue0.txd_tail: 386
                          dev.em.3.queue0.tx_irq: 276
                          dev.em.3.queue0.no_desc_avail: 0
                          dev.em.3.queue0.rxd_head: 218
                          dev.em.3.queue0.rxd_tail: 217
                          dev.em.3.queue0.rx_irq: 978
                          dev.em.3.mac_stats.excess_coll: 0
                          dev.em.3.mac_stats.single_coll: 0
                          dev.em.3.mac_stats.multiple_coll: 0
                          dev.em.3.mac_stats.late_coll: 0
                          dev.em.3.mac_stats.collision_count: 0
                          dev.em.3.mac_stats.symbol_errors: 0
                          dev.em.3.mac_stats.sequence_errors: 0
                          dev.em.3.mac_stats.defer_count: 0
                          dev.em.3.mac_stats.missed_packets: 0
                          dev.em.3.mac_stats.recv_no_buff: 0
                          dev.em.3.mac_stats.recv_undersize: 0
                          dev.em.3.mac_stats.recv_fragmented: 0
                          dev.em.3.mac_stats.recv_oversize: 0
                          dev.em.3.mac_stats.recv_jabber: 0
                          dev.em.3.mac_stats.recv_errs: 0
                          dev.em.3.mac_stats.crc_errs: 0
                          dev.em.3.mac_stats.alignment_errs: 0
                          dev.em.3.mac_stats.coll_ext_errs: 0
                          dev.em.3.mac_stats.xon_recvd: 0
                          dev.em.3.mac_stats.xon_txd: 0
                          dev.em.3.mac_stats.xoff_recvd: 0
                          dev.em.3.mac_stats.xoff_txd: 0
                          dev.em.3.mac_stats.total_pkts_recvd: 979
                          dev.em.3.mac_stats.good_pkts_recvd: 979
                          dev.em.3.mac_stats.bcast_pkts_recvd: 616
                          dev.em.3.mac_stats.mcast_pkts_recvd: 158
                          dev.em.3.mac_stats.rx_frames_64: 63
                          dev.em.3.mac_stats.rx_frames_65_127: 163
                          dev.em.3.mac_stats.rx_frames_128_255: 112
                          dev.em.3.mac_stats.rx_frames_256_511: 641
                          dev.em.3.mac_stats.rx_frames_512_1023: 0
                          dev.em.3.mac_stats.rx_frames_1024_1522: 0
                          dev.em.3.mac_stats.good_octets_recvd: 255346
                          dev.em.3.mac_stats.good_octets_txd: 55615
                          dev.em.3.mac_stats.total_pkts_txd: 303
                          dev.em.3.mac_stats.good_pkts_txd: 303
                          dev.em.3.mac_stats.bcast_pkts_txd: 79
                          dev.em.3.mac_stats.mcast_pkts_txd: 32
                          dev.em.3.mac_stats.tx_frames_64: 32
                          dev.em.3.mac_stats.tx_frames_65_127: 229
                          dev.em.3.mac_stats.tx_frames_128_255: 9
                          dev.em.3.mac_stats.tx_frames_256_511: 14
                          dev.em.3.mac_stats.tx_frames_512_1023: 1
                          dev.em.3.mac_stats.tx_frames_1024_1522: 18
                          dev.em.3.mac_stats.tso_txd: 0
                          dev.em.3.mac_stats.tso_ctx_fail: 0
                          dev.em.3.interrupts.asserts: 9
                          dev.em.3.interrupts.rx_pkt_timer: 0
                          dev.em.3.interrupts.rx_abs_timer: 0
                          dev.em.3.interrupts.tx_pkt_timer: 0
                          dev.em.3.interrupts.tx_abs_timer: 0
                          dev.em.3.interrupts.tx_queue_empty: 0
                          dev.em.3.interrupts.tx_queue_min_thresh: 0
                          dev.em.3.interrupts.rx_desc_min_thresh: 0
                          dev.em.3.interrupts.rx_overrun: 0
                          dev.em.4.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.4.%driver: em
                          dev.em.4.%location: slot=0 function=0
                          dev.em.4.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.4.%parent: pci15
                          dev.em.4.nvm: -1
                          dev.em.4.debug: -1
                          dev.em.4.fc: 3
                          dev.em.4.rx_int_delay: 0
                          dev.em.4.tx_int_delay: 66
                          dev.em.4.rx_abs_int_delay: 66
                          dev.em.4.tx_abs_int_delay: 66
                          dev.em.4.itr: 488
                          dev.em.4.rx_processing_limit: 100
                          dev.em.4.eee_control: 1
                          dev.em.4.link_irq: 0
                          dev.em.4.mbuf_alloc_fail: 0
                          dev.em.4.cluster_alloc_fail: 0
                          dev.em.4.dropped: 0
                          dev.em.4.tx_dma_fail: 0
                          dev.em.4.rx_overruns: 0
                          dev.em.4.watchdog_timeouts: 0
                          dev.em.4.device_control: 1074790984
                          dev.em.4.rx_control: 67141658
                          dev.em.4.fc_high_water: 18432
                          dev.em.4.fc_low_water: 16932
                          dev.em.4.queue0.txd_head: 0
                          dev.em.4.queue0.txd_tail: 0
                          dev.em.4.queue0.tx_irq: 0
                          dev.em.4.queue0.no_desc_avail: 0
                          dev.em.4.queue0.rxd_head: 0
                          dev.em.4.queue0.rxd_tail: 1023
                          dev.em.4.queue0.rx_irq: 0
                          dev.em.4.mac_stats.excess_coll: 0
                          dev.em.4.mac_stats.single_coll: 0
                          dev.em.4.mac_stats.multiple_coll: 0
                          dev.em.4.mac_stats.late_coll: 0
                          dev.em.4.mac_stats.collision_count: 0
                          dev.em.4.mac_stats.symbol_errors: 0
                          dev.em.4.mac_stats.sequence_errors: 0
                          dev.em.4.mac_stats.defer_count: 0
                          dev.em.4.mac_stats.missed_packets: 0
                          dev.em.4.mac_stats.recv_no_buff: 0
                          dev.em.4.mac_stats.recv_undersize: 0
                          dev.em.4.mac_stats.recv_fragmented: 0
                          dev.em.4.mac_stats.recv_oversize: 0
                          dev.em.4.mac_stats.recv_jabber: 0
                          dev.em.4.mac_stats.recv_errs: 0
                          dev.em.4.mac_stats.crc_errs: 0
                          dev.em.4.mac_stats.alignment_errs: 0
                          dev.em.4.mac_stats.coll_ext_errs: 0
                          dev.em.4.mac_stats.xon_recvd: 0
                          dev.em.4.mac_stats.xon_txd: 0
                          dev.em.4.mac_stats.xoff_recvd: 0
                          dev.em.4.mac_stats.xoff_txd: 0
                          dev.em.4.mac_stats.total_pkts_recvd: 0
                          dev.em.4.mac_stats.good_pkts_recvd: 0
                          dev.em.4.mac_stats.bcast_pkts_recvd: 0
                          dev.em.4.mac_stats.mcast_pkts_recvd: 0
                          dev.em.4.mac_stats.rx_frames_64: 0
                          dev.em.4.mac_stats.rx_frames_65_127: 0
                          dev.em.4.mac_stats.rx_frames_128_255: 0
                          dev.em.4.mac_stats.rx_frames_256_511: 0
                          dev.em.4.mac_stats.rx_frames_512_1023: 0
                          dev.em.4.mac_stats.rx_frames_1024_1522: 0
                          dev.em.4.mac_stats.good_octets_recvd: 0
                          dev.em.4.mac_stats.good_octets_txd: 0
                          dev.em.4.mac_stats.total_pkts_txd: 0
                          dev.em.4.mac_stats.good_pkts_txd: 0
                          dev.em.4.mac_stats.bcast_pkts_txd: 0
                          dev.em.4.mac_stats.mcast_pkts_txd: 0
                          dev.em.4.mac_stats.tx_frames_64: 0
                          dev.em.4.mac_stats.tx_frames_65_127: 0
                          dev.em.4.mac_stats.tx_frames_128_255: 0
                          dev.em.4.mac_stats.tx_frames_256_511: 0
                          dev.em.4.mac_stats.tx_frames_512_1023: 0
                          dev.em.4.mac_stats.tx_frames_1024_1522: 0
                          dev.em.4.mac_stats.tso_txd: 0
                          dev.em.4.mac_stats.tso_ctx_fail: 0
                          dev.em.4.interrupts.asserts: 0
                          dev.em.4.interrupts.rx_pkt_timer: 0
                          dev.em.4.interrupts.rx_abs_timer: 0
                          dev.em.4.interrupts.tx_pkt_timer: 0
                          dev.em.4.interrupts.tx_abs_timer: 0
                          dev.em.4.interrupts.tx_queue_empty: 0
                          dev.em.4.interrupts.tx_queue_min_thresh: 0
                          dev.em.4.interrupts.rx_desc_min_thresh: 0
                          dev.em.4.interrupts.rx_overrun: 0
                          dev.em.5.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.5.%driver: em
                          dev.em.5.%location: slot=0 function=0
                          dev.em.5.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.5.%parent: pci16
                          dev.em.5.nvm: -1
                          dev.em.5.debug: -1
                          dev.em.5.fc: 3
                          dev.em.5.rx_int_delay: 0
                          dev.em.5.tx_int_delay: 66
                          dev.em.5.rx_abs_int_delay: 66
                          dev.em.5.tx_abs_int_delay: 66
                          dev.em.5.itr: 488
                          dev.em.5.rx_processing_limit: 100
                          dev.em.5.eee_control: 1
                          dev.em.5.link_irq: 14
                          dev.em.5.mbuf_alloc_fail: 0
                          dev.em.5.cluster_alloc_fail: 0
                          dev.em.5.dropped: 0
                          dev.em.5.tx_dma_fail: 0
                          dev.em.5.rx_overruns: 0
                          dev.em.5.watchdog_timeouts: 0
                          dev.em.5.device_control: 1477444168
                          dev.em.5.rx_control: 67141658
                          dev.em.5.fc_high_water: 18432
                          dev.em.5.fc_low_water: 16932
                          dev.em.5.queue0.txd_head: 908
                          dev.em.5.queue0.txd_tail: 908
                          dev.em.5.queue0.tx_irq: 35644
                          dev.em.5.queue0.no_desc_avail: 0
                          dev.em.5.queue0.rxd_head: 913
                          dev.em.5.queue0.rxd_tail: 912
                          dev.em.5.queue0.rx_irq: 33622
                          dev.em.5.mac_stats.excess_coll: 0
                          dev.em.5.mac_stats.single_coll: 0
                          dev.em.5.mac_stats.multiple_coll: 0
                          dev.em.5.mac_stats.late_coll: 0
                          dev.em.5.mac_stats.collision_count: 0
                          dev.em.5.mac_stats.symbol_errors: 0
                          dev.em.5.mac_stats.sequence_errors: 0
                          dev.em.5.mac_stats.defer_count: 0
                          dev.em.5.mac_stats.missed_packets: 0
                          dev.em.5.mac_stats.recv_no_buff: 0
                          dev.em.5.mac_stats.recv_undersize: 0
                          dev.em.5.mac_stats.recv_fragmented: 0
                          dev.em.5.mac_stats.recv_oversize: 0
                          dev.em.5.mac_stats.recv_jabber: 0
                          dev.em.5.mac_stats.recv_errs: 0
                          dev.em.5.mac_stats.crc_errs: 0
                          dev.em.5.mac_stats.alignment_errs: 0
                          dev.em.5.mac_stats.coll_ext_errs: 0
                          dev.em.5.mac_stats.xon_recvd: 0
                          dev.em.5.mac_stats.xon_txd: 0
                          dev.em.5.mac_stats.xoff_recvd: 0
                          dev.em.5.mac_stats.xoff_txd: 0
                          dev.em.5.mac_stats.total_pkts_recvd: 33681
                          dev.em.5.mac_stats.good_pkts_recvd: 33681
                          dev.em.5.mac_stats.bcast_pkts_recvd: 13265
                          dev.em.5.mac_stats.mcast_pkts_recvd: 6979
                          dev.em.5.mac_stats.rx_frames_64: 13907
                          dev.em.5.mac_stats.rx_frames_65_127: 10809
                          dev.em.5.mac_stats.rx_frames_128_255: 408
                          dev.em.5.mac_stats.rx_frames_256_511: 7079
                          dev.em.5.mac_stats.rx_frames_512_1023: 414
                          dev.em.5.mac_stats.rx_frames_1024_1522: 1064
                          dev.em.5.mac_stats.good_octets_recvd: 6547683
                          dev.em.5.mac_stats.good_octets_txd: 5926006
                          dev.em.5.mac_stats.total_pkts_txd: 36476
                          dev.em.5.mac_stats.good_pkts_txd: 36476
                          dev.em.5.mac_stats.bcast_pkts_txd: 8603
                          dev.em.5.mac_stats.mcast_pkts_txd: 2211
                          dev.em.5.mac_stats.tx_frames_64: 18335
                          dev.em.5.mac_stats.tx_frames_65_127: 14216
                          dev.em.5.mac_stats.tx_frames_128_255: 870
                          dev.em.5.mac_stats.tx_frames_256_511: 719
                          dev.em.5.mac_stats.tx_frames_512_1023: 306
                          dev.em.5.mac_stats.tx_frames_1024_1522: 2030
                          dev.em.5.mac_stats.tso_txd: 0
                          dev.em.5.mac_stats.tso_ctx_fail: 0
                          dev.em.5.interrupts.asserts: 9
                          dev.em.5.interrupts.rx_pkt_timer: 0
                          dev.em.5.interrupts.rx_abs_timer: 0
                          dev.em.5.interrupts.tx_pkt_timer: 0
                          dev.em.5.interrupts.tx_abs_timer: 0
                          dev.em.5.interrupts.tx_queue_empty: 0
                          dev.em.5.interrupts.tx_queue_min_thresh: 0
                          dev.em.5.interrupts.rx_desc_min_thresh: 0
                          dev.em.5.interrupts.rx_overrun: 0
                          dev.em.6.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.6.%driver: em
                          dev.em.6.%location: slot=0 function=0
                          dev.em.6.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.6.%parent: pci17
                          dev.em.6.nvm: -1
                          dev.em.6.debug: -1
                          dev.em.6.fc: 3
                          dev.em.6.rx_int_delay: 0
                          dev.em.6.tx_int_delay: 66
                          dev.em.6.rx_abs_int_delay: 66
                          dev.em.6.tx_abs_int_delay: 66
                          dev.em.6.itr: 488
                          dev.em.6.rx_processing_limit: 100
                          dev.em.6.eee_control: 1
                          dev.em.6.link_irq: 22
                          dev.em.6.mbuf_alloc_fail: 0
                          dev.em.6.cluster_alloc_fail: 0
                          dev.em.6.dropped: 0
                          dev.em.6.tx_dma_fail: 0
                          dev.em.6.rx_overruns: 0
                          dev.em.6.watchdog_timeouts: 0
                          dev.em.6.device_control: 1477444168
                          dev.em.6.rx_control: 67141658
                          dev.em.6.fc_high_water: 18432
                          dev.em.6.fc_low_water: 16932
                          dev.em.6.queue0.txd_head: 103
                          dev.em.6.queue0.txd_tail: 103
                          dev.em.6.queue0.tx_irq: 87586
                          dev.em.6.queue0.no_desc_avail: 0
                          dev.em.6.queue0.rxd_head: 840
                          dev.em.6.queue0.rxd_tail: 839
                          dev.em.6.queue0.rx_irq: 37860
                          dev.em.6.mac_stats.excess_coll: 0
                          dev.em.6.mac_stats.single_coll: 0
                          dev.em.6.mac_stats.multiple_coll: 0
                          dev.em.6.mac_stats.late_coll: 0
                          dev.em.6.mac_stats.collision_count: 0
                          dev.em.6.mac_stats.symbol_errors: 0
                          dev.em.6.mac_stats.sequence_errors: 0
                          dev.em.6.mac_stats.defer_count: 0
                          dev.em.6.mac_stats.missed_packets: 0
                          dev.em.6.mac_stats.recv_no_buff: 0
                          dev.em.6.mac_stats.recv_undersize: 0
                          dev.em.6.mac_stats.recv_fragmented: 0
                          dev.em.6.mac_stats.recv_oversize: 0
                          dev.em.6.mac_stats.recv_jabber: 0
                          dev.em.6.mac_stats.recv_errs: 0
                          dev.em.6.mac_stats.crc_errs: 0
                          dev.em.6.mac_stats.alignment_errs: 0
                          dev.em.6.mac_stats.coll_ext_errs: 0
                          dev.em.6.mac_stats.xon_recvd: 0
                          dev.em.6.mac_stats.xon_txd: 0
                          dev.em.6.mac_stats.xoff_recvd: 0
                          dev.em.6.mac_stats.xoff_txd: 0
                          dev.em.6.mac_stats.total_pkts_recvd: 41796
                          dev.em.6.mac_stats.good_pkts_recvd: 41796
                          dev.em.6.mac_stats.bcast_pkts_recvd: 755
                          dev.em.6.mac_stats.mcast_pkts_recvd: 454
                          dev.em.6.mac_stats.rx_frames_64: 29124
                          dev.em.6.mac_stats.rx_frames_65_127: 8291
                          dev.em.6.mac_stats.rx_frames_128_255: 984
                          dev.em.6.mac_stats.rx_frames_256_511: 824
                          dev.em.6.mac_stats.rx_frames_512_1023: 993
                          dev.em.6.mac_stats.rx_frames_1024_1522: 1580
                          dev.em.6.mac_stats.good_octets_recvd: 6142890
                          dev.em.6.mac_stats.good_octets_txd: 89550641
                          dev.em.6.mac_stats.total_pkts_txd: 95436
                          dev.em.6.mac_stats.good_pkts_txd: 95436
                          dev.em.6.mac_stats.bcast_pkts_txd: 21172
                          dev.em.6.mac_stats.mcast_pkts_txd: 8816
                          dev.em.6.mac_stats.tx_frames_64: 19702
                          dev.em.6.mac_stats.tx_frames_65_127: 9482
                          dev.em.6.mac_stats.tx_frames_128_255: 2561
                          dev.em.6.mac_stats.tx_frames_256_511: 8128
                          dev.em.6.mac_stats.tx_frames_512_1023: 765
                          dev.em.6.mac_stats.tx_frames_1024_1522: 54798
                          dev.em.6.mac_stats.tso_txd: 0
                          dev.em.6.mac_stats.tso_ctx_fail: 0
                          dev.em.6.interrupts.asserts: 14
                          dev.em.6.interrupts.rx_pkt_timer: 0
                          dev.em.6.interrupts.rx_abs_timer: 0
                          dev.em.6.interrupts.tx_pkt_timer: 0
                          dev.em.6.interrupts.tx_abs_timer: 0
                          dev.em.6.interrupts.tx_queue_empty: 0
                          dev.em.6.interrupts.tx_queue_min_thresh: 0
                          dev.em.6.interrupts.rx_desc_min_thresh: 0
                          dev.em.6.interrupts.rx_overrun: 0
                          dev.em.7.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
                          dev.em.7.%driver: em
                          dev.em.7.%location: slot=0 function=0
                          dev.em.7.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
                          dev.em.7.%parent: pci18
                          dev.em.7.nvm: -1
                          dev.em.7.debug: -1
                          dev.em.7.fc: 3
                          dev.em.7.rx_int_delay: 0
                          dev.em.7.tx_int_delay: 66
                          dev.em.7.rx_abs_int_delay: 66
                          dev.em.7.tx_abs_int_delay: 66
                          dev.em.7.itr: 488
                          dev.em.7.rx_processing_limit: 100
                          dev.em.7.eee_control: 1
                          dev.em.7.link_irq: 0
                          dev.em.7.mbuf_alloc_fail: 0
                          dev.em.7.cluster_alloc_fail: 0
                          dev.em.7.dropped: 0
                          dev.em.7.tx_dma_fail: 0
                          dev.em.7.rx_overruns: 0
                          dev.em.7.watchdog_timeouts: 0
                          dev.em.7.device_control: 1074790984
                          dev.em.7.rx_control: 67141658
                          dev.em.7.fc_high_water: 18432
                          dev.em.7.fc_low_water: 16932
                          dev.em.7.queue0.txd_head: 0
                          dev.em.7.queue0.txd_tail: 0
                          dev.em.7.queue0.tx_irq: 0
                          dev.em.7.queue0.no_desc_avail: 0
                          dev.em.7.queue0.rxd_head: 0
                          dev.em.7.queue0.rxd_tail: 1023
                          dev.em.7.queue0.rx_irq: 0
                          dev.em.7.mac_stats.excess_coll: 0
                          dev.em.7.mac_stats.single_coll: 0
                          dev.em.7.mac_stats.multiple_coll: 0
                          dev.em.7.mac_stats.late_coll: 0
                          dev.em.7.mac_stats.collision_count: 0
                          dev.em.7.mac_stats.symbol_errors: 0
                          dev.em.7.mac_stats.sequence_errors: 0
                          dev.em.7.mac_stats.defer_count: 0
                          dev.em.7.mac_stats.missed_packets: 0
                          dev.em.7.mac_stats.recv_no_buff: 0
                          dev.em.7.mac_stats.recv_undersize: 0
                          dev.em.7.mac_stats.recv_fragmented: 0
                          dev.em.7.mac_stats.recv_oversize: 0
                          dev.em.7.mac_stats.recv_jabber: 0
                          dev.em.7.mac_stats.recv_errs: 0
                          dev.em.7.mac_stats.crc_errs: 0
                          dev.em.7.mac_stats.alignment_errs: 0
                          dev.em.7.mac_stats.coll_ext_errs: 0
                          dev.em.7.mac_stats.xon_recvd: 0
                          dev.em.7.mac_stats.xon_txd: 0
                          dev.em.7.mac_stats.xoff_recvd: 0
                          dev.em.7.mac_stats.xoff_txd: 0
                          dev.em.7.mac_stats.total_pkts_recvd: 0
                          dev.em.7.mac_stats.good_pkts_recvd: 0
                          dev.em.7.mac_stats.bcast_pkts_recvd: 0
                          dev.em.7.mac_stats.mcast_pkts_recvd: 0
                          dev.em.7.mac_stats.rx_frames_64: 0
                          dev.em.7.mac_stats.rx_frames_65_127: 0
                          dev.em.7.mac_stats.rx_frames_128_255: 0
                          dev.em.7.mac_stats.rx_frames_256_511: 0
                          dev.em.7.mac_stats.rx_frames_512_1023: 0
                          dev.em.7.mac_stats.rx_frames_1024_1522: 0
                          dev.em.7.mac_stats.good_octets_recvd: 0
                          dev.em.7.mac_stats.good_octets_txd: 0
                          dev.em.7.mac_stats.total_pkts_txd: 0
                          dev.em.7.mac_stats.good_pkts_txd: 0
                          dev.em.7.mac_stats.bcast_pkts_txd: 0
                          dev.em.7.mac_stats.mcast_pkts_txd: 0
                          dev.em.7.mac_stats.tx_frames_64: 0
                          dev.em.7.mac_stats.tx_frames_65_127: 0
                          dev.em.7.mac_stats.tx_frames_128_255: 0
                          dev.em.7.mac_stats.tx_frames_256_511: 0
                          dev.em.7.mac_stats.tx_frames_512_1023: 0
                          dev.em.7.mac_stats.tx_frames_1024_1522: 0
                          dev.em.7.mac_stats.tso_txd: 0
                          dev.em.7.mac_stats.tso_ctx_fail: 0
                          dev.em.7.interrupts.asserts: 0
                          dev.em.7.interrupts.rx_pkt_timer: 0
                          dev.em.7.interrupts.rx_abs_timer: 0
                          dev.em.7.interrupts.tx_pkt_timer: 0
                          dev.em.7.interrupts.tx_abs_timer: 0
                          dev.em.7.interrupts.tx_queue_empty: 0
                          dev.em.7.interrupts.tx_queue_min_thresh: 0
                          dev.em.7.interrupts.rx_desc_min_thresh: 0
                          dev.em.7.interrupts.rx_overrun: 0
                          
                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            I have a box that is quite similar in many ways, same NICs, and I often use it as you do, without a switch. I too have all six em NICs bridged together. However I haven't done so recently as I'm using as a test box with 2.2. Thinking back I'm not sure if I ever had that arrangement with 2.1.1/2.
                            Interestingly I have experienced very similar behavior on that box but not on the em interfaces, on its one fxp NIC. That NIC will not negotiate a connection if it is powered up before the machine at the other end. In that case it was a harware (perhaps firmware) level issue where it would fail to negotiate a speed and duplex wiht the NIC at the other end resulting in no link. The link LED would flash intermittently as it tried and failed. It has no problem doing it it the other end is already powered up. I never found a good solution to it but you work around it by manually setting the link speed and duplex. Try that.

                            Steve

                            1 Reply Last reply Reply Quote 0
                            • E
                              ESPNSTI
                              last edited by

                              I have played around with nic settings on the connecting PC before, and that didn't seem to make a difference, but I will try that again.

                              I looked at some of the differences between the em drivers in 2.1.0 and 2.1.2.
                              In 2.1.2 the version is 7.3.8, I think 2.1.0 might be 7.3.2.
                              One thing I noticed is that the hw.em.eee_setting (Energy Efficient Ethernet) default might have changed.
                              It now seems to default to 1 and I think it defaulted to 0 before.
                              However I can't figure out by the somewhat cryptic code if that means it's now enabled or disabled.
                              Anyway, I set hw.em.eee_setting=0 in /boot/loader.conf.local and rebooted, but that unfortunately didn't fix it.

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                Damn that seems like a great suspect. The sysctl had changed after boot?

                                In my case you could see the NIC wasn't making a connection as ifconfig showed 'no carrier' I don't belive you're seeing that though.

                                Steve

                                1 Reply Last reply Reply Quote 0
                                • E
                                  ESPNSTI
                                  last edited by

                                  After the reboot, sysctl did show eee_setting as 0.

                                  Edit: I tried changing the link speed and duplex of the connecting PC, and that made no difference.

                                  There are some tests included in the Windows nic driver that I ran, and during the time the problem occurs, they all succeed except for the connection test.

                                  ![Cable Test.gif](/public/imported_attachments/1/Cable Test.gif)
                                  ![Cable Test.gif_thumb](/public/imported_attachments/1/Cable Test.gif_thumb)
                                  ![Hardware Test.gif](/public/imported_attachments/1/Hardware Test.gif)
                                  ![Hardware Test.gif_thumb](/public/imported_attachments/1/Hardware Test.gif_thumb)

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post
                                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.