2.1.1/2.1.2 - Unable to connect LAN from power off without firewall reboot



  • Hi,

    I have been running pfSense 2.1.0 for about half a year, it's been rock solid. pfSense has been fantastic.
    After 2.1.1 came out, I upgraded to it and the upgrade itself went very smooth.

    However, after having my desktop machine off (unplugged) for a number of hours, it will not correctly connect to the pfSense box on startup.
    The desktop sends packets, but doesn't seem to receive any (see attachments).

    At first I thought it just wasn't able to get it's IP address from DHCP, but if I set up my desktop with a static IP, it still didn't connect.
    When attempting to ping the pfSense box, I get this:

    C:\Windows\system32>ping 10.x.x.2
    
    Pinging 10.x.x.2 with 32 bytes of data:
    Reply from 10.x.x.11: Destination host unreachable.
    Reply from 10.x.x.11: Destination host unreachable.
    Reply from 10.x.x.11: Destination host unreachable.
    Reply from 10.x.x.11: Destination host unreachable.
    
    Ping statistics for 10.x.x.2:
        Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    

    My current configuration is this: (I have updated to 2.1.2 now, but the problem originally started on 2.1.1)
    pfSense 2.1.2 on a Soekris net6501-50 (Intel 82574L Gigabit Ethernet ports)
    I have WAN assigned to em0, OPT1-7 assigned to em1-7 with IPv4 and IPv6 set to none, and LAN is assigned to Bridge0, consisting of OPT1-7.
    I have these packages installed: ntop, OpenVPN Client Export, pfBlocker, Shellcmd, snort, system patches.
    My computers are directly attached to the pfSense box, no other devices are in between the pfSense box and the computers.

    If I reboot the pfSense box, the Desktop computer will connect just fine again.
    If I then power off the Desktop and unplug it, and start it up again, it will still connect fine.
    It's only after a certain amount of time is past (don't exactly know how much) while the Desktop is unplugged that it will not connect.
    Normally when I get home from work, I switch on my Desktop for about an hour or so and then turn it off (unplugged).
    If I turn it back on a few (2-4) hours later, it still connects fine.
    However, after I shut it down finally at night (unplugged), then 18 or so hours later when I get back from work, it will not connect.

    My main desktop computer isn't the only computer that seems to have this problem, all devices that I have tried so far that were completely powered off (unplugged), seem to have this problem.
    If I shut down, but don't unplug, like I do with my NAS box, then it connects just fine.
    I have also tried this with another computer, where I shut down, but did not unplug, and that connects just fine, while it didn't connect if I unplugged it.

    The last time this happened, I hooked up a computer to the pfSense box using a serial connection.
    Using the "Top" command, I could see that it was running at idle. (0% usage on processes).
    I could also ping my NAS box and google.com just fine.
    I could not ping my main desktop (while a static IP is set), however an ARP entry did show up in the table after I tried that.

    Edit:
    Here is the system log for this situation, my main desktop in this is em2/opt2.
    At 10:31, I started my desktop from power off and it is unable to connect.
    At 10:50, I restart pfSense.
    At 10:52, pfSense is back up and I can connect with the main desktop again.
    At 11:01, I shut down and unplug the main desktop.
    At 11:02, I start up the main desktop.

    Apr 13 10:30:47 syslogd: kernel boot file is /boot/kernel/kernel
    Apr 13 10:30:47 kernel: pid 25196 (syslogd), uid 0: exited on signal 11 (core dumped)
    Apr 13 10:31:40 check_reload_status: Linkup starting em2
    Apr 13 10:31:40 kernel: em2: link state changed to UP
    Apr 13 10:31:43 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 10:31:43 check_reload_status: rc.newwanip starting em2
    Apr 13 10:31:43 check_reload_status: Linkup starting em2
    Apr 13 10:31:43 kernel: em2: link state changed to DOWN
    Apr 13 10:31:47 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 10:31:47 kernel: em2: link state changed to UP
    Apr 13 10:31:47 check_reload_status: Linkup starting em2
    Apr 13 10:31:47 php: rc.newwanip: rc.newwanip: Informational is starting em2.
    Apr 13 10:31:47 php: rc.newwanip: Interface does not have an IP address, nothing to do.
    Apr 13 10:31:50 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 10:31:50 check_reload_status: rc.newwanip starting em2
    Apr 13 10:31:53 php: rc.newwanip: rc.newwanip: Informational is starting em2.
    Apr 13 10:31:53 php: rc.newwanip: Interface does not have an IP address, nothing to do.
    Apr 13 10:32:00 check_reload_status: Linkup starting em2
    Apr 13 10:32:00 kernel: em2: link state changed to DOWN
    Apr 13 10:32:03 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 10:32:03 check_reload_status: Linkup starting em2
    Apr 13 10:32:03 kernel: em2: link state changed to UP
    Apr 13 10:32:06 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 10:32:06 check_reload_status: rc.newwanip starting em2
    Apr 13 10:32:09 php: rc.newwanip: rc.newwanip: Informational is starting em2.
    Apr 13 10:32:09 php: rc.newwanip: Interface does not have an IP address, nothing to do.
    Apr 13 10:50:05 php: /reboot.php: Stopping all packages.
    Apr 13 10:50:05 SnortStartup[65038]: Snort STOP for WAN(28432_em0)...
    Apr 13 10:50:06 snort[90047]: *** Caught Term-Signal
    Apr 13 10:50:06 kernel: em0: promiscuous mode disabled
    Apr 13 10:50:09 SnortStartup[73402]: Snort STOP for LAN(12814_bridge0)...
    Apr 13 10:50:10 snort[21674]: *** Caught Term-Signal
    Apr 13 10:50:10 kernel: bridge0: promiscuous mode disabled
    Apr 13 10:50:16 shutdown: reboot by root: 
    Apr 13 10:50:16 shutdown: reboot by root: 
    Apr 13 10:50:17 sshlockout[97096]: sshlockout/webConfigurator v3.0 starting up
    Apr 13 10:50:19 syslogd: exiting on signal 15
    Apr 13 10:52:16 syslogd: kernel boot file is /boot/kernel/kernel
    Apr 13 10:52:16 kernel: Copyright (c) 1992-2012 The FreeBSD Project.
    Apr 13 10:52:16 kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    Apr 13 10:52:16 kernel: The Regents of the University of California. All rights reserved.
    Apr 13 10:52:16 kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.
    Apr 13 10:52:16 kernel: FreeBSD 8.3-RELEASE-p15 #1: Thu Apr 10 05:45:21 EDT 2014
    Apr 13 10:52:16 kernel: root@pf2_1_1_i386.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense_wrap.8.i386 i386
    Apr 13 10:52:16 kernel: Timecounter "i8254" frequency 1193182 Hz quality 0
    Apr 13 10:52:16 kernel: CPU: Genuine Intel(R) CPU        @ 1.00GHz (1000.01-MHz 686-class CPU)
    Apr 13 10:52:16 kernel: Origin = "GenuineIntel"  Id = 0x20661  Family = 6  Model = 26  Stepping = 1
    Apr 13 10:52:16 kernel: Features=0xbfe9fbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>Apr 13 10:52:16 kernel: Features2=0x40e3bd <sse3,dtes64,mon,ds_cpl,vmx,est,tm2,ssse3,cx16,xtpr,pdcm,movbe>Apr 13 10:52:16 kernel: AMD Features=0x20100000 <nx,lm>Apr 13 10:52:16 kernel: AMD Features2=0x1 <lahf>Apr 13 10:52:16 kernel: TSC: P-state invariant
    Apr 13 10:52:16 kernel: real memory  = 1073610752 (1023 MB)
    Apr 13 10:52:16 kernel: avail memory = 1027940352 (980 MB)
    Apr 13 10:52:16 kernel: MPTable: <soekris  net6501 =""  ="">Apr 13 10:52:16 kernel: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
    Apr 13 10:52:16 kernel: FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 HTT threads
    Apr 13 10:52:16 kernel: cpu0 (BSP): APIC ID:  0
    Apr 13 10:52:16 kernel: cpu1 (AP/HT): APIC ID:  1
    Apr 13 10:52:16 kernel: ioapic0: Assuming intbase of 0
    Apr 13 10:52:16 kernel: ioapic0 <version 2.0="">irqs 0-23 on motherboard
    Apr 13 10:52:16 kernel: wlan: mac acl policy registered
    Apr 13 10:52:16 kernel: cryptosoft0: <software crypto="">on motherboard
    Apr 13 10:52:16 kernel: padlock0: No ACE support.
    Apr 13 10:52:16 kernel: pcib0: <mptable host-pci="" bridge="">pcibus 0 on motherboard
    Apr 13 10:52:16 kernel: pci0: <pci bus="">on pcib0
    Apr 13 10:52:16 kernel: pcib1: <pci-pci bridge="">irq 16 at device 23.0 on pci0
    Apr 13 10:52:16 kernel: pci1: <pci bus="">on pcib1
    Apr 13 10:52:16 kernel: pcib2: <pci-pci bridge="">irq 16 at device 0.0 on pci1
    Apr 13 10:52:16 kernel: pci2: <pci bus="">on pcib2
    Apr 13 10:52:16 kernel: pci2: <unknown>at device 0.0 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <network, ethernet="">at device 0.1 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <unknown>at device 0.2 (no driver attached)
    Apr 13 10:52:16 kernel: ohci0: <ohci (generic)="" usb="" controller="">mem 0xa0000b00-0xa0000bff irq 19 at device 2.0 on pci2
    Apr 13 10:52:16 kernel: ohci0: [ITHREAD]
    Apr 13 10:52:16 kernel: usbus0: <ohci (generic)="" usb="" controller="">on ohci0
    Apr 13 10:52:16 kernel: ohci1: <ohci (generic)="" usb="" controller="">mem 0xa0000c00-0xa0000cff irq 19 at device 2.1 on pci2
    Apr 13 10:52:16 kernel: ohci1: [ITHREAD]
    Apr 13 10:52:16 kernel: usbus1: <ohci (generic)="" usb="" controller="">on ohci1
    Apr 13 10:52:16 kernel: ohci2: <ohci (generic)="" usb="" controller="">mem 0xa0000d00-0xa0000dff irq 19 at device 2.2 on pci2
    Apr 13 10:52:16 kernel: ohci2: [ITHREAD]
    Apr 13 10:52:16 kernel: usbus2: <ohci (generic)="" usb="" controller="">on ohci2
    Apr 13 10:52:16 kernel: ehci0: <ehci (generic)="" usb="" 2.0="" controller="">mem 0xa0000e00-0xa0000eff irq 19 at device 2.3 on pci2
    Apr 13 10:52:16 kernel: ehci0: [ITHREAD]
    Apr 13 10:52:16 kernel: usbus3: EHCI version 1.0
    Apr 13 10:52:16 kernel: usbus3: <ehci (generic)="" usb="" 2.0="" controller="">on ehci0
    Apr 13 10:52:16 kernel: pci2: <serial bus,="" usb="">at device 2.4 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <base peripheral,="" sd="" host="" controller=""> at device 4.0 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <base peripheral,="" sd="" host="" controller=""> at device 4.1 (no driver attached)
    Apr 13 10:52:16 kernel: atapci0: <intel ahci="" controller="">port 0x1020-0x103f mem 0xa0004400-0xa00047ff irq 17 at device 6.0 on pci2
    Apr 13 10:52:16 kernel: atapci0: [ITHREAD]
    Apr 13 10:52:16 kernel: atapci0: AHCI v1.10 controller with 2 3Gbps ports, PM supported
    Apr 13 10:52:16 kernel: ata2: <ata channel="">at channel 0 on atapci0
    Apr 13 10:52:16 kernel: ata2: [ITHREAD]
    Apr 13 10:52:16 kernel: ata3: <ata channel="">at channel 1 on atapci0
    Apr 13 10:52:16 kernel: ata3: [ITHREAD]
    Apr 13 10:52:16 kernel: ohci3: <ohci (generic)="" usb="" controller="">mem 0xa0004800-0xa00048ff irq 16 at device 8.0 on pci2
    Apr 13 10:52:16 kernel: ohci3: [ITHREAD]
    Apr 13 10:52:16 kernel: usbus4: <ohci (generic)="" usb="" controller="">on ohci3
    Apr 13 10:52:16 kernel: ohci4: <ohci (generic)="" usb="" controller="">mem 0xa0004900-0xa00049ff irq 16 at device 8.1 on pci2
    Apr 13 10:52:16 kernel: ohci4: [ITHREAD]
    Apr 13 10:52:16 kernel: usbus5: <ohci (generic)="" usb="" controller="">on ohci4
    Apr 13 10:52:16 kernel: ohci5: <ohci (generic)="" usb="" controller="">mem 0xa0004a00-0xa0004aff irq 16 at device 8.2 on pci2
    Apr 13 10:52:16 kernel: ohci5: [ITHREAD]
    Apr 13 10:52:16 kernel: usbus6: <ohci (generic)="" usb="" controller="">on ohci5
    Apr 13 10:52:16 kernel: ehci1: <ehci (generic)="" usb="" 2.0="" controller="">mem 0xa0004b00-0xa0004bff irq 16 at device 8.3 on pci2
    Apr 13 10:52:16 kernel: ehci1: [ITHREAD]
    Apr 13 10:52:16 kernel: usbus7: EHCI version 1.0
    Apr 13 10:52:16 kernel: usbus7: <ehci (generic)="" usb="" 2.0="" controller="">on ehci1
    Apr 13 10:52:16 kernel: pci2: <unknown>at device 10.0 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <simple comms,="" uart="">at device 10.1 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <simple comms,="" uart="">at device 10.2 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <simple comms,="" uart="">at device 10.3 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <simple comms,="" uart="">at device 10.4 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <unknown>at device 12.0 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <serial bus="">at device 12.1 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <serial bus="">at device 12.2 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <serial bus="">at device 12.3 (no driver attached)
    Apr 13 10:52:16 kernel: pci2: <unknown>at device 12.4 (no driver attached)
    Apr 13 10:52:16 kernel: pcib3: <pci-pci bridge="">irq 17 at device 24.0 on pci0
    Apr 13 10:52:16 kernel: pci3: <pci bus="">on pcib3
    Apr 13 10:52:16 kernel: pcib4: <pci-pci bridge="">at device 0.0 on pci3
    Apr 13 10:52:16 kernel: pci4: <pci bus="">on pcib4
    Apr 13 10:52:16 kernel: pcib5: <pci-pci bridge="">at device 2.0 on pci4
    Apr 13 10:52:16 kernel: pci5: <pci bus="">on pcib5
    Apr 13 10:52:16 kernel: em0: <intel(r) 1000="" pro="" network="" connection="" 7.3.8="">port 0x2000-0x201f mem 0xa1000000-0xa101ffff,0xa1020000-0xa1023fff irq 19 at device 0.0 on pci5
    Apr 13 10:52:16 kernel: em0: Using MSIX interrupts with 3 vectors
    Apr 13 10:52:16 kernel: em0: [ITHREAD]
    Apr 13 10:52:16 kernel: em0: [ITHREAD]
    Apr 13 10:52:16 kernel: em0: [ITHREAD]
    Apr 13 10:52:16 kernel: pcib6: <pci-pci bridge="">at device 3.0 on pci4
    Apr 13 10:52:16 kernel: pci6: <pci bus="">on pcib6
    Apr 13 10:52:16 kernel: em1: <intel(r) 1000="" pro="" network="" connection="" 7.3.8="">port 0x3000-0x301f mem 0xa2000000-0xa201ffff,0xa2020000-0xa2023fff irq 16 at device 0.0 on pci6
    Apr 13 10:52:16 kernel: em1: Using MSIX interrupts with 3 vectors
    Apr 13 10:52:16 kernel: em1: [ITHREAD]
    Apr 13 10:52:16 kernel: em1: [ITHREAD]
    Apr 13 10:52:16 kernel: em1: [ITHREAD]
    Apr 13 10:52:16 kernel: pcib7: <pci-pci bridge="">at device 4.0 on pci4
    Apr 13 10:52:16 kernel: pci7: <pci bus="">on pcib7
    Apr 13 10:52:16 kernel: pcib8: <pci-pci bridge="">irq 18 at device 25.0 on pci0
    Apr 13 10:52:16 kernel: pci8: <pci bus="">on pcib8
    Apr 13 10:52:16 kernel: pcib9: <pci-pci bridge="">at device 0.0 on pci8
    Apr 13 10:52:16 kernel: pci9: <pci bus="">on pcib9
    Apr 13 10:52:16 kernel: pcib10: <pci-pci bridge="">at device 2.0 on pci9
    Apr 13 10:52:16 kernel: pci10: <pci bus="">on pcib10
    Apr 13 10:52:16 kernel: em2: <intel(r) 1000="" pro="" network="" connection="" 7.3.8="">port 0x4000-0x401f mem 0xa3000000-0xa301ffff,0xa3020000-0xa3023fff irq 16 at device 0.0 on pci10
    Apr 13 10:52:16 kernel: em2: Using MSIX interrupts with 3 vectors
    Apr 13 10:52:16 kernel: em2: [ITHREAD]
    Apr 13 10:52:16 kernel: em2: [ITHREAD]
    Apr 13 10:52:16 kernel: em2: [ITHREAD]
    Apr 13 10:52:16 kernel: pcib11: <pci-pci bridge="">at device 3.0 on pci9
    Apr 13 10:52:16 kernel: pci11: <pci bus="">on pcib11
    Apr 13 10:52:16 kernel: em3: <intel(r) 1000="" pro="" network="" connection="" 7.3.8="">port 0x5000-0x501f mem 0xa4000000-0xa401ffff,0xa4020000-0xa4023fff irq 17 at device 0.0 on pci11
    Apr 13 10:52:16 kernel: em3: Using MSIX interrupts with 3 vectors
    Apr 13 10:52:16 kernel: em3: [ITHREAD]
    Apr 13 10:52:16 kernel: em3: [ITHREAD]
    Apr 13 10:52:16 kernel: em3: [ITHREAD]
    Apr 13 10:52:16 kernel: pcib12: <pci-pci bridge="">at device 4.0 on pci9
    Apr 13 10:52:16 kernel: pci12: <pci bus="">on pcib12
    Apr 13 10:52:16 kernel: pcib13: <pci-pci bridge="">irq 19 at device 26.0 on pci0
    Apr 13 10:52:16 kernel: pci13: <pci bus="">on pcib13
    Apr 13 10:52:16 kernel: pcib14: <pci-pci bridge="">at device 0.0 on pci13
    Apr 13 10:52:16 kernel: pci14: <pci bus="">on pcib14
    Apr 13 10:52:16 kernel: pcib15: <pci-pci bridge="">at device 2.0 on pci14
    Apr 13 10:52:16 kernel: pci15: <pci bus="">on pcib15
    Apr 13 10:52:16 kernel: em4: <intel(r) 1000="" pro="" network="" connection="" 7.3.8="">port 0x6000-0x601f mem 0xa5000000-0xa501ffff,0xa5020000-0xa5023fff irq 17 at device 0.0 on pci15
    Apr 13 10:52:16 kernel: em4: Using MSIX interrupts with 3 vectors
    Apr 13 10:52:16 kernel: em4: [ITHREAD]
    Apr 13 10:52:16 kernel: em4: [ITHREAD]
    Apr 13 10:52:16 kernel: em4: [ITHREAD]
    Apr 13 10:52:16 kernel: pcib16: <pci-pci bridge="">at device 3.0 on pci14
    Apr 13 10:52:16 kernel: pci16: <pci bus="">on pcib16
    Apr 13 10:52:16 kernel: em5: <intel(r) 1000="" pro="" network="" connection="" 7.3.8="">port 0x7000-0x701f mem 0xa6000000-0xa601ffff,0xa6020000-0xa6023fff irq 18 at device 0.0 on pci16
    Apr 13 10:52:16 kernel: em5: Using MSIX interrupts with 3 vectors
    Apr 13 10:52:16 kernel: em5: [ITHREAD]
    Apr 13 10:52:16 kernel: em5: [ITHREAD]
    Apr 13 10:52:16 kernel: em5: [ITHREAD]
    Apr 13 10:52:16 kernel: pcib17: <pci-pci bridge="">at device 4.0 on pci14
    Apr 13 10:52:16 kernel: pci17: <pci bus="">on pcib17
    Apr 13 10:52:16 kernel: em6: <intel(r) 1000="" pro="" network="" connection="" 7.3.8="">port 0x8000-0x801f mem 0xa7000000-0xa701ffff,0xa7020000-0xa7023fff irq 19 at device 0.0 on pci17
    Apr 13 10:52:16 kernel: em6: Using MSIX interrupts with 3 vectors
    Apr 13 10:52:16 kernel: em6: [ITHREAD]
    Apr 13 10:52:16 kernel: em6: [ITHREAD]
    Apr 13 10:52:16 kernel: em6: [ITHREAD]
    Apr 13 10:52:16 kernel: pcib18: <pci-pci bridge="">at device 5.0 on pci14
    Apr 13 10:52:16 kernel: pci18: <pci bus="">on pcib18
    Apr 13 10:52:16 kernel: em7: <intel(r) 1000="" pro="" network="" connection="" 7.3.8="">port 0x9000-0x901f mem 0xa8000000-0xa801ffff,0xa8020000-0xa8023fff irq 16 at device 0.0 on pci18
    Apr 13 10:52:16 kernel: em7: Using MSIX interrupts with 3 vectors
    Apr 13 10:52:16 kernel: em7: [ITHREAD]
    Apr 13 10:52:16 kernel: em7: [ITHREAD]
    Apr 13 10:52:16 kernel: em7: [ITHREAD]
    Apr 13 10:52:16 kernel: isab0: <pci-isa bridge="">at device 31.0 on pci0
    Apr 13 10:52:16 kernel: isa0: <isa bus="">on isab0
    Apr 13 10:52:16 kernel: cpu0 on motherboard
    Apr 13 10:52:16 kernel: cpu1 on motherboard
    Apr 13 10:52:16 kernel: ata0: <ata channel="">at port 0x1f0-0x1f7,0x3f6 irq 14 on isa0
    Apr 13 10:52:16 kernel: ata0: [ITHREAD]
    Apr 13 10:52:16 kernel: ata1: <ata channel="">at port 0x170-0x177,0x376 irq 15 on isa0
    Apr 13 10:52:16 kernel: ata1: [ITHREAD]
    Apr 13 10:52:16 kernel: atrtc0: <at real="" time="" clock="">at port 0x70 irq 8 on isa0
    Apr 13 10:52:16 kernel: ppc0: parallel port not found.
    Apr 13 10:52:16 kernel: uart0: <16550 or compatible> at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
    Apr 13 10:52:16 kernel: uart0: [FILTER]
    Apr 13 10:52:16 kernel: uart0: console (115200,n,8,1)
    Apr 13 10:52:16 kernel: Timecounters tick every 10.000 msec
    Apr 13 10:52:16 kernel: IPsec: Initialized Security Association Processing.
    Apr 13 10:52:16 kernel: usbus0: 12Mbps Full Speed USB v1.0
    Apr 13 10:52:16 kernel: usbus1: 12Mbps Full Speed USB v1.0
    Apr 13 10:52:16 kernel: usbus2: 12Mbps Full Speed USB v1.0
    Apr 13 10:52:16 kernel: usbus3: 480Mbps High Speed USB v2.0
    Apr 13 10:52:16 kernel: usbus4: 12Mbps Full Speed USB v1.0
    Apr 13 10:52:16 kernel: usbus5: 12Mbps Full Speed USB v1.0
    Apr 13 10:52:16 kernel: ugen0.1: <0x8086> at usbus0
    Apr 13 10:52:16 kernel: uhub0: <0x8086 OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
    Apr 13 10:52:16 kernel: ugen1.1: <0x8086> at usbus1
    Apr 13 10:52:16 kernel: uhub1: <0x8086 OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
    Apr 13 10:52:16 kernel: ugen2.1: <0x8086> at usbus2
    Apr 13 10:52:16 kernel: uhub2: <0x8086 OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
    Apr 13 10:52:16 kernel: ugen3.1: <intel>at usbus3
    Apr 13 10:52:16 kernel: uhub3: <intel 1="" 9="" ehci="" root="" hub,="" class="" 0,="" rev="" 2.00="" 1.00,="" addr="">on usbus3
    Apr 13 10:52:16 kernel: ugen4.1: <0x8086> at usbus4
    Apr 13 10:52:16 kernel: uhub4: <0x8086 OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
    Apr 13 10:52:16 kernel: ugen5.1: <0x8086> at usbus5
    Apr 13 10:52:16 kernel: uhub5: <0x8086 OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
    Apr 13 10:52:16 kernel: usbus6: 12Mbps Full Speed USB v1.0
    Apr 13 10:52:16 kernel: usbus7: 480Mbps High Speed USB v2.0
    Apr 13 10:52:16 kernel: ad4: 3775MB <ts4gmsa500 20120131="">at ata2-master UDMA100 SATA 3Gb/s
    Apr 13 10:52:16 kernel: ugen6.1: <0x8086> at usbus6
    Apr 13 10:52:16 kernel: uhub6: <0x8086 OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
    Apr 13 10:52:16 kernel: ugen7.1: <intel>at usbus7
    Apr 13 10:52:16 kernel: uhub7: <intel 1="" 9="" ehci="" root="" hub,="" class="" 0,="" rev="" 2.00="" 1.00,="" addr="">on usbus7
    Apr 13 10:52:16 kernel: uhub0: 1 port with 1 removable, self powered
    Apr 13 10:52:16 kernel: uhub1: 1 port with 1 removable, self powered
    Apr 13 10:52:16 kernel: uhub2: 1 port with 1 removable, self powered
    Apr 13 10:52:16 kernel: uhub4: 1 port with 1 removable, self powered
    Apr 13 10:52:16 kernel: uhub5: 1 port with 1 removable, self powered
    Apr 13 10:52:16 kernel: uhub6: 1 port with 1 removable, self powered
    Apr 13 10:52:16 kernel: uhub3: 3 ports with 3 removable, self powered
    Apr 13 10:52:16 kernel: uhub7: 3 ports with 3 removable, self powered
    Apr 13 10:52:16 kernel: SMP: AP CPU #1 Launched!
    Apr 13 10:52:16 kernel: Trying to mount root from ufs:/dev/ad4s1a
    Apr 13 10:52:16 kernel: ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable,
    Apr 13 10:52:16 kernel: add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
    Apr 13 10:52:16 kernel: ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable behavior.
    Apr 13 10:52:16 kernel: Consider tuning vm.kmem_size and vm.kmem_size_max
    Apr 13 10:52:16 kernel: in /boot/loader.conf.
    Apr 13 10:52:16 kernel: ZFS filesystem version 5
    Apr 13 10:52:16 kernel: ZFS storage pool version 28
    Apr 13 10:52:16 kernel: coretemp0: <cpu on-die="" thermal="" sensors="">on cpu0
    Apr 13 10:52:16 kernel: coretemp0: Can not get Tj(target) from your CPU, using 100C.
    Apr 13 10:52:16 kernel: coretemp1: <cpu on-die="" thermal="" sensors="">on cpu1
    Apr 13 10:52:16 kernel: coretemp1: Can not get Tj(target) from your CPU, using 100C.
    Apr 13 10:52:18 check_reload_status: Linkup starting em0
    Apr 13 10:52:18 kernel: em0: link state changed to UP
    Apr 13 10:52:19 check_reload_status: rc.newwanip starting em0
    Apr 13 10:52:20 kernel: em1: promiscuous mode enabled
    Apr 13 10:52:20 kernel: em2: promiscuous mode enabled
    Apr 13 10:52:20 kernel: em3: promiscuous mode enabled
    Apr 13 10:52:20 kernel: em4: promiscuous mode enabled
    Apr 13 10:52:20 kernel: em5: promiscuous mode enabled
    Apr 13 10:52:20 kernel: em6: promiscuous mode enabled
    Apr 13 10:52:20 kernel: em7: promiscuous mode enabled
    Apr 13 10:52:20 php: rc.bootup: Resyncing OpenVPN instances.
    Apr 13 10:52:21 kernel: tun1: changing name to 'ovpns1'
    Apr 13 10:52:22 check_reload_status: Linkup starting em1
    Apr 13 10:52:22 kernel: em1: link state changed to UP
    Apr 13 10:52:22 check_reload_status: Linkup starting em5
    Apr 13 10:52:22 kernel: em5: link state changed to UP
    Apr 13 10:52:22 php: rc.newwanip: rc.newwanip: Informational is starting em0.
    Apr 13 10:52:22 php: rc.newwanip: rc.newwanip: on (IP address: x.y.z.205) (interface: wan) (real interface: em0).
    Apr 13 10:52:23 php: rc.newwanip: ROUTING: setting default route to x.y.z.254
    Apr 13 10:52:23 kernel: pflog0: promiscuous mode enabled
    Apr 13 10:52:23 kernel: ovpns1: link state changed to UP
    Apr 13 10:52:23 check_reload_status: rc.newwanip starting ovpns1
    Apr 13 10:52:23 check_reload_status: Linkup starting em2
    Apr 13 10:52:23 kernel: em2: link state changed to UP
    Apr 13 10:52:23 kernel: em6: link state changed to UP
    Apr 13 10:52:23 check_reload_status: Linkup starting em6
    Apr 13 10:52:26 php: rc.newwanip: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Apr 13 10:52:28 php: rc.newwanip: rc.newwanip: Informational is starting ovpns1.
    Apr 13 10:52:28 php: rc.newwanip: rc.newwanip: on (IP address: 10.y.y.1) (interface: ) (real interface: ovpns1).
    Apr 13 10:52:28 check_reload_status: Reloading filter
    Apr 13 10:52:28 php: rc.newwanip: package system has detected an ip change  ->  10.y.y.1 ... Restarting packages.
    Apr 13 10:52:28 check_reload_status: Starting packages
    Apr 13 10:52:30 php: rc.bootup: ROUTING: setting default route to x.y.z.254
    Apr 13 10:52:32 php: rc.start_packages: Restarting/Starting all packages.
    Apr 13 10:52:33 php: rc.newwanip: Resyncing OpenVPN instances for interface WAN.
    Apr 13 10:52:33 kernel: ovpns1: link state changed to DOWN
    Apr 13 10:52:33 check_reload_status: Reloading filter
    Apr 13 10:52:34 check_reload_status: Updating all dyndns
    Apr 13 10:52:35 php: rc.newwanip: Creating rrd update script
    Apr 13 10:52:35 kernel: ovpns1: link state changed to UP
    Apr 13 10:52:35 check_reload_status: rc.newwanip starting ovpns1
    Apr 13 10:52:37 php: rc.newwanip: package system has detected an ip change x.y.z.205 ->  x.y.z.205 ... Restarting packages.
    Apr 13 10:52:37 check_reload_status: Starting packages
    Apr 13 10:52:40 php: rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Apr 13 10:52:40 php: rc.newwanip: rc.newwanip: Informational is starting ovpns1.
    Apr 13 10:52:40 php: rc.newwanip: rc.newwanip: on (IP address: 10.y.y.1) (interface: ) (real interface: ovpns1).
    Apr 13 10:52:40 php: rc.newwanip: package system has detected an ip change  ->  10.y.y.1 ... Restarting packages.
    Apr 13 10:52:42 php: rc.start_packages: Restarting/Starting all packages.
    Apr 13 10:52:44 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:44 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:44 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:44 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:46 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:46 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:47 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:47 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:47 php: /diag_logs.php: Successful login for user '' from: 10.x.x.12
    Apr 13 10:52:47 php: /diag_logs.php: Successful login for user '' from: 10.x.x.12
    Apr 13 10:52:47 php: rc.start_packages: Restarting/Starting all packages.
    Apr 13 10:52:47 sshlockout[8176]: sshlockout/webConfigurator v3.0 starting up
    Apr 13 10:52:50 php: rc.bootup: Creating rrd update script
    Apr 13 10:52:51 php: rc.bootup: The command '/usr/sbin/powerd -b hadp -a hadp' returned exit code '1', the output was 'powerd: lookup freq: No such file or directory' 
    Apr 13 10:52:51 syslogd: exiting on signal 15
    Apr 13 10:52:51 syslogd: kernel boot file is /boot/kernel/kernel
    Apr 13 10:52:52 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:52 php: rc.start_packages: Restarting/Starting all packages.
    Apr 13 10:52:53 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:53 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:53 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:57 kernel: bridge0: promiscuous mode enabled
    Apr 13 10:52:57 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:57 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:57 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:52:57 php: rc.start_packages: No pfBlocker action during boot process.
    Apr 13 10:53:00 kernel: bridge0: promiscuous mode disabled
    Apr 13 10:53:01 check_reload_status: Syncing firewall
    Apr 13 10:53:03 check_reload_status: Syncing firewall
    Apr 13 10:53:04 kernel: bridge0: promiscuous mode enabled
    Apr 13 10:53:05 kernel: bridge0: promiscuous mode disabled
    Apr 13 10:53:07 kernel: bridge0: promiscuous mode enabled
    Apr 13 10:53:12 kernel: bridge0: promiscuous mode disabled
    Apr 13 10:53:16 kernel: bridge0: promiscuous mode enabled
    Apr 13 10:53:18 SnortStartup[26522]: Snort START for WAN(28432_em0)...
    Apr 13 10:53:25 kernel: em0: promiscuous mode enabled
    Apr 13 10:53:28 SnortStartup[32287]: Snort START for LAN(12814_bridge0)...
    Apr 13 10:53:34 SnortStartup[33557]: Snort STOP for WAN(28432_em0)...
    Apr 13 10:53:35 snort[31343]: *** Caught Term-Signal
    Apr 13 10:53:35 kernel: em0: promiscuous mode disabled
    Apr 13 10:53:38 SnortStartup[35439]: Snort STOP for LAN(12814_bridge0)...
    Apr 13 10:53:43 SnortStartup[39298]: Snort START for WAN(28432_em0)...
    Apr 13 10:53:43 SnortStartup[41691]: Snort STOP for WAN(28432_em0)...
    Apr 13 10:53:44 SnortStartup[44881]: Snort START for WAN(28432_em0)...
    Apr 13 10:53:45 login: login on console as root
    Apr 13 10:53:45 sshlockout[45979]: sshlockout/webConfigurator v3.0 starting up
    Apr 13 10:53:45 SnortStartup[48337]: Snort START for LAN(12814_bridge0)...
    Apr 13 10:53:46 SnortStartup[49564]: Snort STOP for LAN(12814_bridge0)...
    Apr 13 10:53:47 kernel: em0: promiscuous mode enabled
    Apr 13 10:53:49 SnortStartup[52956]: Snort START for LAN(12814_bridge0)...
    Apr 13 10:53:50 SnortStartup[54295]: Snort SOFT RESTART for WAN(28432_em0)...
    Apr 13 10:53:52 SnortStartup[56249]: Snort SOFT RESTART for LAN(12814_bridge0)...
    Apr 13 10:55:34 php: /snort/snort_interfaces_suppress.php: Successful login for user '' from: 10.x.x.11
    Apr 13 10:55:34 php: /snort/snort_interfaces_suppress.php: Successful login for user '' from: 10.x.x.11
    Apr 13 10:59:59 check_reload_status: Linkup starting em1
    Apr 13 10:59:59 kernel: em1: link state changed to DOWN
    Apr 13 11:00:02 check_reload_status: Linkup starting em1
    Apr 13 11:00:02 kernel: em1: link state changed to UP
    Apr 13 11:00:03 php: rc.linkup: Hotplug event detected for OPT1(opt1) but ignoring since interface is configured with static IP ( )
    Apr 13 11:00:05 php: rc.linkup: Hotplug event detected for OPT1(opt1) but ignoring since interface is configured with static IP ( )
    Apr 13 11:00:05 check_reload_status: rc.newwanip starting em1
    Apr 13 11:00:08 php: rc.newwanip: rc.newwanip: Informational is starting em1.
    Apr 13 11:00:08 php: rc.newwanip: Interface does not have an IP address, nothing to do.
    Apr 13 11:01:41 check_reload_status: Linkup starting em2
    Apr 13 11:01:41 kernel: em2: link state changed to DOWN
    Apr 13 11:01:44 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 11:02:23 check_reload_status: Linkup starting em1
    Apr 13 11:02:23 kernel: em1: link state changed to DOWN
    Apr 13 11:02:25 check_reload_status: Linkup starting em1
    Apr 13 11:02:25 kernel: em1: link state changed to UP
    Apr 13 11:02:25 kernel: em2: link state changed to UP
    Apr 13 11:02:25 check_reload_status: Linkup starting em2
    Apr 13 11:02:26 php: rc.linkup: Hotplug event detected for OPT1(opt1) but ignoring since interface is configured with static IP ( )
    Apr 13 11:02:29 kernel: em2: link state changed to DOWN
    Apr 13 11:02:28 check_reload_status: Linkup starting em2
    Apr 13 11:02:29 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 11:02:29 php: rc.linkup: Hotplug event detected for OPT1(opt1) but ignoring since interface is configured with static IP ( )
    Apr 13 11:02:29 check_reload_status: rc.newwanip starting em1
    Apr 13 11:02:29 check_reload_status: rc.newwanip starting em2
    Apr 13 11:02:31 kernel: em2: link state changed to UP
    Apr 13 11:02:31 check_reload_status: Linkup starting em2
    Apr 13 11:02:32 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 11:02:33 php: rc.newwanip: rc.newwanip: Informational is starting em2.
    Apr 13 11:02:33 php: rc.newwanip: Interface does not have an IP address, nothing to do.
    Apr 13 11:02:33 php: rc.newwanip: rc.newwanip: Informational is starting em1.
    Apr 13 11:02:33 php: rc.newwanip: Interface does not have an IP address, nothing to do.
    Apr 13 11:02:34 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 11:02:34 check_reload_status: rc.newwanip starting em2
    Apr 13 11:02:37 php: rc.newwanip: rc.newwanip: Informational is starting em2.
    Apr 13 11:02:37 php: rc.newwanip: Interface does not have an IP address, nothing to do.
    Apr 13 11:02:45 check_reload_status: Linkup starting em2
    Apr 13 11:02:45 kernel: em2: link state changed to DOWN
    Apr 13 11:02:48 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 11:02:48 check_reload_status: Linkup starting em2
    Apr 13 11:02:48 kernel: em2: link state changed to UP
    Apr 13 11:02:51 php: rc.linkup: Hotplug event detected for OPT2(opt2) but ignoring since interface is configured with static IP ( )
    Apr 13 11:02:51 check_reload_status: rc.newwanip starting em2
    Apr 13 11:02:54 php: rc.newwanip: rc.newwanip: Informational is starting em2.
    Apr 13 11:02:54 php: rc.newwanip: Interface does not have an IP address, nothing to do.</cpu></cpu></intel></intel></ts4gmsa500></intel></intel></at></ata></ata></isa></pci-isa></intel(r)></pci></pci-pci></intel(r)></pci></pci-pci></intel(r)></pci></pci-pci></intel(r)></pci></pci-pci></pci></pci-pci></pci></pci-pci></pci></pci-pci></intel(r)></pci></pci-pci></intel(r)></pci></pci-pci></pci></pci-pci></pci></pci-pci></pci></pci-pci></intel(r)></pci></pci-pci></intel(r)></pci></pci-pci></pci></pci-pci></pci></pci-pci></unknown></serial></serial></serial></unknown></simple></simple></simple></simple></unknown></ehci></ehci></ohci></ohci></ohci></ohci></ohci></ohci></ata></ata></intel></serial></ehci></ehci></ohci></ohci></ohci></ohci></ohci></ohci></unknown></network,></unknown></pci></pci-pci></pci></pci-pci></pci></mptable></software></version></soekris ></lahf></nx,lm></sse3,dtes64,mon,ds_cpl,vmx,est,tm2,ssse3,cx16,xtpr,pdcm,movbe></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe> 
    

    Here is the DHCP log, my desktop computer is 11.x.x.11 in this.

    
    Apr 13 10:32:07 dhcpd: DHCPDISCOVER from 11:11:11:11:11:11 via bridge0
    Apr 13 10:32:07 dhcpd: DHCPOFFER on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:32:11 dhcpd: DHCPDISCOVER from 11:11:11:11:11:11 via bridge0
    Apr 13 10:32:11 dhcpd: DHCPOFFER on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:32:18 dhcpd: DHCPDISCOVER from 11:11:11:11:11:11 via bridge0
    Apr 13 10:32:18 dhcpd: DHCPOFFER on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:32:35 dhcpd: DHCPDISCOVER from 11:11:11:11:11:11 via bridge0
    Apr 13 10:32:35 dhcpd: DHCPOFFER on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:33:06 dhcpd: DHCPDISCOVER from 11:11:11:11:11:11 via bridge0
    Apr 13 10:33:06 dhcpd: DHCPOFFER on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:33:11 dhcpd: DHCPDISCOVER from 11:11:11:11:11:11 via bridge0
    Apr 13 10:33:11 dhcpd: DHCPOFFER on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:33:18 dhcpd: DHCPDISCOVER from 11:11:11:11:11:11 via bridge0
    Apr 13 10:33:18 dhcpd: DHCPOFFER on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:33:32 dhcpd: DHCPINFORM from 10.x.x.12 via bridge0
    Apr 13 10:33:32 dhcpd: DHCPACK to 10.x.x.12 (12:12:12:12:12:12) via bridge0
    Apr 13 10:33:34 dhcpd: DHCPDISCOVER from 11:11:11:11:11:11 via bridge0
    Apr 13 10:33:34 dhcpd: DHCPOFFER on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:36:23 dhcpd: DHCPINFORM from 10.x.x.12 via bridge0
    Apr 13 10:36:23 dhcpd: DHCPACK to 10.x.x.12 (12:12:12:12:12:12) via bridge0
    Apr 13 10:37:33 dhcpd: DHCPINFORM from 10.x.x.12 via bridge0
    Apr 13 10:37:33 dhcpd: DHCPACK to 10.x.x.12 (12:12:12:12:12:12) via bridge0
    Apr 13 10:38:43 dhcpd: DHCPINFORM from 10.x.x.12 via bridge0
    Apr 13 10:38:43 dhcpd: DHCPACK to 10.x.x.12 (12:12:12:12:12:12) via bridge0
    Apr 13 10:52:18 dhclient: PREINIT
    Apr 13 10:52:18 dhclient: Starting delete_old_states()
    Apr 13 10:52:18 dhclient: Comparing IPs: Old:  New: 
    Apr 13 10:52:18 dhclient[8790]: DHCPREQUEST on em0 to 255.255.255.255 port 67
    Apr 13 10:52:19 dhclient[8790]: DHCPACK from a.b.c.238
    Apr 13 10:52:19 dhclient: REBOOT
    Apr 13 10:52:19 dhclient: Starting delete_old_states()
    Apr 13 10:52:19 dhclient: Comparing IPs: Old:  New: x.y.z.205
    Apr 13 10:52:19 dhclient: Starting add_new_address()
    Apr 13 10:52:19 dhclient: ifconfig em0 inet x.y.z.205 netmask 255.255.255.192 broadcast x.y.z.255 
    Apr 13 10:52:19 dhclient: New IP Address (em0): x.y.z.205
    Apr 13 10:52:19 dhclient: New Subnet Mask (em0): 255.255.255.192
    Apr 13 10:52:19 dhclient: New Broadcast Address (em0): x.y.z.255
    Apr 13 10:52:19 dhclient: New Routers (em0): x.y.z.254
    Apr 13 10:52:19 dhclient: Adding new routes to interface: em0
    Apr 13 10:52:19 dhclient: /sbin/route add default x.y.z.254
    Apr 13 10:52:19 dhclient: Creating resolv.conf
    Apr 13 10:52:19 dhclient[8790]: bound to x.y.z.205 -- renewal in 3600 seconds.
    Apr 13 10:52:33 dhcpd: Internet Systems Consortium DHCP Server 4.2.6
    Apr 13 10:52:33 dhcpd: Copyright 2004-2014 Internet Systems Consortium.
    Apr 13 10:52:33 dhcpd: All rights reserved.
    Apr 13 10:52:33 dhcpd: For info, please visit https://www.isc.org/software/dhcp/
    Apr 13 10:52:33 dhcpd: Internet Systems Consortium DHCP Server 4.2.6
    Apr 13 10:52:33 dhcpd: Copyright 2004-2014 Internet Systems Consortium.
    Apr 13 10:52:33 dhcpd: All rights reserved.
    Apr 13 10:52:33 dhcpd: For info, please visit https://www.isc.org/software/dhcp/
    Apr 13 10:52:33 dhcpd: Wrote 0 deleted host decls to leases file.
    Apr 13 10:52:33 dhcpd: Wrote 0 new dynamic host decls to leases file.
    Apr 13 10:52:33 dhcpd: Wrote 6 leases to leases file.
    Apr 13 10:52:33 dhcpd: Listening on BPF/bridge0/01:01:01:01:01:01/10.x.x.0/24
    Apr 13 10:52:33 dhcpd: Sending on   BPF/bridge0/01:01:01:01:01:01/10.x.x.0/24
    Apr 13 10:52:33 dhcpd: Sending on   Socket/fallback/fallback-net
    Apr 13 10:52:33 dhcpd: DHCPDISCOVER from 14:14:14:14:14:14 via bridge0
    Apr 13 10:52:33 dhcpd: DHCPOFFER on 10.x.x.14 to 14:14:14:14:14:14 via bridge0
    Apr 13 10:52:33 dhcpd: DHCPREQUEST for 10.x.x.14 (10.x.x.2) from 14:14:14:14:14:14 via bridge0
    Apr 13 10:52:33 dhcpd: DHCPACK on 10.x.x.14 to 14:14:14:14:14:14 via bridge0
    Apr 13 10:52:37 dhcpd: DHCPREQUEST for 10.x.x.12 from 12:12:12:12:12:12 via bridge0
    Apr 13 10:52:37 dhcpd: DHCPACK on 10.x.x.12 to 12:12:12:12:12:12 via bridge0
    Apr 13 10:55:13 dhcpd: DHCPDISCOVER from 11:11:11:11:11:11 via bridge0
    Apr 13 10:55:13 dhcpd: DHCPOFFER on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:55:13 dhcpd: DHCPDISCOVER from 11:11:11:11:11:11 via bridge0
    Apr 13 10:55:13 dhcpd: DHCPOFFER on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:55:13 dhcpd: DHCPREQUEST for 10.x.x.11 (10.x.x.2) from 11:11:11:11:11:11 via bridge0
    Apr 13 10:55:13 dhcpd: DHCPACK on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    Apr 13 10:55:19 dhcpd: DHCPINFORM from 10.x.x.11 via bridge0
    Apr 13 10:55:19 dhcpd: DHCPACK to 10.x.x.11 (11:11:11:11:11:11) via bridge0
    Apr 13 10:56:43 dhcpd: DHCPINFORM from 10.x.x.11 via bridge0
    Apr 13 10:56:43 dhcpd: DHCPACK to 10.x.x.11 (11:11:11:11:11:11) via bridge0
    Apr 13 11:02:51 dhcpd: DHCPREQUEST for 10.x.x.11 from 11:11:11:11:11:11 via bridge0
    Apr 13 11:02:51 dhcpd: DHCPACK on 10.x.x.11 to 11:11:11:11:11:11 via bridge0
    
    

    IfConfig:

    $ ifconfig
    em0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
    	options=4209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso>ether e0:e0:e0:e0:e0:e0
    	inet x.y.z.205 netmask 0xffffffc0 broadcast x.y.z.255
    	inet6 a::b:c:d:3b08%em0 prefixlen 64 scopeid 0x1 
    	nd6 options=1 <performnud>media: Ethernet autoselect (100baseTX <full-duplex>)
    	status: active
    em1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
    	options=4209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso>ether e1:e1:e1:e1:e1:e1
    	inet6 a::b:c:d:3b09%em1 prefixlen 64 scopeid 0x2 
    	nd6 options=1 <performnud>media: Ethernet autoselect (100baseTX <full-duplex>)
    	status: active
    em2: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
    	options=4209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso>ether e2:e2:e2:e2:e2:e2
    	inet6 a::b:c:d:3b0a%em2 prefixlen 64 scopeid 0x3 
    	nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
    	status: active
    em3: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
    	options=4209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso>ether e3:e3:e3:e3:e3:e3
    	inet6 a::b:c:d:3b0b%em3 prefixlen 64 scopeid 0x4 
    	nd6 options=1 <performnud>media: Ethernet autoselect
    	status: no carrier
    em4: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
    	options=4209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso>ether e4:e4:e4:e4:e4:e4
    	inet6 a::b:c:d:5540%em4 prefixlen 64 scopeid 0x5 
    	nd6 options=1 <performnud>media: Ethernet autoselect
    	status: no carrier
    em5: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
    	options=4209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso>ether e5:e5:e5:e5:e5:e5
    	inet6 a::b:c:d:5541%em5 prefixlen 64 scopeid 0x6 
    	nd6 options=1 <performnud>media: Ethernet autoselect (100baseTX <full-duplex>)
    	status: active
    em6: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
    	options=4209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso>ether e6:e6:e6:e6:e6:e6
    	inet6 a::b:c:d:5542%em6 prefixlen 64 scopeid 0x7 
    	nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
    	status: active
    em7: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
    	options=4209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso>ether e7:e7:e7:e7:e7:e7
    	inet6 a::b:c:d:5543%em7 prefixlen 64 scopeid 0x8 
    	nd6 options=1 <performnud>media: Ethernet autoselect
    	status: no carrier
    enc0: flags=0<> metric 0 mtu 1536
    pflog0: flags=100 <promisc>metric 0 mtu 33192
    pfsync0: flags=0<> metric 0 mtu 1460
    	syncpeer: 224.0.0.240 maxupd: 128 syncok: 1
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
    	options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000 
    	inet6 ::1 prefixlen 128 
    	inet6 fe80::1%lo0 prefixlen 64 scopeid 0xc 
    	nd6 options=3 <performnud,accept_rtadv>bridge0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>metric 0 mtu 1500
    	ether 01:01:01:01:01:01
    	inet 10.x.x.2 netmask 0xffffff00 broadcast 10.x.x.255
    	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    	maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
    	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    	member: em7 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 8 priority 128 path cost 2000000
    	member: em6 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 7 priority 128 path cost 2000000
    	member: em5 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 6 priority 128 path cost 2000000
    	member: em4 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 5 priority 128 path cost 2000000
    	member: em3 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 4 priority 128 path cost 2000000
    	member: em2 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 3 priority 128 path cost 2000000
    	member: em1 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 2 priority 128 path cost 2000000
    ovpns1: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
    	options=80000 <linkstate>inet6 a::b:c:d:3b08%ovpns1 prefixlen 64 scopeid 0xe 
    	inet 10.y.y.1 --> 10.y.y.2 netmask 0xffffffff 
    	nd6 options=3 <performnud,accept_rtadv>Opened by PID 21747</performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast></learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></up,broadcast,running,promisc,simplex,multicast></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></promisc></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic,vlan_hwtso></up,broadcast,running,promisc,simplex,multicast>
    

    What can I do to diagnose this problem?
    The next time this problem happens, what should I look at?
    Any guess what this might be? A problem with the em driver maybe?

    Thanks,
        Erik
    ![No Connection - DHCP.gif](/public/imported_attachments/1/No Connection - DHCP.gif)
    ![No Connection - DHCP.gif_thumb](/public/imported_attachments/1/No Connection - DHCP.gif_thumb)
    ![No Connection - Static.gif](/public/imported_attachments/1/No Connection - Static.gif)
    ![No Connection - Static.gif_thumb](/public/imported_attachments/1/No Connection - Static.gif_thumb)



  • Sounds like the problem I had with my virtual instance of pfsense, I always had to reboot the guest twice before any traffic would commence. I solved this by increasing mbuf, so check your mbuf.



  • Thanks for the reply.  :)

    I don't believe I am running in to this as I increased my mbuf when I first installed pfSense, I have this in my /boot/loader.conf.local:

    kern.cam.boot_delay=10000
    hint.acpi.0.disabled=1
    legal.intel_ipw.license_ack=1
    legal.intel_wpi.license_ack=1
    kern.ipc.nmbclusters="131072"
    

    However I don't think I actually looked at mbuf the last time this happened, so I can't rule it out 100%.

    Edit:
    Looks like mbuf is fine, it's only at 10%:

    $ netstat -m
    32658/627/33285 mbufs in use (current/cache/total)
    12123/637/12760/131072 mbuf clusters in use (current/cache/total/max)
    12122/294 mbuf+clusters out of packet secondary zone in use (current/cache)
    0/44/44/65536 4k (page size) jumbo clusters in use (current/cache/total/max)
    0/0/0/32768 9k jumbo clusters in use (current/cache/total/max)
    0/0/0/16384 16k jumbo clusters in use (current/cache/total/max)
    32907K/1606K/34514K bytes allocated to network (current/cache/total)
    0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
    0/0/0 requests for jumbo clusters denied (4k/9k/16k)
    0/6/6656 sfbufs in use (current/peak/max)
    0 requests for sfbufs denied
    0 requests for sfbufs delayed
    0 requests for I/O initiated by sendfile
    0 calls to protocol drain routines
    

  • Netgate Administrator

    If you have the bridge interface set as LAN have you spoofed the MAC? This doesn't match your symptoms but if you don't spoof the MAC address it will be assigned a random one that may change betweem boots. Windows then sees that as a new DHCP server and will set it's firewall appropriately. Something to watch out for.

    My first suspect here would be Snort. Are you running it on LAN?

    Steve



  • I have spoofed the LAN bridge interface, yes.
    I do run Snort on LAN, but the configuration has not changed compared to when it was running on 2.1.0.
    The Snort version did change though I think.
    I will disable it on LAN and see if that makes a difference.

    Thanks. :)



  • Disabling Snort on the LAN side did not make a difference.

    Strangely, unchecking the Snort Interface - General Settings - "Enable or Disable" check box and saving, changes some of the settings.
    I did a config diff after I re-enabled and these items were all different:

    • blockoffenders7

    • cksumcheck

    • fpm_search_optimize

    • fpm_no_stream_inserts

    • suppresslistname



  • I have tried a few more things.
    While being unable to connect with my main desktop and having it set up with a static IP:

    • I stopped all services (using status\services).

    • I disabled all the block/reject rules on the LAN interface.
      The only ones there were pfBlocker reject rules.
      The firewall log was not showing any LAN rejects.

    • Reloaded filters (using Status\Filter Reload)

    • Reset states (using Diagnostics\Sates)

    • Killed of a number of processes that were killable (using "kill" in Diagnostics\Command Prompt)

    After this, I still was unable to ping my main desktop either from pfSense or from another PC.
    After a pfSense reboot however, it connected just fine.


  • Netgate Administrator

    Rogue DHCP server on your LAN? Can cause odd symptoms like this. Especially as a delay is involved. Is it of the order of your DHCP lease time?
    Doesn't explain why a static IP doesn't work though. Some other local IP conflict then perhaps a rogue device with the same IP as the pfSense LAN or your device.

    Try running a packet capture on LAN to see if your pings are arriving or responses being sent.

    Steve

    Edit: Typo, rogue not rouge!



  • Thanks for the suggestions.

    I would think that being unable to connect while having a static IP would eliminate any sort of DHCP problem (like you said).
    The two other devices that could have DHCP servers running, my NAS and WAP have them both disabled, and the problem still occurs with them off.
    I have my devices set up with static leases, including the main PC.
    The lease time for the DHCP server on pfSense is left at the defaults, which is 2 hours I think.
    When the PC is connected, the DHCP logs show a DHCPREQUEST once an hour, and a DHCPINFORM and DHCPACK once every 10 minutes.
    I haven't quite figured out the exact threshold is for this problem to occur, but I think it is somewhere past 4 hours after powering off a PC.
    I hadn't kept track of exact times between power off and power on, but I have started doing that now.
    So far, the longest time between power off and power on and still being able to connect has been 3 hours and 40 minutes, but I don't have a lot of data yet.

    An IP conflict/rogue device is an interesting thought, however I have a limited set of devices (about a half dozen or so) and most are set up with static DHCP leases (half of them are usually off).
    The only ones that don't have static IPs are devices that are connecting to the wireless access point, and I almost never have the WAP on.
    And virtual machines on the main PC itself, which is a rare occurrence as well.

    When this first started, I did try a packet capture on the LAN interface on pfSense before, while pinging from the PC.
    It showed no traffic for the relevant address in the capture.
    However, that has been a while ago, I will try that again next time.



  • More information:

    This time I checked "disable all packet filtering" in "System: Advanced: Firewall and NAT" after being unable to connect.
    It did not make a difference, I still couldn't connect.
    FYI, the time the PC was powered off this time was 5 hours.

    However, I re-ran packet captures (with "disable all packet filtering" back to unchecked I think), and I must have done it incorrectly last time, since it showed some traffic this time:

    • On the LAN/BRIDGE0 bridge interface, it shows ARP broadcast requests from the main PC and ARP replies from pfSense LAN/BRIDGE0 to the main PC

    • On the OPT2/em2 interface that the main PC is connected to, it only shows ARP broadcast requests from the main PC, but no replies.

    So for some reason pfSense is able to direct the OPT2/em2 ARP broadcast to LAN/BRIDGE0, but it doesn't  forward the ARP reply from LAN/BRIDGE0 to OPT2/em2.

    I did run a ping from another PC to pfSense to confirm that ARP replies do normally show up on the specific OPT/em interface, and it shows that normally they do.

    FYI, in case it is relevant for this, in "System: Advanced: System Tunables", I have all the settings set at their defaults, except "net.link.bridge.pfil_member" is set to 0 and "net.link.bridge.pfil_bridge" is set to 1.


  • Netgate Administrator

    Hmm. If you run ifconfig at the console do the bridge member interfaces show as 'promiscuous mode'?

    Why are arp replies from LAN not appearing on OPT2? Perhaps because the bridge device thinks it has a route the main PC via some other interface? Maybe try packet captures on other bridge members to see if replies are ending up on the wrong interface. Just speculation.  ;)

    Steve



  • The first post shows output from ifconfig, it shows PROMISC in the interfaces flags of the bridge member interfaces.
    I will re-run it next time to make sure.

    I will run package captures on the other interfaces next time to see if the replies end up elsewhere.

    Thanks for the suggestions.



  • Packet captures on other interfaces don't show the missing ARP replies.

    I re-ran ifconfig, and it shows the same info as in the first post.

    However, I had success coaxing the connection back to life without a reboot.

    I ran "ifconfig em2 down" and "ifconfig em2 up" and I think that got it going again.
    However, I am not 100% sure, since I also played around with "ifconfig em2 -promisc" and "ifconfig em2 promisc", but nothing seemed to happen when I ran those.

    Edit:
    Tried it on a different PC, and it's the "ifconfig em3 up" after the "ifconfig em3 down" that does it.

    Here is the log, I ran the ifconfig commands at 07:22:09 and 07:22:13.
    It came back to life shortly after 07:22:13.

    Apr 21 07:22:19 	php: rc.newwanip: Interface does not have an IP address, nothing to do.
    Apr 21 07:22:19 	php: rc.newwanip: rc.newwanip: Informational is starting em3.
    Apr 21 07:22:16 	check_reload_status: rc.newwanip starting em3
    Apr 21 07:22:15 	php: rc.linkup: Hotplug event detected for OPT3(opt3) but ignoring since interface is configured with static IP ( )
    Apr 21 07:22:13 	php: rc.linkup: Hotplug event detected for OPT3(opt3) but ignoring since interface is configured with static IP ( )
    Apr 21 07:22:13 	check_reload_status: Linkup starting em3
    Apr 21 07:22:13 	kernel: em3: link state changed to UP
    Apr 21 07:22:09 	kernel: em3: link state changed to DOWN
    Apr 21 07:22:09 	check_reload_status: Linkup starting em3
    Apr 21 07:20:05 	php: rc.newwanip: Interface does not have an IP address, nothing to do.
    Apr 21 07:20:05 	php: rc.newwanip: rc.newwanip: Informational is starting em3.
    Apr 21 07:20:01 	check_reload_status: rc.newwanip starting em3
    Apr 21 07:20:00 	php: rc.linkup: Hotplug event detected for OPT3(opt3) but ignoring since interface is configured with static IP ( )
    Apr 21 07:19:57 	kernel: em3: link state changed to UP
    Apr 21 07:19:57 	check_reload_status: Linkup starting em3
    

  • Netgate Administrator

    Hmm, interesting. I wonder why the interface stops responding yet nothing in the logs.  :-
    Can you bring the interface back by unplugging and re-plugging the cable?

    Steve



  • When I tried it in the past, unplugging and re-plugging the cable didn't make a difference, neither did switching to a different port.
    I will try that again though next time.

    I am wondering if it's not some sort of sleep or power saving mode that is enabled in the 2.1.1 nic driver that isn't in 2.1.0 (if such a thing exists)?
    I haven't narrowed down the exact time threshold, but it seems consistent somewhere between 4 and 5 hours, so maybe it's some sort of sleep time setting (although I would think that would be much shorter).


  • Netgate Administrator

    Good question. The em drivers were updated for 2.1.1 so that's a distinct possibility. What are the actual chips in the NICs you have? Anything relevant in hw.em? This looks a likely suspect:

    [2.1.2-RELEASE][root@pfsense.fire.box]/root(3): sysctl hw.em.smart_pwr_down
    hw.em.smart_pwr_down: 0
    

    Anything suspect in dev.em?

    Steve



  • The nics are all Intel 82574L.

    $ sysctl hw.em
    hw.em.num_queues: 0
    hw.em.eee_setting: 1
    hw.em.rx_process_limit: 100
    hw.em.enable_msix: 1
    hw.em.sbp: 0
    hw.em.smart_pwr_down: 0
    hw.em.txd: 1024
    hw.em.rxd: 1024
    hw.em.rx_abs_int_delay: 66
    hw.em.tx_abs_int_delay: 66
    hw.em.rx_int_delay: 0
    hw.em.tx_int_delay: 66
    
    $ sysctl dev.em
    dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.0.%driver: em
    dev.em.0.%location: slot=0 function=0
    dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.0.%parent: pci5
    dev.em.0.nvm: -1
    dev.em.0.debug: -1
    dev.em.0.fc: 3
    dev.em.0.rx_int_delay: 0
    dev.em.0.tx_int_delay: 66
    dev.em.0.rx_abs_int_delay: 66
    dev.em.0.tx_abs_int_delay: 66
    dev.em.0.itr: 488
    dev.em.0.rx_processing_limit: 100
    dev.em.0.eee_control: 1
    dev.em.0.link_irq: 3
    dev.em.0.mbuf_alloc_fail: 0
    dev.em.0.cluster_alloc_fail: 0
    dev.em.0.dropped: 0
    dev.em.0.tx_dma_fail: 0
    dev.em.0.rx_overruns: 0
    dev.em.0.watchdog_timeouts: 0
    dev.em.0.device_control: 1074790984
    dev.em.0.rx_control: 67141658
    dev.em.0.fc_high_water: 18432
    dev.em.0.fc_low_water: 16932
    dev.em.0.queue0.txd_head: 66
    dev.em.0.queue0.txd_tail: 66
    dev.em.0.queue0.tx_irq: 1332941
    dev.em.0.queue0.no_desc_avail: 0
    dev.em.0.queue0.rxd_head: 543
    dev.em.0.queue0.rxd_tail: 542
    dev.em.0.queue0.rx_irq: 2110549
    dev.em.0.mac_stats.excess_coll: 0
    dev.em.0.mac_stats.single_coll: 0
    dev.em.0.mac_stats.multiple_coll: 0
    dev.em.0.mac_stats.late_coll: 0
    dev.em.0.mac_stats.collision_count: 0
    dev.em.0.mac_stats.symbol_errors: 0
    dev.em.0.mac_stats.sequence_errors: 0
    dev.em.0.mac_stats.defer_count: 0
    dev.em.0.mac_stats.missed_packets: 0
    dev.em.0.mac_stats.recv_no_buff: 0
    dev.em.0.mac_stats.recv_undersize: 0
    dev.em.0.mac_stats.recv_fragmented: 0
    dev.em.0.mac_stats.recv_oversize: 14
    dev.em.0.mac_stats.recv_jabber: 0
    dev.em.0.mac_stats.recv_errs: 1
    dev.em.0.mac_stats.crc_errs: 1616
    dev.em.0.mac_stats.alignment_errs: 0
    dev.em.0.mac_stats.coll_ext_errs: 0
    dev.em.0.mac_stats.xon_recvd: 0
    dev.em.0.mac_stats.xon_txd: 0
    dev.em.0.mac_stats.xoff_recvd: 0
    dev.em.0.mac_stats.xoff_txd: 0
    dev.em.0.mac_stats.total_pkts_recvd: 2601085
    dev.em.0.mac_stats.good_pkts_recvd: 2599455
    dev.em.0.mac_stats.bcast_pkts_recvd: 86896
    dev.em.0.mac_stats.mcast_pkts_recvd: 144132
    dev.em.0.mac_stats.rx_frames_64: 207244
    dev.em.0.mac_stats.rx_frames_65_127: 839440
    dev.em.0.mac_stats.rx_frames_128_255: 225214
    dev.em.0.mac_stats.rx_frames_256_511: 121557
    dev.em.0.mac_stats.rx_frames_512_1023: 43046
    dev.em.0.mac_stats.rx_frames_1024_1522: 1162954
    dev.em.0.mac_stats.good_octets_recvd: 1925347696
    dev.em.0.mac_stats.good_octets_txd: 265656113
    dev.em.0.mac_stats.total_pkts_txd: 2281444
    dev.em.0.mac_stats.good_pkts_txd: 2281444
    dev.em.0.mac_stats.bcast_pkts_txd: 19
    dev.em.0.mac_stats.mcast_pkts_txd: 5
    dev.em.0.mac_stats.tx_frames_64: 401078
    dev.em.0.mac_stats.tx_frames_65_127: 1682199
    dev.em.0.mac_stats.tx_frames_128_255: 59450
    dev.em.0.mac_stats.tx_frames_256_511: 57767
    dev.em.0.mac_stats.tx_frames_512_1023: 37172
    dev.em.0.mac_stats.tx_frames_1024_1522: 43778
    dev.em.0.mac_stats.tso_txd: 0
    dev.em.0.mac_stats.tso_ctx_fail: 0
    dev.em.0.interrupts.asserts: 5
    dev.em.0.interrupts.rx_pkt_timer: 0
    dev.em.0.interrupts.rx_abs_timer: 0
    dev.em.0.interrupts.tx_pkt_timer: 0
    dev.em.0.interrupts.tx_abs_timer: 0
    dev.em.0.interrupts.tx_queue_empty: 0
    dev.em.0.interrupts.tx_queue_min_thresh: 0
    dev.em.0.interrupts.rx_desc_min_thresh: 0
    dev.em.0.interrupts.rx_overrun: 0
    dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.1.%driver: em
    dev.em.1.%location: slot=0 function=0
    dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.1.%parent: pci6
    dev.em.1.nvm: -1
    dev.em.1.debug: -1
    dev.em.1.fc: 3
    dev.em.1.rx_int_delay: 0
    dev.em.1.tx_int_delay: 66
    dev.em.1.rx_abs_int_delay: 66
    dev.em.1.tx_abs_int_delay: 66
    dev.em.1.itr: 488
    dev.em.1.rx_processing_limit: 100
    dev.em.1.eee_control: 1
    dev.em.1.link_irq: 13
    dev.em.1.mbuf_alloc_fail: 0
    dev.em.1.cluster_alloc_fail: 0
    dev.em.1.dropped: 0
    dev.em.1.tx_dma_fail: 0
    dev.em.1.rx_overruns: 0
    dev.em.1.watchdog_timeouts: 0
    dev.em.1.device_control: 1477444168
    dev.em.1.rx_control: 67141658
    dev.em.1.fc_high_water: 18432
    dev.em.1.fc_low_water: 16932
    dev.em.1.queue0.txd_head: 829
    dev.em.1.queue0.txd_tail: 829
    dev.em.1.queue0.tx_irq: 1680542
    dev.em.1.queue0.no_desc_avail: 0
    dev.em.1.queue0.rxd_head: 18
    dev.em.1.queue0.rxd_tail: 17
    dev.em.1.queue0.rx_irq: 1377586
    dev.em.1.mac_stats.excess_coll: 0
    dev.em.1.mac_stats.single_coll: 0
    dev.em.1.mac_stats.multiple_coll: 0
    dev.em.1.mac_stats.late_coll: 0
    dev.em.1.mac_stats.collision_count: 0
    dev.em.1.mac_stats.symbol_errors: 0
    dev.em.1.mac_stats.sequence_errors: 0
    dev.em.1.mac_stats.defer_count: 0
    dev.em.1.mac_stats.missed_packets: 0
    dev.em.1.mac_stats.recv_no_buff: 0
    dev.em.1.mac_stats.recv_undersize: 0
    dev.em.1.mac_stats.recv_fragmented: 0
    dev.em.1.mac_stats.recv_oversize: 0
    dev.em.1.mac_stats.recv_jabber: 0
    dev.em.1.mac_stats.recv_errs: 0
    dev.em.1.mac_stats.crc_errs: 0
    dev.em.1.mac_stats.alignment_errs: 0
    dev.em.1.mac_stats.coll_ext_errs: 0
    dev.em.1.mac_stats.xon_recvd: 0
    dev.em.1.mac_stats.xon_txd: 0
    dev.em.1.mac_stats.xoff_recvd: 0
    dev.em.1.mac_stats.xoff_txd: 0
    dev.em.1.mac_stats.total_pkts_recvd: 6054930
    dev.em.1.mac_stats.good_pkts_recvd: 6054930
    dev.em.1.mac_stats.bcast_pkts_recvd: 814
    dev.em.1.mac_stats.mcast_pkts_recvd: 18
    dev.em.1.mac_stats.rx_frames_64: 3394973
    dev.em.1.mac_stats.rx_frames_65_127: 1581197
    dev.em.1.mac_stats.rx_frames_128_255: 145932
    dev.em.1.mac_stats.rx_frames_256_511: 68688
    dev.em.1.mac_stats.rx_frames_512_1023: 57308
    dev.em.1.mac_stats.rx_frames_1024_1522: 806832
    dev.em.1.mac_stats.good_octets_recvd: 1643667828
    dev.em.1.mac_stats.good_octets_txd: 10924986586
    dev.em.1.mac_stats.total_pkts_txd: 8607743
    dev.em.1.mac_stats.good_pkts_txd: 8607743
    dev.em.1.mac_stats.bcast_pkts_txd: 16764
    dev.em.1.mac_stats.mcast_pkts_txd: 8128
    dev.em.1.mac_stats.tx_frames_64: 435053
    dev.em.1.mac_stats.tx_frames_65_127: 621579
    dev.em.1.mac_stats.tx_frames_128_255: 326492
    dev.em.1.mac_stats.tx_frames_256_511: 104439
    dev.em.1.mac_stats.tx_frames_512_1023: 44603
    dev.em.1.mac_stats.tx_frames_1024_1522: 7075577
    dev.em.1.mac_stats.tso_txd: 0
    dev.em.1.mac_stats.tso_ctx_fail: 0
    dev.em.1.interrupts.asserts: 7
    dev.em.1.interrupts.rx_pkt_timer: 0
    dev.em.1.interrupts.rx_abs_timer: 0
    dev.em.1.interrupts.tx_pkt_timer: 0
    dev.em.1.interrupts.tx_abs_timer: 0
    dev.em.1.interrupts.tx_queue_empty: 0
    dev.em.1.interrupts.tx_queue_min_thresh: 0
    dev.em.1.interrupts.rx_desc_min_thresh: 0
    dev.em.1.interrupts.rx_overrun: 0
    dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.2.%driver: em
    dev.em.2.%location: slot=0 function=0
    dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.2.%parent: pci10
    dev.em.2.nvm: -1
    dev.em.2.debug: -1
    dev.em.2.fc: 3
    dev.em.2.rx_int_delay: 0
    dev.em.2.tx_int_delay: 66
    dev.em.2.rx_abs_int_delay: 66
    dev.em.2.tx_abs_int_delay: 66
    dev.em.2.itr: 488
    dev.em.2.rx_processing_limit: 100
    dev.em.2.eee_control: 1
    dev.em.2.link_irq: 57
    dev.em.2.mbuf_alloc_fail: 0
    dev.em.2.cluster_alloc_fail: 0
    dev.em.2.dropped: 0
    dev.em.2.tx_dma_fail: 0
    dev.em.2.rx_overruns: 0
    dev.em.2.watchdog_timeouts: 0
    dev.em.2.device_control: 1074790984
    dev.em.2.rx_control: 67141658
    dev.em.2.fc_high_water: 18432
    dev.em.2.fc_low_water: 16932
    dev.em.2.queue0.txd_head: 475
    dev.em.2.queue0.txd_tail: 475
    dev.em.2.queue0.tx_irq: 1314386
    dev.em.2.queue0.no_desc_avail: 4
    dev.em.2.queue0.rxd_head: 565
    dev.em.2.queue0.rxd_tail: 564
    dev.em.2.queue0.rx_irq: 1025355
    dev.em.2.mac_stats.excess_coll: 0
    dev.em.2.mac_stats.single_coll: 0
    dev.em.2.mac_stats.multiple_coll: 0
    dev.em.2.mac_stats.late_coll: 0
    dev.em.2.mac_stats.collision_count: 0
    dev.em.2.mac_stats.symbol_errors: 0
    dev.em.2.mac_stats.sequence_errors: 0
    dev.em.2.mac_stats.defer_count: 0
    dev.em.2.mac_stats.missed_packets: 0
    dev.em.2.mac_stats.recv_no_buff: 0
    dev.em.2.mac_stats.recv_undersize: 0
    dev.em.2.mac_stats.recv_fragmented: 0
    dev.em.2.mac_stats.recv_oversize: 0
    dev.em.2.mac_stats.recv_jabber: 0
    dev.em.2.mac_stats.recv_errs: 0
    dev.em.2.mac_stats.crc_errs: 0
    dev.em.2.mac_stats.alignment_errs: 0
    dev.em.2.mac_stats.coll_ext_errs: 0
    dev.em.2.mac_stats.xon_recvd: 0
    dev.em.2.mac_stats.xon_txd: 67
    dev.em.2.mac_stats.xoff_recvd: 0
    dev.em.2.mac_stats.xoff_txd: 67
    dev.em.2.mac_stats.total_pkts_recvd: 7579284
    dev.em.2.mac_stats.good_pkts_recvd: 7579284
    dev.em.2.mac_stats.bcast_pkts_recvd: 4056
    dev.em.2.mac_stats.mcast_pkts_recvd: 1894
    dev.em.2.mac_stats.rx_frames_64: 732927
    dev.em.2.mac_stats.rx_frames_65_127: 127308
    dev.em.2.mac_stats.rx_frames_128_255: 236522
    dev.em.2.mac_stats.rx_frames_256_511: 32858
    dev.em.2.mac_stats.rx_frames_512_1023: 70604
    dev.em.2.mac_stats.rx_frames_1024_1522: 6379065
    dev.em.2.mac_stats.good_octets_recvd: 9847640417
    dev.em.2.mac_stats.good_octets_txd: 2161810275
    dev.em.2.mac_stats.total_pkts_txd: 5162048
    dev.em.2.mac_stats.good_pkts_txd: 5161914
    dev.em.2.mac_stats.bcast_pkts_txd: 5818
    dev.em.2.mac_stats.mcast_pkts_txd: 3845
    dev.em.2.mac_stats.tx_frames_64: 3311268
    dev.em.2.mac_stats.tx_frames_65_127: 402284
    dev.em.2.mac_stats.tx_frames_128_255: 129308
    dev.em.2.mac_stats.tx_frames_256_511: 25539
    dev.em.2.mac_stats.tx_frames_512_1023: 92406
    dev.em.2.mac_stats.tx_frames_1024_1522: 1201109
    dev.em.2.mac_stats.tso_txd: 0
    dev.em.2.mac_stats.tso_ctx_fail: 0
    dev.em.2.interrupts.asserts: 36
    dev.em.2.interrupts.rx_pkt_timer: 0
    dev.em.2.interrupts.rx_abs_timer: 0
    dev.em.2.interrupts.tx_pkt_timer: 0
    dev.em.2.interrupts.tx_abs_timer: 0
    dev.em.2.interrupts.tx_queue_empty: 0
    dev.em.2.interrupts.tx_queue_min_thresh: 0
    dev.em.2.interrupts.rx_desc_min_thresh: 0
    dev.em.2.interrupts.rx_overrun: 0
    dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.3.%driver: em
    dev.em.3.%location: slot=0 function=0
    dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.3.%parent: pci11
    dev.em.3.nvm: -1
    dev.em.3.debug: -1
    dev.em.3.fc: 3
    dev.em.3.rx_int_delay: 0
    dev.em.3.tx_int_delay: 66
    dev.em.3.rx_abs_int_delay: 66
    dev.em.3.tx_abs_int_delay: 66
    dev.em.3.itr: 488
    dev.em.3.rx_processing_limit: 100
    dev.em.3.eee_control: 1
    dev.em.3.link_irq: 13
    dev.em.3.mbuf_alloc_fail: 0
    dev.em.3.cluster_alloc_fail: 0
    dev.em.3.dropped: 0
    dev.em.3.tx_dma_fail: 0
    dev.em.3.rx_overruns: 0
    dev.em.3.watchdog_timeouts: 0
    dev.em.3.device_control: 1074790984
    dev.em.3.rx_control: 67141658
    dev.em.3.fc_high_water: 18432
    dev.em.3.fc_low_water: 16932
    dev.em.3.queue0.txd_head: 386
    dev.em.3.queue0.txd_tail: 386
    dev.em.3.queue0.tx_irq: 276
    dev.em.3.queue0.no_desc_avail: 0
    dev.em.3.queue0.rxd_head: 218
    dev.em.3.queue0.rxd_tail: 217
    dev.em.3.queue0.rx_irq: 978
    dev.em.3.mac_stats.excess_coll: 0
    dev.em.3.mac_stats.single_coll: 0
    dev.em.3.mac_stats.multiple_coll: 0
    dev.em.3.mac_stats.late_coll: 0
    dev.em.3.mac_stats.collision_count: 0
    dev.em.3.mac_stats.symbol_errors: 0
    dev.em.3.mac_stats.sequence_errors: 0
    dev.em.3.mac_stats.defer_count: 0
    dev.em.3.mac_stats.missed_packets: 0
    dev.em.3.mac_stats.recv_no_buff: 0
    dev.em.3.mac_stats.recv_undersize: 0
    dev.em.3.mac_stats.recv_fragmented: 0
    dev.em.3.mac_stats.recv_oversize: 0
    dev.em.3.mac_stats.recv_jabber: 0
    dev.em.3.mac_stats.recv_errs: 0
    dev.em.3.mac_stats.crc_errs: 0
    dev.em.3.mac_stats.alignment_errs: 0
    dev.em.3.mac_stats.coll_ext_errs: 0
    dev.em.3.mac_stats.xon_recvd: 0
    dev.em.3.mac_stats.xon_txd: 0
    dev.em.3.mac_stats.xoff_recvd: 0
    dev.em.3.mac_stats.xoff_txd: 0
    dev.em.3.mac_stats.total_pkts_recvd: 979
    dev.em.3.mac_stats.good_pkts_recvd: 979
    dev.em.3.mac_stats.bcast_pkts_recvd: 616
    dev.em.3.mac_stats.mcast_pkts_recvd: 158
    dev.em.3.mac_stats.rx_frames_64: 63
    dev.em.3.mac_stats.rx_frames_65_127: 163
    dev.em.3.mac_stats.rx_frames_128_255: 112
    dev.em.3.mac_stats.rx_frames_256_511: 641
    dev.em.3.mac_stats.rx_frames_512_1023: 0
    dev.em.3.mac_stats.rx_frames_1024_1522: 0
    dev.em.3.mac_stats.good_octets_recvd: 255346
    dev.em.3.mac_stats.good_octets_txd: 55615
    dev.em.3.mac_stats.total_pkts_txd: 303
    dev.em.3.mac_stats.good_pkts_txd: 303
    dev.em.3.mac_stats.bcast_pkts_txd: 79
    dev.em.3.mac_stats.mcast_pkts_txd: 32
    dev.em.3.mac_stats.tx_frames_64: 32
    dev.em.3.mac_stats.tx_frames_65_127: 229
    dev.em.3.mac_stats.tx_frames_128_255: 9
    dev.em.3.mac_stats.tx_frames_256_511: 14
    dev.em.3.mac_stats.tx_frames_512_1023: 1
    dev.em.3.mac_stats.tx_frames_1024_1522: 18
    dev.em.3.mac_stats.tso_txd: 0
    dev.em.3.mac_stats.tso_ctx_fail: 0
    dev.em.3.interrupts.asserts: 9
    dev.em.3.interrupts.rx_pkt_timer: 0
    dev.em.3.interrupts.rx_abs_timer: 0
    dev.em.3.interrupts.tx_pkt_timer: 0
    dev.em.3.interrupts.tx_abs_timer: 0
    dev.em.3.interrupts.tx_queue_empty: 0
    dev.em.3.interrupts.tx_queue_min_thresh: 0
    dev.em.3.interrupts.rx_desc_min_thresh: 0
    dev.em.3.interrupts.rx_overrun: 0
    dev.em.4.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.4.%driver: em
    dev.em.4.%location: slot=0 function=0
    dev.em.4.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.4.%parent: pci15
    dev.em.4.nvm: -1
    dev.em.4.debug: -1
    dev.em.4.fc: 3
    dev.em.4.rx_int_delay: 0
    dev.em.4.tx_int_delay: 66
    dev.em.4.rx_abs_int_delay: 66
    dev.em.4.tx_abs_int_delay: 66
    dev.em.4.itr: 488
    dev.em.4.rx_processing_limit: 100
    dev.em.4.eee_control: 1
    dev.em.4.link_irq: 0
    dev.em.4.mbuf_alloc_fail: 0
    dev.em.4.cluster_alloc_fail: 0
    dev.em.4.dropped: 0
    dev.em.4.tx_dma_fail: 0
    dev.em.4.rx_overruns: 0
    dev.em.4.watchdog_timeouts: 0
    dev.em.4.device_control: 1074790984
    dev.em.4.rx_control: 67141658
    dev.em.4.fc_high_water: 18432
    dev.em.4.fc_low_water: 16932
    dev.em.4.queue0.txd_head: 0
    dev.em.4.queue0.txd_tail: 0
    dev.em.4.queue0.tx_irq: 0
    dev.em.4.queue0.no_desc_avail: 0
    dev.em.4.queue0.rxd_head: 0
    dev.em.4.queue0.rxd_tail: 1023
    dev.em.4.queue0.rx_irq: 0
    dev.em.4.mac_stats.excess_coll: 0
    dev.em.4.mac_stats.single_coll: 0
    dev.em.4.mac_stats.multiple_coll: 0
    dev.em.4.mac_stats.late_coll: 0
    dev.em.4.mac_stats.collision_count: 0
    dev.em.4.mac_stats.symbol_errors: 0
    dev.em.4.mac_stats.sequence_errors: 0
    dev.em.4.mac_stats.defer_count: 0
    dev.em.4.mac_stats.missed_packets: 0
    dev.em.4.mac_stats.recv_no_buff: 0
    dev.em.4.mac_stats.recv_undersize: 0
    dev.em.4.mac_stats.recv_fragmented: 0
    dev.em.4.mac_stats.recv_oversize: 0
    dev.em.4.mac_stats.recv_jabber: 0
    dev.em.4.mac_stats.recv_errs: 0
    dev.em.4.mac_stats.crc_errs: 0
    dev.em.4.mac_stats.alignment_errs: 0
    dev.em.4.mac_stats.coll_ext_errs: 0
    dev.em.4.mac_stats.xon_recvd: 0
    dev.em.4.mac_stats.xon_txd: 0
    dev.em.4.mac_stats.xoff_recvd: 0
    dev.em.4.mac_stats.xoff_txd: 0
    dev.em.4.mac_stats.total_pkts_recvd: 0
    dev.em.4.mac_stats.good_pkts_recvd: 0
    dev.em.4.mac_stats.bcast_pkts_recvd: 0
    dev.em.4.mac_stats.mcast_pkts_recvd: 0
    dev.em.4.mac_stats.rx_frames_64: 0
    dev.em.4.mac_stats.rx_frames_65_127: 0
    dev.em.4.mac_stats.rx_frames_128_255: 0
    dev.em.4.mac_stats.rx_frames_256_511: 0
    dev.em.4.mac_stats.rx_frames_512_1023: 0
    dev.em.4.mac_stats.rx_frames_1024_1522: 0
    dev.em.4.mac_stats.good_octets_recvd: 0
    dev.em.4.mac_stats.good_octets_txd: 0
    dev.em.4.mac_stats.total_pkts_txd: 0
    dev.em.4.mac_stats.good_pkts_txd: 0
    dev.em.4.mac_stats.bcast_pkts_txd: 0
    dev.em.4.mac_stats.mcast_pkts_txd: 0
    dev.em.4.mac_stats.tx_frames_64: 0
    dev.em.4.mac_stats.tx_frames_65_127: 0
    dev.em.4.mac_stats.tx_frames_128_255: 0
    dev.em.4.mac_stats.tx_frames_256_511: 0
    dev.em.4.mac_stats.tx_frames_512_1023: 0
    dev.em.4.mac_stats.tx_frames_1024_1522: 0
    dev.em.4.mac_stats.tso_txd: 0
    dev.em.4.mac_stats.tso_ctx_fail: 0
    dev.em.4.interrupts.asserts: 0
    dev.em.4.interrupts.rx_pkt_timer: 0
    dev.em.4.interrupts.rx_abs_timer: 0
    dev.em.4.interrupts.tx_pkt_timer: 0
    dev.em.4.interrupts.tx_abs_timer: 0
    dev.em.4.interrupts.tx_queue_empty: 0
    dev.em.4.interrupts.tx_queue_min_thresh: 0
    dev.em.4.interrupts.rx_desc_min_thresh: 0
    dev.em.4.interrupts.rx_overrun: 0
    dev.em.5.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.5.%driver: em
    dev.em.5.%location: slot=0 function=0
    dev.em.5.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.5.%parent: pci16
    dev.em.5.nvm: -1
    dev.em.5.debug: -1
    dev.em.5.fc: 3
    dev.em.5.rx_int_delay: 0
    dev.em.5.tx_int_delay: 66
    dev.em.5.rx_abs_int_delay: 66
    dev.em.5.tx_abs_int_delay: 66
    dev.em.5.itr: 488
    dev.em.5.rx_processing_limit: 100
    dev.em.5.eee_control: 1
    dev.em.5.link_irq: 14
    dev.em.5.mbuf_alloc_fail: 0
    dev.em.5.cluster_alloc_fail: 0
    dev.em.5.dropped: 0
    dev.em.5.tx_dma_fail: 0
    dev.em.5.rx_overruns: 0
    dev.em.5.watchdog_timeouts: 0
    dev.em.5.device_control: 1477444168
    dev.em.5.rx_control: 67141658
    dev.em.5.fc_high_water: 18432
    dev.em.5.fc_low_water: 16932
    dev.em.5.queue0.txd_head: 1015
    dev.em.5.queue0.txd_tail: 1015
    dev.em.5.queue0.tx_irq: 31244
    dev.em.5.queue0.no_desc_avail: 0
    dev.em.5.queue0.rxd_head: 771
    dev.em.5.queue0.rxd_tail: 770
    dev.em.5.queue0.rx_irq: 29384
    dev.em.5.mac_stats.excess_coll: 0
    dev.em.5.mac_stats.single_coll: 0
    dev.em.5.mac_stats.multiple_coll: 0
    dev.em.5.mac_stats.late_coll: 0
    dev.em.5.mac_stats.collision_count: 0
    dev.em.5.mac_stats.symbol_errors: 0
    dev.em.5.mac_stats.sequence_errors: 0
    dev.em.5.mac_stats.defer_count: 0
    dev.em.5.mac_stats.missed_packets: 0
    dev.em.5.mac_stats.recv_no_buff: 0
    dev.em.5.mac_stats.recv_undersize: 0
    dev.em.5.mac_stats.recv_fragmented: 0
    dev.em.5.mac_stats.recv_oversize: 0
    dev.em.5.mac_stats.recv_jabber: 0
    dev.em.5.mac_stats.recv_errs: 0
    dev.em.5.mac_stats.crc_errs: 0
    dev.em.5.mac_stats.alignment_errs: 0
    dev.em.5.mac_stats.coll_ext_errs: 0
    dev.em.5.mac_stats.xon_recvd: 0
    dev.em.5.mac_stats.xon_txd: 0
    dev.em.5.mac_stats.xoff_recvd: 0
    dev.em.5.mac_stats.xoff_txd: 0
    dev.em.5.mac_stats.total_pkts_recvd: 29443
    dev.em.5.mac_stats.good_pkts_recvd: 29443
    dev.em.5.mac_stats.bcast_pkts_recvd: 10091
    dev.em.5.mac_stats.mcast_pkts_recvd: 6055
    dev.em.5.mac_stats.rx_frames_64: 10733
    dev.em.5.mac_stats.rx_frames_65_127: 10674
    dev.em.5.mac_stats.rx_frames_128_255: 408
    dev.em.5.mac_stats.rx_frames_256_511: 6155
    dev.em.5.mac_stats.rx_frames_512_1023: 409
    dev.em.5.mac_stats.rx_frames_1024_1522: 1064
    dev.em.5.mac_stats.good_octets_recvd: 5941407
    dev.em.5.mac_stats.good_octets_txd: 5567151
    dev.em.5.mac_stats.total_pkts_txd: 31914
    dev.em.5.mac_stats.good_pkts_txd: 31914
    dev.em.5.mac_stats.bcast_pkts_txd: 7475
    dev.em.5.mac_stats.mcast_pkts_txd: 2087
    dev.em.5.mac_stats.tx_frames_64: 14358
    dev.em.5.mac_stats.tx_frames_65_127: 13797
    dev.em.5.mac_stats.tx_frames_128_255: 790
    dev.em.5.mac_stats.tx_frames_256_511: 681
    dev.em.5.mac_stats.tx_frames_512_1023: 274
    dev.em.5.mac_stats.tx_frames_1024_1522: 2014
    dev.em.5.mac_stats.tso_txd: 0
    dev.em.5.mac_stats.tso_ctx_fail: 0
    dev.em.5.interrupts.asserts: 9
    dev.em.5.interrupts.rx_pkt_timer: 0
    dev.em.5.interrupts.rx_abs_timer: 0
    dev.em.5.interrupts.tx_pkt_timer: 0
    dev.em.5.interrupts.tx_abs_timer: 0
    dev.em.5.interrupts.tx_queue_empty: 0
    dev.em.5.interrupts.tx_queue_min_thresh: 0
    dev.em.5.interrupts.rx_desc_min_thresh: 0
    dev.em.5.interrupts.rx_overrun: 0
    dev.em.6.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.6.%driver: em
    dev.em.6.%location: slot=0 function=0
    dev.em.6.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.6.%parent: pci17
    dev.em.6.nvm: -1
    dev.em.6.debug: -1
    dev.em.6.fc: 3
    dev.em.6.rx_int_delay: 0
    dev.em.6.tx_int_delay: 66
    dev.em.6.rx_abs_int_delay: 66
    dev.em.6.tx_abs_int_delay: 66
    dev.em.6.itr: 488
    dev.em.6.rx_processing_limit: 100
    dev.em.6.eee_control: 1
    dev.em.6.link_irq: 17
    dev.em.6.mbuf_alloc_fail: 0
    dev.em.6.cluster_alloc_fail: 0
    dev.em.6.dropped: 0
    dev.em.6.tx_dma_fail: 0
    dev.em.6.rx_overruns: 0
    dev.em.6.watchdog_timeouts: 0
    dev.em.6.device_control: 1477444168
    dev.em.6.rx_control: 67141658
    dev.em.6.fc_high_water: 18432
    dev.em.6.fc_low_water: 16932
    dev.em.6.queue0.txd_head: 817
    dev.em.6.queue0.txd_tail: 817
    dev.em.6.queue0.tx_irq: 81780
    dev.em.6.queue0.no_desc_avail: 0
    dev.em.6.queue0.rxd_head: 348
    dev.em.6.queue0.rxd_tail: 347
    dev.em.6.queue0.rx_irq: 36669
    dev.em.6.mac_stats.excess_coll: 0
    dev.em.6.mac_stats.single_coll: 0
    dev.em.6.mac_stats.multiple_coll: 0
    dev.em.6.mac_stats.late_coll: 0
    dev.em.6.mac_stats.collision_count: 0
    dev.em.6.mac_stats.symbol_errors: 0
    dev.em.6.mac_stats.sequence_errors: 0
    dev.em.6.mac_stats.defer_count: 0
    dev.em.6.mac_stats.missed_packets: 0
    dev.em.6.mac_stats.recv_no_buff: 0
    dev.em.6.mac_stats.recv_undersize: 0
    dev.em.6.mac_stats.recv_fragmented: 0
    dev.em.6.mac_stats.recv_oversize: 0
    dev.em.6.mac_stats.recv_jabber: 0
    dev.em.6.mac_stats.recv_errs: 0
    dev.em.6.mac_stats.crc_errs: 0
    dev.em.6.mac_stats.alignment_errs: 0
    dev.em.6.mac_stats.coll_ext_errs: 0
    dev.em.6.mac_stats.xon_recvd: 0
    dev.em.6.mac_stats.xon_txd: 0
    dev.em.6.mac_stats.xoff_recvd: 0
    dev.em.6.mac_stats.xoff_txd: 0
    dev.em.6.mac_stats.total_pkts_recvd: 40284
    dev.em.6.mac_stats.good_pkts_recvd: 40284
    dev.em.6.mac_stats.bcast_pkts_recvd: 420
    dev.em.6.mac_stats.mcast_pkts_recvd: 376
    dev.em.6.mac_stats.rx_frames_64: 28074
    dev.em.6.mac_stats.rx_frames_65_127: 8076
    dev.em.6.mac_stats.rx_frames_128_255: 927
    dev.em.6.mac_stats.rx_frames_256_511: 800
    dev.em.6.mac_stats.rx_frames_512_1023: 843
    dev.em.6.mac_stats.rx_frames_1024_1522: 1564
    dev.em.6.mac_stats.good_octets_recvd: 5925684
    dev.em.6.mac_stats.good_octets_txd: 87073020
    dev.em.6.mac_stats.total_pkts_txd: 88589
    dev.em.6.mac_stats.good_pkts_txd: 88589
    dev.em.6.mac_stats.bcast_pkts_txd: 17191
    dev.em.6.mac_stats.mcast_pkts_txd: 7828
    dev.em.6.mac_stats.tx_frames_64: 15737
    dev.em.6.mac_stats.tx_frames_65_127: 8968
    dev.em.6.mac_stats.tx_frames_128_255: 2483
    dev.em.6.mac_stats.tx_frames_256_511: 6989
    dev.em.6.mac_stats.tx_frames_512_1023: 704
    dev.em.6.mac_stats.tx_frames_1024_1522: 53708
    dev.em.6.mac_stats.tso_txd: 0
    dev.em.6.mac_stats.tso_ctx_fail: 0
    dev.em.6.interrupts.asserts: 11
    dev.em.6.interrupts.rx_pkt_timer: 0
    dev.em.6.interrupts.rx_abs_timer: 0
    dev.em.6.interrupts.tx_pkt_timer: 0
    dev.em.6.interrupts.tx_abs_timer: 0
    dev.em.6.interrupts.tx_queue_empty: 0
    dev.em.6.interrupts.tx_queue_min_thresh: 0
    dev.em.6.interrupts.rx_desc_min_thresh: 0
    dev.em.6.interrupts.rx_overrun: 0
    dev.em.7.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.7.%driver: em
    dev.em.7.%location: slot=0 function=0
    dev.em.7.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.7.%parent: pci18
    dev.em.7.nvm: -1
    dev.em.7.debug: -1
    dev.em.7.fc: 3
    dev.em.7.rx_int_delay: 0
    dev.em.7.tx_int_delay: 66
    dev.em.7.rx_abs_int_delay: 66
    dev.em.7.tx_abs_int_delay: 66
    dev.em.7.itr: 488
    dev.em.7.rx_processing_limit: 100
    dev.em.7.eee_control: 1
    dev.em.7.link_irq: 0
    dev.em.7.mbuf_alloc_fail: 0
    dev.em.7.cluster_alloc_fail: 0
    dev.em.7.dropped: 0
    dev.em.7.tx_dma_fail: 0
    dev.em.7.rx_overruns: 0
    dev.em.7.watchdog_timeouts: 0
    dev.em.7.device_control: 1074790984
    dev.em.7.rx_control: 67141658
    dev.em.7.fc_high_water: 18432
    dev.em.7.fc_low_water: 16932
    dev.em.7.queue0.txd_head: 0
    dev.em.7.queue0.txd_tail: 0
    dev.em.7.queue0.tx_irq: 0
    dev.em.7.queue0.no_desc_avail: 0
    dev.em.7.queue0.rxd_head: 0
    dev.em.7.queue0.rxd_tail: 1023
    dev.em.7.queue0.rx_irq: 0
    dev.em.7.mac_stats.excess_coll: 0
    dev.em.7.mac_stats.single_coll: 0
    dev.em.7.mac_stats.multiple_coll: 0
    dev.em.7.mac_stats.late_coll: 0
    dev.em.7.mac_stats.collision_count: 0
    dev.em.7.mac_stats.symbol_errors: 0
    dev.em.7.mac_stats.sequence_errors: 0
    dev.em.7.mac_stats.defer_count: 0
    dev.em.7.mac_stats.missed_packets: 0
    dev.em.7.mac_stats.recv_no_buff: 0
    dev.em.7.mac_stats.recv_undersize: 0
    dev.em.7.mac_stats.recv_fragmented: 0
    dev.em.7.mac_stats.recv_oversize: 0
    dev.em.7.mac_stats.recv_jabber: 0
    dev.em.7.mac_stats.recv_errs: 0
    dev.em.7.mac_stats.crc_errs: 0
    dev.em.7.mac_stats.alignment_errs: 0
    dev.em.7.mac_stats.coll_ext_errs: 0
    dev.em.7.mac_stats.xon_recvd: 0
    dev.em.7.mac_stats.xon_txd: 0
    dev.em.7.mac_stats.xoff_recvd: 0
    dev.em.7.mac_stats.xoff_txd: 0
    dev.em.7.mac_stats.total_pkts_recvd: 0
    dev.em.7.mac_stats.good_pkts_recvd: 0
    dev.em.7.mac_stats.bcast_pkts_recvd: 0
    dev.em.7.mac_stats.mcast_pkts_recvd: 0
    dev.em.7.mac_stats.rx_frames_64: 0
    dev.em.7.mac_stats.rx_frames_65_127: 0
    dev.em.7.mac_stats.rx_frames_128_255: 0
    dev.em.7.mac_stats.rx_frames_256_511: 0
    dev.em.7.mac_stats.rx_frames_512_1023: 0
    dev.em.7.mac_stats.rx_frames_1024_1522: 0
    dev.em.7.mac_stats.good_octets_recvd: 0
    dev.em.7.mac_stats.good_octets_txd: 0
    dev.em.7.mac_stats.total_pkts_txd: 0
    dev.em.7.mac_stats.good_pkts_txd: 0
    dev.em.7.mac_stats.bcast_pkts_txd: 0
    dev.em.7.mac_stats.mcast_pkts_txd: 0
    dev.em.7.mac_stats.tx_frames_64: 0
    dev.em.7.mac_stats.tx_frames_65_127: 0
    dev.em.7.mac_stats.tx_frames_128_255: 0
    dev.em.7.mac_stats.tx_frames_256_511: 0
    dev.em.7.mac_stats.tx_frames_512_1023: 0
    dev.em.7.mac_stats.tx_frames_1024_1522: 0
    dev.em.7.mac_stats.tso_txd: 0
    dev.em.7.mac_stats.tso_ctx_fail: 0
    dev.em.7.interrupts.asserts: 0
    dev.em.7.interrupts.rx_pkt_timer: 0
    dev.em.7.interrupts.rx_abs_timer: 0
    dev.em.7.interrupts.tx_pkt_timer: 0
    dev.em.7.interrupts.tx_abs_timer: 0
    dev.em.7.interrupts.tx_queue_empty: 0
    dev.em.7.interrupts.tx_queue_min_thresh: 0
    dev.em.7.interrupts.rx_desc_min_thresh: 0
    dev.em.7.interrupts.rx_overrun: 0
    

    I'm not sure what to watch for.
    FYI, this was not taken while the problem was occurring.


  • Netgate Administrator

    Hmm, very common NIC so should be reported if it was a driver issue.
    You have a lot of CRC errors on em0. Which interface is giving trouble?
    Try to get a set of data when it's stopped responding.

    Steve



  • The nic may be common, but perhaps not a lot of people use pfsense the way I do?
    Meaning, directly connecting PCs to the pfSense box rather than having a switch in the middle, in combination with completely unplugging the PC rather than leaving it powered off, but not unplugged.

    My main PC is connected to em2, but I have reproduced the problem with it connected to other interfaces and with other PCs on other interfaces.

    My WAN interface is em0, which is connected using WiMAX, don't  know if that explains the number of CRC errors or not.

    I will try to get dev.em data the next time the problem happens.

    Edit:

    I tried unplugging and re-plugging the connection, that does not seem to solve the problem, but ifconfig em2 down followed by ifconfig em2 up does.

    Here is the dev.em data while the problem happens:

    $ sysctl dev.em
    dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.0.%driver: em
    dev.em.0.%location: slot=0 function=0
    dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.0.%parent: pci5
    dev.em.0.nvm: -1
    dev.em.0.debug: -1
    dev.em.0.fc: 3
    dev.em.0.rx_int_delay: 0
    dev.em.0.tx_int_delay: 66
    dev.em.0.rx_abs_int_delay: 66
    dev.em.0.tx_abs_int_delay: 66
    dev.em.0.itr: 488
    dev.em.0.rx_processing_limit: 100
    dev.em.0.eee_control: 1
    dev.em.0.link_irq: 3
    dev.em.0.mbuf_alloc_fail: 0
    dev.em.0.cluster_alloc_fail: 0
    dev.em.0.dropped: 0
    dev.em.0.tx_dma_fail: 0
    dev.em.0.rx_overruns: 0
    dev.em.0.watchdog_timeouts: 0
    dev.em.0.device_control: 1074790984
    dev.em.0.rx_control: 67141658
    dev.em.0.fc_high_water: 18432
    dev.em.0.fc_low_water: 16932
    dev.em.0.queue0.txd_head: 955
    dev.em.0.queue0.txd_tail: 955
    dev.em.0.queue0.tx_irq: 1353023
    dev.em.0.queue0.no_desc_avail: 0
    dev.em.0.queue0.rxd_head: 646
    dev.em.0.queue0.rxd_tail: 645
    dev.em.0.queue0.rx_irq: 2194821
    dev.em.0.mac_stats.excess_coll: 0
    dev.em.0.mac_stats.single_coll: 0
    dev.em.0.mac_stats.multiple_coll: 0
    dev.em.0.mac_stats.late_coll: 0
    dev.em.0.mac_stats.collision_count: 0
    dev.em.0.mac_stats.symbol_errors: 0
    dev.em.0.mac_stats.sequence_errors: 0
    dev.em.0.mac_stats.defer_count: 0
    dev.em.0.mac_stats.missed_packets: 0
    dev.em.0.mac_stats.recv_no_buff: 0
    dev.em.0.mac_stats.recv_undersize: 0
    dev.em.0.mac_stats.recv_fragmented: 0
    dev.em.0.mac_stats.recv_oversize: 14
    dev.em.0.mac_stats.recv_jabber: 0
    dev.em.0.mac_stats.recv_errs: 1
    dev.em.0.mac_stats.crc_errs: 1858
    dev.em.0.mac_stats.alignment_errs: 0
    dev.em.0.mac_stats.coll_ext_errs: 0
    dev.em.0.mac_stats.xon_recvd: 0
    dev.em.0.mac_stats.xon_txd: 0
    dev.em.0.mac_stats.xoff_recvd: 0
    dev.em.0.mac_stats.xoff_txd: 0
    dev.em.0.mac_stats.total_pkts_recvd: 2693590
    dev.em.0.mac_stats.good_pkts_recvd: 2691718
    dev.em.0.mac_stats.bcast_pkts_recvd: 114676
    dev.em.0.mac_stats.mcast_pkts_recvd: 169968
    dev.em.0.mac_stats.rx_frames_64: 230759
    dev.em.0.mac_stats.rx_frames_65_127: 881015
    dev.em.0.mac_stats.rx_frames_128_255: 240095
    dev.em.0.mac_stats.rx_frames_256_511: 127210
    dev.em.0.mac_stats.rx_frames_512_1023: 43897
    dev.em.0.mac_stats.rx_frames_1024_1522: 1168742
    dev.em.0.mac_stats.good_octets_recvd: 1944245208
    dev.em.0.mac_stats.good_octets_txd: 267430581
    dev.em.0.mac_stats.total_pkts_txd: 2301786
    dev.em.0.mac_stats.good_pkts_txd: 2301786
    dev.em.0.mac_stats.bcast_pkts_txd: 21
    dev.em.0.mac_stats.mcast_pkts_txd: 5
    dev.em.0.mac_stats.tx_frames_64: 401303
    dev.em.0.mac_stats.tx_frames_65_127: 1701968
    dev.em.0.mac_stats.tx_frames_128_255: 59555
    dev.em.0.mac_stats.tx_frames_256_511: 57933
    dev.em.0.mac_stats.tx_frames_512_1023: 37200
    dev.em.0.mac_stats.tx_frames_1024_1522: 43827
    dev.em.0.mac_stats.tso_txd: 0
    dev.em.0.mac_stats.tso_ctx_fail: 0
    dev.em.0.interrupts.asserts: 5
    dev.em.0.interrupts.rx_pkt_timer: 0
    dev.em.0.interrupts.rx_abs_timer: 0
    dev.em.0.interrupts.tx_pkt_timer: 0
    dev.em.0.interrupts.tx_abs_timer: 0
    dev.em.0.interrupts.tx_queue_empty: 0
    dev.em.0.interrupts.tx_queue_min_thresh: 0
    dev.em.0.interrupts.rx_desc_min_thresh: 0
    dev.em.0.interrupts.rx_overrun: 0
    dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.1.%driver: em
    dev.em.1.%location: slot=0 function=0
    dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.1.%parent: pci6
    dev.em.1.nvm: -1
    dev.em.1.debug: -1
    dev.em.1.fc: 3
    dev.em.1.rx_int_delay: 0
    dev.em.1.tx_int_delay: 66
    dev.em.1.rx_abs_int_delay: 66
    dev.em.1.tx_abs_int_delay: 66
    dev.em.1.itr: 488
    dev.em.1.rx_processing_limit: 100
    dev.em.1.eee_control: 1
    dev.em.1.link_irq: 22
    dev.em.1.mbuf_alloc_fail: 0
    dev.em.1.cluster_alloc_fail: 0
    dev.em.1.dropped: 0
    dev.em.1.tx_dma_fail: 0
    dev.em.1.rx_overruns: 0
    dev.em.1.watchdog_timeouts: 0
    dev.em.1.device_control: 1477444168
    dev.em.1.rx_control: 67141658
    dev.em.1.fc_high_water: 18432
    dev.em.1.fc_low_water: 16932
    dev.em.1.queue0.txd_head: 112
    dev.em.1.queue0.txd_tail: 112
    dev.em.1.queue0.tx_irq: 1690961
    dev.em.1.queue0.no_desc_avail: 0
    dev.em.1.queue0.rxd_head: 37
    dev.em.1.queue0.rxd_tail: 36
    dev.em.1.queue0.rx_irq: 1381555
    dev.em.1.mac_stats.excess_coll: 0
    dev.em.1.mac_stats.single_coll: 0
    dev.em.1.mac_stats.multiple_coll: 0
    dev.em.1.mac_stats.late_coll: 0
    dev.em.1.mac_stats.collision_count: 0
    dev.em.1.mac_stats.symbol_errors: 0
    dev.em.1.mac_stats.sequence_errors: 0
    dev.em.1.mac_stats.defer_count: 0
    dev.em.1.mac_stats.missed_packets: 0
    dev.em.1.mac_stats.recv_no_buff: 0
    dev.em.1.mac_stats.recv_undersize: 0
    dev.em.1.mac_stats.recv_fragmented: 0
    dev.em.1.mac_stats.recv_oversize: 0
    dev.em.1.mac_stats.recv_jabber: 0
    dev.em.1.mac_stats.recv_errs: 0
    dev.em.1.mac_stats.crc_errs: 0
    dev.em.1.mac_stats.alignment_errs: 0
    dev.em.1.mac_stats.coll_ext_errs: 0
    dev.em.1.mac_stats.xon_recvd: 0
    dev.em.1.mac_stats.xon_txd: 0
    dev.em.1.mac_stats.xoff_recvd: 0
    dev.em.1.mac_stats.xoff_txd: 0
    dev.em.1.mac_stats.total_pkts_recvd: 6059045
    dev.em.1.mac_stats.good_pkts_recvd: 6059045
    dev.em.1.mac_stats.bcast_pkts_recvd: 1001
    dev.em.1.mac_stats.mcast_pkts_recvd: 36
    dev.em.1.mac_stats.rx_frames_64: 3395197
    dev.em.1.mac_stats.rx_frames_65_127: 1584918
    dev.em.1.mac_stats.rx_frames_128_255: 146033
    dev.em.1.mac_stats.rx_frames_256_511: 68732
    dev.em.1.mac_stats.rx_frames_512_1023: 57333
    dev.em.1.mac_stats.rx_frames_1024_1522: 806832
    dev.em.1.mac_stats.good_octets_recvd: 1644008445
    dev.em.1.mac_stats.good_octets_txd: 10934359364
    dev.em.1.mac_stats.total_pkts_txd: 8618288
    dev.em.1.mac_stats.good_pkts_txd: 8618288
    dev.em.1.mac_stats.bcast_pkts_txd: 20285
    dev.em.1.mac_stats.mcast_pkts_txd: 9084
    dev.em.1.mac_stats.tx_frames_64: 438597
    dev.em.1.mac_stats.tx_frames_65_127: 621818
    dev.em.1.mac_stats.tx_frames_128_255: 326533
    dev.em.1.mac_stats.tx_frames_256_511: 105392
    dev.em.1.mac_stats.tx_frames_512_1023: 44645
    dev.em.1.mac_stats.tx_frames_1024_1522: 7081303
    dev.em.1.mac_stats.tso_txd: 0
    dev.em.1.mac_stats.tso_ctx_fail: 0
    dev.em.1.interrupts.asserts: 13
    dev.em.1.interrupts.rx_pkt_timer: 0
    dev.em.1.interrupts.rx_abs_timer: 0
    dev.em.1.interrupts.tx_pkt_timer: 0
    dev.em.1.interrupts.tx_abs_timer: 0
    dev.em.1.interrupts.tx_queue_empty: 0
    dev.em.1.interrupts.tx_queue_min_thresh: 0
    dev.em.1.interrupts.rx_desc_min_thresh: 0
    dev.em.1.interrupts.rx_overrun: 0
    dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.2.%driver: em
    dev.em.2.%location: slot=0 function=0
    dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.2.%parent: pci10
    dev.em.2.nvm: -1
    dev.em.2.debug: -1
    dev.em.2.fc: 3
    dev.em.2.rx_int_delay: 0
    dev.em.2.tx_int_delay: 66
    dev.em.2.rx_abs_int_delay: 66
    dev.em.2.tx_abs_int_delay: 66
    dev.em.2.itr: 488
    dev.em.2.rx_processing_limit: 100
    dev.em.2.eee_control: 1
    dev.em.2.link_irq: 65
    dev.em.2.mbuf_alloc_fail: 0
    dev.em.2.cluster_alloc_fail: 0
    dev.em.2.dropped: 0
    dev.em.2.tx_dma_fail: 0
    dev.em.2.rx_overruns: 0
    dev.em.2.watchdog_timeouts: 0
    dev.em.2.device_control: 1477444168
    dev.em.2.rx_control: 67141658
    dev.em.2.fc_high_water: 18432
    dev.em.2.fc_low_water: 16932
    dev.em.2.queue0.txd_head: 475
    dev.em.2.queue0.txd_tail: 475
    dev.em.2.queue0.tx_irq: 1314386
    dev.em.2.queue0.no_desc_avail: 4
    dev.em.2.queue0.rxd_head: 737
    dev.em.2.queue0.rxd_tail: 736
    dev.em.2.queue0.rx_irq: 1025527
    dev.em.2.mac_stats.excess_coll: 0
    dev.em.2.mac_stats.single_coll: 0
    dev.em.2.mac_stats.multiple_coll: 0
    dev.em.2.mac_stats.late_coll: 0
    dev.em.2.mac_stats.collision_count: 0
    dev.em.2.mac_stats.symbol_errors: 0
    dev.em.2.mac_stats.sequence_errors: 0
    dev.em.2.mac_stats.defer_count: 0
    dev.em.2.mac_stats.missed_packets: 0
    dev.em.2.mac_stats.recv_no_buff: 0
    dev.em.2.mac_stats.recv_undersize: 0
    dev.em.2.mac_stats.recv_fragmented: 0
    dev.em.2.mac_stats.recv_oversize: 0
    dev.em.2.mac_stats.recv_jabber: 0
    dev.em.2.mac_stats.recv_errs: 0
    dev.em.2.mac_stats.crc_errs: 0
    dev.em.2.mac_stats.alignment_errs: 0
    dev.em.2.mac_stats.coll_ext_errs: 0
    dev.em.2.mac_stats.xon_recvd: 0
    dev.em.2.mac_stats.xon_txd: 67
    dev.em.2.mac_stats.xoff_recvd: 0
    dev.em.2.mac_stats.xoff_txd: 67
    dev.em.2.mac_stats.total_pkts_recvd: 7579455
    dev.em.2.mac_stats.good_pkts_recvd: 7579455
    dev.em.2.mac_stats.bcast_pkts_recvd: 4221
    dev.em.2.mac_stats.mcast_pkts_recvd: 1900
    dev.em.2.mac_stats.rx_frames_64: 733094
    dev.em.2.mac_stats.rx_frames_65_127: 127312
    dev.em.2.mac_stats.rx_frames_128_255: 236522
    dev.em.2.mac_stats.rx_frames_256_511: 32858
    dev.em.2.mac_stats.rx_frames_512_1023: 70604
    dev.em.2.mac_stats.rx_frames_1024_1522: 6379065
    dev.em.2.mac_stats.good_octets_recvd: 9847651473
    dev.em.2.mac_stats.good_octets_txd: 2161810275
    dev.em.2.mac_stats.total_pkts_txd: 5162048
    dev.em.2.mac_stats.good_pkts_txd: 5161914
    dev.em.2.mac_stats.bcast_pkts_txd: 5818
    dev.em.2.mac_stats.mcast_pkts_txd: 3845
    dev.em.2.mac_stats.tx_frames_64: 3311268
    dev.em.2.mac_stats.tx_frames_65_127: 402284
    dev.em.2.mac_stats.tx_frames_128_255: 129308
    dev.em.2.mac_stats.tx_frames_256_511: 25539
    dev.em.2.mac_stats.tx_frames_512_1023: 92406
    dev.em.2.mac_stats.tx_frames_1024_1522: 1201109
    dev.em.2.mac_stats.tso_txd: 0
    dev.em.2.mac_stats.tso_ctx_fail: 0
    dev.em.2.interrupts.asserts: 41
    dev.em.2.interrupts.rx_pkt_timer: 0
    dev.em.2.interrupts.rx_abs_timer: 0
    dev.em.2.interrupts.tx_pkt_timer: 0
    dev.em.2.interrupts.tx_abs_timer: 0
    dev.em.2.interrupts.tx_queue_empty: 0
    dev.em.2.interrupts.tx_queue_min_thresh: 0
    dev.em.2.interrupts.rx_desc_min_thresh: 0
    dev.em.2.interrupts.rx_overrun: 0
    dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.3.%driver: em
    dev.em.3.%location: slot=0 function=0
    dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.3.%parent: pci11
    dev.em.3.nvm: -1
    dev.em.3.debug: -1
    dev.em.3.fc: 3
    dev.em.3.rx_int_delay: 0
    dev.em.3.tx_int_delay: 66
    dev.em.3.rx_abs_int_delay: 66
    dev.em.3.tx_abs_int_delay: 66
    dev.em.3.itr: 488
    dev.em.3.rx_processing_limit: 100
    dev.em.3.eee_control: 1
    dev.em.3.link_irq: 13
    dev.em.3.mbuf_alloc_fail: 0
    dev.em.3.cluster_alloc_fail: 0
    dev.em.3.dropped: 0
    dev.em.3.tx_dma_fail: 0
    dev.em.3.rx_overruns: 0
    dev.em.3.watchdog_timeouts: 0
    dev.em.3.device_control: 1074790984
    dev.em.3.rx_control: 67141658
    dev.em.3.fc_high_water: 18432
    dev.em.3.fc_low_water: 16932
    dev.em.3.queue0.txd_head: 386
    dev.em.3.queue0.txd_tail: 386
    dev.em.3.queue0.tx_irq: 276
    dev.em.3.queue0.no_desc_avail: 0
    dev.em.3.queue0.rxd_head: 218
    dev.em.3.queue0.rxd_tail: 217
    dev.em.3.queue0.rx_irq: 978
    dev.em.3.mac_stats.excess_coll: 0
    dev.em.3.mac_stats.single_coll: 0
    dev.em.3.mac_stats.multiple_coll: 0
    dev.em.3.mac_stats.late_coll: 0
    dev.em.3.mac_stats.collision_count: 0
    dev.em.3.mac_stats.symbol_errors: 0
    dev.em.3.mac_stats.sequence_errors: 0
    dev.em.3.mac_stats.defer_count: 0
    dev.em.3.mac_stats.missed_packets: 0
    dev.em.3.mac_stats.recv_no_buff: 0
    dev.em.3.mac_stats.recv_undersize: 0
    dev.em.3.mac_stats.recv_fragmented: 0
    dev.em.3.mac_stats.recv_oversize: 0
    dev.em.3.mac_stats.recv_jabber: 0
    dev.em.3.mac_stats.recv_errs: 0
    dev.em.3.mac_stats.crc_errs: 0
    dev.em.3.mac_stats.alignment_errs: 0
    dev.em.3.mac_stats.coll_ext_errs: 0
    dev.em.3.mac_stats.xon_recvd: 0
    dev.em.3.mac_stats.xon_txd: 0
    dev.em.3.mac_stats.xoff_recvd: 0
    dev.em.3.mac_stats.xoff_txd: 0
    dev.em.3.mac_stats.total_pkts_recvd: 979
    dev.em.3.mac_stats.good_pkts_recvd: 979
    dev.em.3.mac_stats.bcast_pkts_recvd: 616
    dev.em.3.mac_stats.mcast_pkts_recvd: 158
    dev.em.3.mac_stats.rx_frames_64: 63
    dev.em.3.mac_stats.rx_frames_65_127: 163
    dev.em.3.mac_stats.rx_frames_128_255: 112
    dev.em.3.mac_stats.rx_frames_256_511: 641
    dev.em.3.mac_stats.rx_frames_512_1023: 0
    dev.em.3.mac_stats.rx_frames_1024_1522: 0
    dev.em.3.mac_stats.good_octets_recvd: 255346
    dev.em.3.mac_stats.good_octets_txd: 55615
    dev.em.3.mac_stats.total_pkts_txd: 303
    dev.em.3.mac_stats.good_pkts_txd: 303
    dev.em.3.mac_stats.bcast_pkts_txd: 79
    dev.em.3.mac_stats.mcast_pkts_txd: 32
    dev.em.3.mac_stats.tx_frames_64: 32
    dev.em.3.mac_stats.tx_frames_65_127: 229
    dev.em.3.mac_stats.tx_frames_128_255: 9
    dev.em.3.mac_stats.tx_frames_256_511: 14
    dev.em.3.mac_stats.tx_frames_512_1023: 1
    dev.em.3.mac_stats.tx_frames_1024_1522: 18
    dev.em.3.mac_stats.tso_txd: 0
    dev.em.3.mac_stats.tso_ctx_fail: 0
    dev.em.3.interrupts.asserts: 9
    dev.em.3.interrupts.rx_pkt_timer: 0
    dev.em.3.interrupts.rx_abs_timer: 0
    dev.em.3.interrupts.tx_pkt_timer: 0
    dev.em.3.interrupts.tx_abs_timer: 0
    dev.em.3.interrupts.tx_queue_empty: 0
    dev.em.3.interrupts.tx_queue_min_thresh: 0
    dev.em.3.interrupts.rx_desc_min_thresh: 0
    dev.em.3.interrupts.rx_overrun: 0
    dev.em.4.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.4.%driver: em
    dev.em.4.%location: slot=0 function=0
    dev.em.4.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.4.%parent: pci15
    dev.em.4.nvm: -1
    dev.em.4.debug: -1
    dev.em.4.fc: 3
    dev.em.4.rx_int_delay: 0
    dev.em.4.tx_int_delay: 66
    dev.em.4.rx_abs_int_delay: 66
    dev.em.4.tx_abs_int_delay: 66
    dev.em.4.itr: 488
    dev.em.4.rx_processing_limit: 100
    dev.em.4.eee_control: 1
    dev.em.4.link_irq: 0
    dev.em.4.mbuf_alloc_fail: 0
    dev.em.4.cluster_alloc_fail: 0
    dev.em.4.dropped: 0
    dev.em.4.tx_dma_fail: 0
    dev.em.4.rx_overruns: 0
    dev.em.4.watchdog_timeouts: 0
    dev.em.4.device_control: 1074790984
    dev.em.4.rx_control: 67141658
    dev.em.4.fc_high_water: 18432
    dev.em.4.fc_low_water: 16932
    dev.em.4.queue0.txd_head: 0
    dev.em.4.queue0.txd_tail: 0
    dev.em.4.queue0.tx_irq: 0
    dev.em.4.queue0.no_desc_avail: 0
    dev.em.4.queue0.rxd_head: 0
    dev.em.4.queue0.rxd_tail: 1023
    dev.em.4.queue0.rx_irq: 0
    dev.em.4.mac_stats.excess_coll: 0
    dev.em.4.mac_stats.single_coll: 0
    dev.em.4.mac_stats.multiple_coll: 0
    dev.em.4.mac_stats.late_coll: 0
    dev.em.4.mac_stats.collision_count: 0
    dev.em.4.mac_stats.symbol_errors: 0
    dev.em.4.mac_stats.sequence_errors: 0
    dev.em.4.mac_stats.defer_count: 0
    dev.em.4.mac_stats.missed_packets: 0
    dev.em.4.mac_stats.recv_no_buff: 0
    dev.em.4.mac_stats.recv_undersize: 0
    dev.em.4.mac_stats.recv_fragmented: 0
    dev.em.4.mac_stats.recv_oversize: 0
    dev.em.4.mac_stats.recv_jabber: 0
    dev.em.4.mac_stats.recv_errs: 0
    dev.em.4.mac_stats.crc_errs: 0
    dev.em.4.mac_stats.alignment_errs: 0
    dev.em.4.mac_stats.coll_ext_errs: 0
    dev.em.4.mac_stats.xon_recvd: 0
    dev.em.4.mac_stats.xon_txd: 0
    dev.em.4.mac_stats.xoff_recvd: 0
    dev.em.4.mac_stats.xoff_txd: 0
    dev.em.4.mac_stats.total_pkts_recvd: 0
    dev.em.4.mac_stats.good_pkts_recvd: 0
    dev.em.4.mac_stats.bcast_pkts_recvd: 0
    dev.em.4.mac_stats.mcast_pkts_recvd: 0
    dev.em.4.mac_stats.rx_frames_64: 0
    dev.em.4.mac_stats.rx_frames_65_127: 0
    dev.em.4.mac_stats.rx_frames_128_255: 0
    dev.em.4.mac_stats.rx_frames_256_511: 0
    dev.em.4.mac_stats.rx_frames_512_1023: 0
    dev.em.4.mac_stats.rx_frames_1024_1522: 0
    dev.em.4.mac_stats.good_octets_recvd: 0
    dev.em.4.mac_stats.good_octets_txd: 0
    dev.em.4.mac_stats.total_pkts_txd: 0
    dev.em.4.mac_stats.good_pkts_txd: 0
    dev.em.4.mac_stats.bcast_pkts_txd: 0
    dev.em.4.mac_stats.mcast_pkts_txd: 0
    dev.em.4.mac_stats.tx_frames_64: 0
    dev.em.4.mac_stats.tx_frames_65_127: 0
    dev.em.4.mac_stats.tx_frames_128_255: 0
    dev.em.4.mac_stats.tx_frames_256_511: 0
    dev.em.4.mac_stats.tx_frames_512_1023: 0
    dev.em.4.mac_stats.tx_frames_1024_1522: 0
    dev.em.4.mac_stats.tso_txd: 0
    dev.em.4.mac_stats.tso_ctx_fail: 0
    dev.em.4.interrupts.asserts: 0
    dev.em.4.interrupts.rx_pkt_timer: 0
    dev.em.4.interrupts.rx_abs_timer: 0
    dev.em.4.interrupts.tx_pkt_timer: 0
    dev.em.4.interrupts.tx_abs_timer: 0
    dev.em.4.interrupts.tx_queue_empty: 0
    dev.em.4.interrupts.tx_queue_min_thresh: 0
    dev.em.4.interrupts.rx_desc_min_thresh: 0
    dev.em.4.interrupts.rx_overrun: 0
    dev.em.5.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.5.%driver: em
    dev.em.5.%location: slot=0 function=0
    dev.em.5.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.5.%parent: pci16
    dev.em.5.nvm: -1
    dev.em.5.debug: -1
    dev.em.5.fc: 3
    dev.em.5.rx_int_delay: 0
    dev.em.5.tx_int_delay: 66
    dev.em.5.rx_abs_int_delay: 66
    dev.em.5.tx_abs_int_delay: 66
    dev.em.5.itr: 488
    dev.em.5.rx_processing_limit: 100
    dev.em.5.eee_control: 1
    dev.em.5.link_irq: 14
    dev.em.5.mbuf_alloc_fail: 0
    dev.em.5.cluster_alloc_fail: 0
    dev.em.5.dropped: 0
    dev.em.5.tx_dma_fail: 0
    dev.em.5.rx_overruns: 0
    dev.em.5.watchdog_timeouts: 0
    dev.em.5.device_control: 1477444168
    dev.em.5.rx_control: 67141658
    dev.em.5.fc_high_water: 18432
    dev.em.5.fc_low_water: 16932
    dev.em.5.queue0.txd_head: 118
    dev.em.5.queue0.txd_tail: 118
    dev.em.5.queue0.tx_irq: 35021
    dev.em.5.queue0.no_desc_avail: 0
    dev.em.5.queue0.rxd_head: 813
    dev.em.5.queue0.rxd_tail: 812
    dev.em.5.queue0.rx_irq: 33522
    dev.em.5.mac_stats.excess_coll: 0
    dev.em.5.mac_stats.single_coll: 0
    dev.em.5.mac_stats.multiple_coll: 0
    dev.em.5.mac_stats.late_coll: 0
    dev.em.5.mac_stats.collision_count: 0
    dev.em.5.mac_stats.symbol_errors: 0
    dev.em.5.mac_stats.sequence_errors: 0
    dev.em.5.mac_stats.defer_count: 0
    dev.em.5.mac_stats.missed_packets: 0
    dev.em.5.mac_stats.recv_no_buff: 0
    dev.em.5.mac_stats.recv_undersize: 0
    dev.em.5.mac_stats.recv_fragmented: 0
    dev.em.5.mac_stats.recv_oversize: 0
    dev.em.5.mac_stats.recv_jabber: 0
    dev.em.5.mac_stats.recv_errs: 0
    dev.em.5.mac_stats.crc_errs: 0
    dev.em.5.mac_stats.alignment_errs: 0
    dev.em.5.mac_stats.coll_ext_errs: 0
    dev.em.5.mac_stats.xon_recvd: 0
    dev.em.5.mac_stats.xon_txd: 0
    dev.em.5.mac_stats.xoff_recvd: 0
    dev.em.5.mac_stats.xoff_txd: 0
    dev.em.5.mac_stats.total_pkts_recvd: 33581
    dev.em.5.mac_stats.good_pkts_recvd: 33581
    dev.em.5.mac_stats.bcast_pkts_recvd: 13190
    dev.em.5.mac_stats.mcast_pkts_recvd: 6958
    dev.em.5.mac_stats.rx_frames_64: 13832
    dev.em.5.mac_stats.rx_frames_65_127: 10806
    dev.em.5.mac_stats.rx_frames_128_255: 408
    dev.em.5.mac_stats.rx_frames_256_511: 7058
    dev.em.5.mac_stats.rx_frames_512_1023: 413
    dev.em.5.mac_stats.rx_frames_1024_1522: 1064
    dev.em.5.mac_stats.good_octets_recvd: 6533199
    dev.em.5.mac_stats.good_octets_txd: 5861382
    dev.em.5.mac_stats.total_pkts_txd: 35811
    dev.em.5.mac_stats.good_pkts_txd: 35811
    dev.em.5.mac_stats.bcast_pkts_txd: 8074
    dev.em.5.mac_stats.mcast_pkts_txd: 2154
    dev.em.5.mac_stats.tx_frames_64: 17797
    dev.em.5.mac_stats.tx_frames_65_127: 14125
    dev.em.5.mac_stats.tx_frames_128_255: 859
    dev.em.5.mac_stats.tx_frames_256_511: 716
    dev.em.5.mac_stats.tx_frames_512_1023: 294
    dev.em.5.mac_stats.tx_frames_1024_1522: 2020
    dev.em.5.mac_stats.tso_txd: 0
    dev.em.5.mac_stats.tso_ctx_fail: 0
    dev.em.5.interrupts.asserts: 9
    dev.em.5.interrupts.rx_pkt_timer: 0
    dev.em.5.interrupts.rx_abs_timer: 0
    dev.em.5.interrupts.tx_pkt_timer: 0
    dev.em.5.interrupts.tx_abs_timer: 0
    dev.em.5.interrupts.tx_queue_empty: 0
    dev.em.5.interrupts.tx_queue_min_thresh: 0
    dev.em.5.interrupts.rx_desc_min_thresh: 0
    dev.em.5.interrupts.rx_overrun: 0
    dev.em.6.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.6.%driver: em
    dev.em.6.%location: slot=0 function=0
    dev.em.6.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.6.%parent: pci17
    dev.em.6.nvm: -1
    dev.em.6.debug: -1
    dev.em.6.fc: 3
    dev.em.6.rx_int_delay: 0
    dev.em.6.tx_int_delay: 66
    dev.em.6.rx_abs_int_delay: 66
    dev.em.6.tx_abs_int_delay: 66
    dev.em.6.itr: 488
    dev.em.6.rx_processing_limit: 100
    dev.em.6.eee_control: 1
    dev.em.6.link_irq: 22
    dev.em.6.mbuf_alloc_fail: 0
    dev.em.6.cluster_alloc_fail: 0
    dev.em.6.dropped: 0
    dev.em.6.tx_dma_fail: 0
    dev.em.6.rx_overruns: 0
    dev.em.6.watchdog_timeouts: 0
    dev.em.6.device_control: 1477444168
    dev.em.6.rx_control: 67141658
    dev.em.6.fc_high_water: 18432
    dev.em.6.fc_low_water: 16932
    dev.em.6.queue0.txd_head: 172
    dev.em.6.queue0.txd_tail: 172
    dev.em.6.queue0.tx_irq: 86612
    dev.em.6.queue0.no_desc_avail: 0
    dev.em.6.queue0.rxd_head: 886
    dev.em.6.queue0.rxd_tail: 885
    dev.em.6.queue0.rx_irq: 37129
    dev.em.6.mac_stats.excess_coll: 0
    dev.em.6.mac_stats.single_coll: 0
    dev.em.6.mac_stats.multiple_coll: 0
    dev.em.6.mac_stats.late_coll: 0
    dev.em.6.mac_stats.collision_count: 0
    dev.em.6.mac_stats.symbol_errors: 0
    dev.em.6.mac_stats.sequence_errors: 0
    dev.em.6.mac_stats.defer_count: 0
    dev.em.6.mac_stats.missed_packets: 0
    dev.em.6.mac_stats.recv_no_buff: 0
    dev.em.6.mac_stats.recv_undersize: 0
    dev.em.6.mac_stats.recv_fragmented: 0
    dev.em.6.mac_stats.recv_oversize: 0
    dev.em.6.mac_stats.recv_jabber: 0
    dev.em.6.mac_stats.recv_errs: 0
    dev.em.6.mac_stats.crc_errs: 0
    dev.em.6.mac_stats.alignment_errs: 0
    dev.em.6.mac_stats.coll_ext_errs: 0
    dev.em.6.mac_stats.xon_recvd: 0
    dev.em.6.mac_stats.xon_txd: 0
    dev.em.6.mac_stats.xoff_recvd: 0
    dev.em.6.mac_stats.xoff_txd: 0
    dev.em.6.mac_stats.total_pkts_recvd: 40822
    dev.em.6.mac_stats.good_pkts_recvd: 40822
    dev.em.6.mac_stats.bcast_pkts_recvd: 538
    dev.em.6.mac_stats.mcast_pkts_recvd: 445
    dev.em.6.mac_stats.rx_frames_64: 28322
    dev.em.6.mac_stats.rx_frames_65_127: 8188
    dev.em.6.mac_stats.rx_frames_128_255: 966
    dev.em.6.mac_stats.rx_frames_256_511: 818
    dev.em.6.mac_stats.rx_frames_512_1023: 957
    dev.em.6.mac_stats.rx_frames_1024_1522: 1571
    dev.em.6.mac_stats.good_octets_recvd: 6040146
    dev.em.6.mac_stats.good_octets_txd: 88048854
    dev.em.6.mac_stats.total_pkts_txd: 93690
    dev.em.6.mac_stats.good_pkts_txd: 93690
    dev.em.6.mac_stats.bcast_pkts_txd: 20785
    dev.em.6.mac_stats.mcast_pkts_txd: 8747
    dev.em.6.mac_stats.tx_frames_64: 19296
    dev.em.6.mac_stats.tx_frames_65_127: 9220
    dev.em.6.mac_stats.tx_frames_128_255: 2549
    dev.em.6.mac_stats.tx_frames_256_511: 7998
    dev.em.6.mac_stats.tx_frames_512_1023: 746
    dev.em.6.mac_stats.tx_frames_1024_1522: 53881
    dev.em.6.mac_stats.tso_txd: 0
    dev.em.6.mac_stats.tso_ctx_fail: 0
    dev.em.6.interrupts.asserts: 14
    dev.em.6.interrupts.rx_pkt_timer: 0
    dev.em.6.interrupts.rx_abs_timer: 0
    dev.em.6.interrupts.tx_pkt_timer: 0
    dev.em.6.interrupts.tx_abs_timer: 0
    dev.em.6.interrupts.tx_queue_empty: 0
    dev.em.6.interrupts.tx_queue_min_thresh: 0
    dev.em.6.interrupts.rx_desc_min_thresh: 0
    dev.em.6.interrupts.rx_overrun: 0
    dev.em.7.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.7.%driver: em
    dev.em.7.%location: slot=0 function=0
    dev.em.7.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.7.%parent: pci18
    dev.em.7.nvm: -1
    dev.em.7.debug: -1
    dev.em.7.fc: 3
    dev.em.7.rx_int_delay: 0
    dev.em.7.tx_int_delay: 66
    dev.em.7.rx_abs_int_delay: 66
    dev.em.7.tx_abs_int_delay: 66
    dev.em.7.itr: 488
    dev.em.7.rx_processing_limit: 100
    dev.em.7.eee_control: 1
    dev.em.7.link_irq: 0
    dev.em.7.mbuf_alloc_fail: 0
    dev.em.7.cluster_alloc_fail: 0
    dev.em.7.dropped: 0
    dev.em.7.tx_dma_fail: 0
    dev.em.7.rx_overruns: 0
    dev.em.7.watchdog_timeouts: 0
    dev.em.7.device_control: 1074790984
    dev.em.7.rx_control: 67141658
    dev.em.7.fc_high_water: 18432
    dev.em.7.fc_low_water: 16932
    dev.em.7.queue0.txd_head: 0
    dev.em.7.queue0.txd_tail: 0
    dev.em.7.queue0.tx_irq: 0
    dev.em.7.queue0.no_desc_avail: 0
    dev.em.7.queue0.rxd_head: 0
    dev.em.7.queue0.rxd_tail: 1023
    dev.em.7.queue0.rx_irq: 0
    dev.em.7.mac_stats.excess_coll: 0
    dev.em.7.mac_stats.single_coll: 0
    dev.em.7.mac_stats.multiple_coll: 0
    dev.em.7.mac_stats.late_coll: 0
    dev.em.7.mac_stats.collision_count: 0
    dev.em.7.mac_stats.symbol_errors: 0
    dev.em.7.mac_stats.sequence_errors: 0
    dev.em.7.mac_stats.defer_count: 0
    dev.em.7.mac_stats.missed_packets: 0
    dev.em.7.mac_stats.recv_no_buff: 0
    dev.em.7.mac_stats.recv_undersize: 0
    dev.em.7.mac_stats.recv_fragmented: 0
    dev.em.7.mac_stats.recv_oversize: 0
    dev.em.7.mac_stats.recv_jabber: 0
    dev.em.7.mac_stats.recv_errs: 0
    dev.em.7.mac_stats.crc_errs: 0
    dev.em.7.mac_stats.alignment_errs: 0
    dev.em.7.mac_stats.coll_ext_errs: 0
    dev.em.7.mac_stats.xon_recvd: 0
    dev.em.7.mac_stats.xon_txd: 0
    dev.em.7.mac_stats.xoff_recvd: 0
    dev.em.7.mac_stats.xoff_txd: 0
    dev.em.7.mac_stats.total_pkts_recvd: 0
    dev.em.7.mac_stats.good_pkts_recvd: 0
    dev.em.7.mac_stats.bcast_pkts_recvd: 0
    dev.em.7.mac_stats.mcast_pkts_recvd: 0
    dev.em.7.mac_stats.rx_frames_64: 0
    dev.em.7.mac_stats.rx_frames_65_127: 0
    dev.em.7.mac_stats.rx_frames_128_255: 0
    dev.em.7.mac_stats.rx_frames_256_511: 0
    dev.em.7.mac_stats.rx_frames_512_1023: 0
    dev.em.7.mac_stats.rx_frames_1024_1522: 0
    dev.em.7.mac_stats.good_octets_recvd: 0
    dev.em.7.mac_stats.good_octets_txd: 0
    dev.em.7.mac_stats.total_pkts_txd: 0
    dev.em.7.mac_stats.good_pkts_txd: 0
    dev.em.7.mac_stats.bcast_pkts_txd: 0
    dev.em.7.mac_stats.mcast_pkts_txd: 0
    dev.em.7.mac_stats.tx_frames_64: 0
    dev.em.7.mac_stats.tx_frames_65_127: 0
    dev.em.7.mac_stats.tx_frames_128_255: 0
    dev.em.7.mac_stats.tx_frames_256_511: 0
    dev.em.7.mac_stats.tx_frames_512_1023: 0
    dev.em.7.mac_stats.tx_frames_1024_1522: 0
    dev.em.7.mac_stats.tso_txd: 0
    dev.em.7.mac_stats.tso_ctx_fail: 0
    dev.em.7.interrupts.asserts: 0
    dev.em.7.interrupts.rx_pkt_timer: 0
    dev.em.7.interrupts.rx_abs_timer: 0
    dev.em.7.interrupts.tx_pkt_timer: 0
    dev.em.7.interrupts.tx_abs_timer: 0
    dev.em.7.interrupts.tx_queue_empty: 0
    dev.em.7.interrupts.tx_queue_min_thresh: 0
    dev.em.7.interrupts.rx_desc_min_thresh: 0
    dev.em.7.interrupts.rx_overrun: 0
    

    And this is shortly after, having revived the connection using ifconfig:

    $ sysctl dev.em
    dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.0.%driver: em
    dev.em.0.%location: slot=0 function=0
    dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.0.%parent: pci5
    dev.em.0.nvm: -1
    dev.em.0.debug: -1
    dev.em.0.fc: 3
    dev.em.0.rx_int_delay: 0
    dev.em.0.tx_int_delay: 66
    dev.em.0.rx_abs_int_delay: 66
    dev.em.0.tx_abs_int_delay: 66
    dev.em.0.itr: 488
    dev.em.0.rx_processing_limit: 100
    dev.em.0.eee_control: 1
    dev.em.0.link_irq: 3
    dev.em.0.mbuf_alloc_fail: 0
    dev.em.0.cluster_alloc_fail: 0
    dev.em.0.dropped: 0
    dev.em.0.tx_dma_fail: 0
    dev.em.0.rx_overruns: 0
    dev.em.0.watchdog_timeouts: 0
    dev.em.0.device_control: 1074790984
    dev.em.0.rx_control: 67141658
    dev.em.0.fc_high_water: 18432
    dev.em.0.fc_low_water: 16932
    dev.em.0.queue0.txd_head: 685
    dev.em.0.queue0.txd_tail: 685
    dev.em.0.queue0.tx_irq: 1353736
    dev.em.0.queue0.no_desc_avail: 0
    dev.em.0.queue0.rxd_head: 676
    dev.em.0.queue0.rxd_tail: 675
    dev.em.0.queue0.rx_irq: 2196823
    dev.em.0.mac_stats.excess_coll: 0
    dev.em.0.mac_stats.single_coll: 0
    dev.em.0.mac_stats.multiple_coll: 0
    dev.em.0.mac_stats.late_coll: 0
    dev.em.0.mac_stats.collision_count: 0
    dev.em.0.mac_stats.symbol_errors: 0
    dev.em.0.mac_stats.sequence_errors: 0
    dev.em.0.mac_stats.defer_count: 0
    dev.em.0.mac_stats.missed_packets: 0
    dev.em.0.mac_stats.recv_no_buff: 0
    dev.em.0.mac_stats.recv_undersize: 0
    dev.em.0.mac_stats.recv_fragmented: 0
    dev.em.0.mac_stats.recv_oversize: 14
    dev.em.0.mac_stats.recv_jabber: 0
    dev.em.0.mac_stats.recv_errs: 1
    dev.em.0.mac_stats.crc_errs: 1864
    dev.em.0.mac_stats.alignment_errs: 0
    dev.em.0.mac_stats.coll_ext_errs: 0
    dev.em.0.mac_stats.xon_recvd: 0
    dev.em.0.mac_stats.xon_txd: 0
    dev.em.0.mac_stats.xoff_recvd: 0
    dev.em.0.mac_stats.xoff_txd: 0
    dev.em.0.mac_stats.total_pkts_recvd: 2695671
    dev.em.0.mac_stats.good_pkts_recvd: 2693793
    dev.em.0.mac_stats.bcast_pkts_recvd: 115211
    dev.em.0.mac_stats.mcast_pkts_recvd: 170573
    dev.em.0.mac_stats.rx_frames_64: 231428
    dev.em.0.mac_stats.rx_frames_65_127: 881736
    dev.em.0.mac_stats.rx_frames_128_255: 240428
    dev.em.0.mac_stats.rx_frames_256_511: 127402
    dev.em.0.mac_stats.rx_frames_512_1023: 43918
    dev.em.0.mac_stats.rx_frames_1024_1522: 1168881
    dev.em.0.mac_stats.good_octets_recvd: 1944697083
    dev.em.0.mac_stats.good_octets_txd: 267511513
    dev.em.0.mac_stats.total_pkts_txd: 2302538
    dev.em.0.mac_stats.good_pkts_txd: 2302538
    dev.em.0.mac_stats.bcast_pkts_txd: 21
    dev.em.0.mac_stats.mcast_pkts_txd: 5
    dev.em.0.mac_stats.tx_frames_64: 401423
    dev.em.0.mac_stats.tx_frames_65_127: 1702538
    dev.em.0.mac_stats.tx_frames_128_255: 59586
    dev.em.0.mac_stats.tx_frames_256_511: 57937
    dev.em.0.mac_stats.tx_frames_512_1023: 37227
    dev.em.0.mac_stats.tx_frames_1024_1522: 43827
    dev.em.0.mac_stats.tso_txd: 0
    dev.em.0.mac_stats.tso_ctx_fail: 0
    dev.em.0.interrupts.asserts: 5
    dev.em.0.interrupts.rx_pkt_timer: 0
    dev.em.0.interrupts.rx_abs_timer: 0
    dev.em.0.interrupts.tx_pkt_timer: 0
    dev.em.0.interrupts.tx_abs_timer: 0
    dev.em.0.interrupts.tx_queue_empty: 0
    dev.em.0.interrupts.tx_queue_min_thresh: 0
    dev.em.0.interrupts.rx_desc_min_thresh: 0
    dev.em.0.interrupts.rx_overrun: 0
    dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.1.%driver: em
    dev.em.1.%location: slot=0 function=0
    dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.1.%parent: pci6
    dev.em.1.nvm: -1
    dev.em.1.debug: -1
    dev.em.1.fc: 3
    dev.em.1.rx_int_delay: 0
    dev.em.1.tx_int_delay: 66
    dev.em.1.rx_abs_int_delay: 66
    dev.em.1.tx_abs_int_delay: 66
    dev.em.1.itr: 488
    dev.em.1.rx_processing_limit: 100
    dev.em.1.eee_control: 1
    dev.em.1.link_irq: 22
    dev.em.1.mbuf_alloc_fail: 0
    dev.em.1.cluster_alloc_fail: 0
    dev.em.1.dropped: 0
    dev.em.1.tx_dma_fail: 0
    dev.em.1.rx_overruns: 0
    dev.em.1.watchdog_timeouts: 0
    dev.em.1.device_control: 1477444168
    dev.em.1.rx_control: 67141658
    dev.em.1.fc_high_water: 18432
    dev.em.1.fc_low_water: 16932
    dev.em.1.queue0.txd_head: 839
    dev.em.1.queue0.txd_tail: 839
    dev.em.1.queue0.tx_irq: 1691646
    dev.em.1.queue0.no_desc_avail: 0
    dev.em.1.queue0.rxd_head: 100
    dev.em.1.queue0.rxd_tail: 99
    dev.em.1.queue0.rx_irq: 1381613
    dev.em.1.mac_stats.excess_coll: 0
    dev.em.1.mac_stats.single_coll: 0
    dev.em.1.mac_stats.multiple_coll: 0
    dev.em.1.mac_stats.late_coll: 0
    dev.em.1.mac_stats.collision_count: 0
    dev.em.1.mac_stats.symbol_errors: 0
    dev.em.1.mac_stats.sequence_errors: 0
    dev.em.1.mac_stats.defer_count: 0
    dev.em.1.mac_stats.missed_packets: 0
    dev.em.1.mac_stats.recv_no_buff: 0
    dev.em.1.mac_stats.recv_undersize: 0
    dev.em.1.mac_stats.recv_fragmented: 0
    dev.em.1.mac_stats.recv_oversize: 0
    dev.em.1.mac_stats.recv_jabber: 0
    dev.em.1.mac_stats.recv_errs: 0
    dev.em.1.mac_stats.crc_errs: 0
    dev.em.1.mac_stats.alignment_errs: 0
    dev.em.1.mac_stats.coll_ext_errs: 0
    dev.em.1.mac_stats.xon_recvd: 0
    dev.em.1.mac_stats.xon_txd: 0
    dev.em.1.mac_stats.xoff_recvd: 0
    dev.em.1.mac_stats.xoff_txd: 0
    dev.em.1.mac_stats.total_pkts_recvd: 6059107
    dev.em.1.mac_stats.good_pkts_recvd: 6059107
    dev.em.1.mac_stats.bcast_pkts_recvd: 1003
    dev.em.1.mac_stats.mcast_pkts_recvd: 36
    dev.em.1.mac_stats.rx_frames_64: 3395204
    dev.em.1.mac_stats.rx_frames_65_127: 1584944
    dev.em.1.mac_stats.rx_frames_128_255: 146061
    dev.em.1.mac_stats.rx_frames_256_511: 68733
    dev.em.1.mac_stats.rx_frames_512_1023: 57333
    dev.em.1.mac_stats.rx_frames_1024_1522: 806832
    dev.em.1.mac_stats.good_octets_recvd: 1644015882
    dev.em.1.mac_stats.good_octets_txd: 10934440772
    dev.em.1.mac_stats.total_pkts_txd: 8619016
    dev.em.1.mac_stats.good_pkts_txd: 8619016
    dev.em.1.mac_stats.bcast_pkts_txd: 20889
    dev.em.1.mac_stats.mcast_pkts_txd: 9162
    dev.em.1.mac_stats.tx_frames_64: 439143
    dev.em.1.mac_stats.tx_frames_65_127: 621925
    dev.em.1.mac_stats.tx_frames_128_255: 326546
    dev.em.1.mac_stats.tx_frames_256_511: 105432
    dev.em.1.mac_stats.tx_frames_512_1023: 44657
    dev.em.1.mac_stats.tx_frames_1024_1522: 7081313
    dev.em.1.mac_stats.tso_txd: 0
    dev.em.1.mac_stats.tso_ctx_fail: 0
    dev.em.1.interrupts.asserts: 13
    dev.em.1.interrupts.rx_pkt_timer: 0
    dev.em.1.interrupts.rx_abs_timer: 0
    dev.em.1.interrupts.tx_pkt_timer: 0
    dev.em.1.interrupts.tx_abs_timer: 0
    dev.em.1.interrupts.tx_queue_empty: 0
    dev.em.1.interrupts.tx_queue_min_thresh: 0
    dev.em.1.interrupts.rx_desc_min_thresh: 0
    dev.em.1.interrupts.rx_overrun: 0
    dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.2.%driver: em
    dev.em.2.%location: slot=0 function=0
    dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.2.%parent: pci10
    dev.em.2.nvm: -1
    dev.em.2.debug: -1
    dev.em.2.fc: 3
    dev.em.2.rx_int_delay: 0
    dev.em.2.tx_int_delay: 66
    dev.em.2.rx_abs_int_delay: 66
    dev.em.2.tx_abs_int_delay: 66
    dev.em.2.itr: 488
    dev.em.2.rx_processing_limit: 100
    dev.em.2.eee_control: 1
    dev.em.2.link_irq: 69
    dev.em.2.mbuf_alloc_fail: 0
    dev.em.2.cluster_alloc_fail: 0
    dev.em.2.dropped: 0
    dev.em.2.tx_dma_fail: 0
    dev.em.2.rx_overruns: 0
    dev.em.2.watchdog_timeouts: 0
    dev.em.2.device_control: 1477444168
    dev.em.2.rx_control: 67141658
    dev.em.2.fc_high_water: 18432
    dev.em.2.fc_low_water: 16932
    dev.em.2.queue0.txd_head: 240
    dev.em.2.queue0.txd_tail: 240
    dev.em.2.queue0.tx_irq: 1314599
    dev.em.2.queue0.no_desc_avail: 4
    dev.em.2.queue0.rxd_head: 293
    dev.em.2.queue0.rxd_tail: 292
    dev.em.2.queue0.rx_irq: 1026038
    dev.em.2.mac_stats.excess_coll: 0
    dev.em.2.mac_stats.single_coll: 0
    dev.em.2.mac_stats.multiple_coll: 0
    dev.em.2.mac_stats.late_coll: 0
    dev.em.2.mac_stats.collision_count: 0
    dev.em.2.mac_stats.symbol_errors: 0
    dev.em.2.mac_stats.sequence_errors: 0
    dev.em.2.mac_stats.defer_count: 0
    dev.em.2.mac_stats.missed_packets: 0
    dev.em.2.mac_stats.recv_no_buff: 0
    dev.em.2.mac_stats.recv_undersize: 0
    dev.em.2.mac_stats.recv_fragmented: 0
    dev.em.2.mac_stats.recv_oversize: 0
    dev.em.2.mac_stats.recv_jabber: 0
    dev.em.2.mac_stats.recv_errs: 0
    dev.em.2.mac_stats.crc_errs: 0
    dev.em.2.mac_stats.alignment_errs: 0
    dev.em.2.mac_stats.coll_ext_errs: 0
    dev.em.2.mac_stats.xon_recvd: 0
    dev.em.2.mac_stats.xon_txd: 67
    dev.em.2.mac_stats.xoff_recvd: 0
    dev.em.2.mac_stats.xoff_txd: 67
    dev.em.2.mac_stats.total_pkts_recvd: 7579992
    dev.em.2.mac_stats.good_pkts_recvd: 7579992
    dev.em.2.mac_stats.bcast_pkts_recvd: 4530
    dev.em.2.mac_stats.mcast_pkts_recvd: 1960
    dev.em.2.mac_stats.rx_frames_64: 733376
    dev.em.2.mac_stats.rx_frames_65_127: 127545
    dev.em.2.mac_stats.rx_frames_128_255: 236524
    dev.em.2.mac_stats.rx_frames_256_511: 32859
    dev.em.2.mac_stats.rx_frames_512_1023: 70613
    dev.em.2.mac_stats.rx_frames_1024_1522: 6379075
    dev.em.2.mac_stats.good_octets_recvd: 9847706051
    dev.em.2.mac_stats.good_octets_txd: 2161839705
    dev.em.2.mac_stats.total_pkts_txd: 5162265
    dev.em.2.mac_stats.good_pkts_txd: 5162131
    dev.em.2.mac_stats.bcast_pkts_txd: 5846
    dev.em.2.mac_stats.mcast_pkts_txd: 3858
    dev.em.2.mac_stats.tx_frames_64: 3311306
    dev.em.2.mac_stats.tx_frames_65_127: 402438
    dev.em.2.mac_stats.tx_frames_128_255: 129314
    dev.em.2.mac_stats.tx_frames_256_511: 25549
    dev.em.2.mac_stats.tx_frames_512_1023: 92410
    dev.em.2.mac_stats.tx_frames_1024_1522: 1201114
    dev.em.2.mac_stats.tso_txd: 0
    dev.em.2.mac_stats.tso_ctx_fail: 0
    dev.em.2.interrupts.asserts: 44
    dev.em.2.interrupts.rx_pkt_timer: 0
    dev.em.2.interrupts.rx_abs_timer: 0
    dev.em.2.interrupts.tx_pkt_timer: 0
    dev.em.2.interrupts.tx_abs_timer: 0
    dev.em.2.interrupts.tx_queue_empty: 0
    dev.em.2.interrupts.tx_queue_min_thresh: 0
    dev.em.2.interrupts.rx_desc_min_thresh: 0
    dev.em.2.interrupts.rx_overrun: 0
    dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.3.%driver: em
    dev.em.3.%location: slot=0 function=0
    dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.3.%parent: pci11
    dev.em.3.nvm: -1
    dev.em.3.debug: -1
    dev.em.3.fc: 3
    dev.em.3.rx_int_delay: 0
    dev.em.3.tx_int_delay: 66
    dev.em.3.rx_abs_int_delay: 66
    dev.em.3.tx_abs_int_delay: 66
    dev.em.3.itr: 488
    dev.em.3.rx_processing_limit: 100
    dev.em.3.eee_control: 1
    dev.em.3.link_irq: 13
    dev.em.3.mbuf_alloc_fail: 0
    dev.em.3.cluster_alloc_fail: 0
    dev.em.3.dropped: 0
    dev.em.3.tx_dma_fail: 0
    dev.em.3.rx_overruns: 0
    dev.em.3.watchdog_timeouts: 0
    dev.em.3.device_control: 1074790984
    dev.em.3.rx_control: 67141658
    dev.em.3.fc_high_water: 18432
    dev.em.3.fc_low_water: 16932
    dev.em.3.queue0.txd_head: 386
    dev.em.3.queue0.txd_tail: 386
    dev.em.3.queue0.tx_irq: 276
    dev.em.3.queue0.no_desc_avail: 0
    dev.em.3.queue0.rxd_head: 218
    dev.em.3.queue0.rxd_tail: 217
    dev.em.3.queue0.rx_irq: 978
    dev.em.3.mac_stats.excess_coll: 0
    dev.em.3.mac_stats.single_coll: 0
    dev.em.3.mac_stats.multiple_coll: 0
    dev.em.3.mac_stats.late_coll: 0
    dev.em.3.mac_stats.collision_count: 0
    dev.em.3.mac_stats.symbol_errors: 0
    dev.em.3.mac_stats.sequence_errors: 0
    dev.em.3.mac_stats.defer_count: 0
    dev.em.3.mac_stats.missed_packets: 0
    dev.em.3.mac_stats.recv_no_buff: 0
    dev.em.3.mac_stats.recv_undersize: 0
    dev.em.3.mac_stats.recv_fragmented: 0
    dev.em.3.mac_stats.recv_oversize: 0
    dev.em.3.mac_stats.recv_jabber: 0
    dev.em.3.mac_stats.recv_errs: 0
    dev.em.3.mac_stats.crc_errs: 0
    dev.em.3.mac_stats.alignment_errs: 0
    dev.em.3.mac_stats.coll_ext_errs: 0
    dev.em.3.mac_stats.xon_recvd: 0
    dev.em.3.mac_stats.xon_txd: 0
    dev.em.3.mac_stats.xoff_recvd: 0
    dev.em.3.mac_stats.xoff_txd: 0
    dev.em.3.mac_stats.total_pkts_recvd: 979
    dev.em.3.mac_stats.good_pkts_recvd: 979
    dev.em.3.mac_stats.bcast_pkts_recvd: 616
    dev.em.3.mac_stats.mcast_pkts_recvd: 158
    dev.em.3.mac_stats.rx_frames_64: 63
    dev.em.3.mac_stats.rx_frames_65_127: 163
    dev.em.3.mac_stats.rx_frames_128_255: 112
    dev.em.3.mac_stats.rx_frames_256_511: 641
    dev.em.3.mac_stats.rx_frames_512_1023: 0
    dev.em.3.mac_stats.rx_frames_1024_1522: 0
    dev.em.3.mac_stats.good_octets_recvd: 255346
    dev.em.3.mac_stats.good_octets_txd: 55615
    dev.em.3.mac_stats.total_pkts_txd: 303
    dev.em.3.mac_stats.good_pkts_txd: 303
    dev.em.3.mac_stats.bcast_pkts_txd: 79
    dev.em.3.mac_stats.mcast_pkts_txd: 32
    dev.em.3.mac_stats.tx_frames_64: 32
    dev.em.3.mac_stats.tx_frames_65_127: 229
    dev.em.3.mac_stats.tx_frames_128_255: 9
    dev.em.3.mac_stats.tx_frames_256_511: 14
    dev.em.3.mac_stats.tx_frames_512_1023: 1
    dev.em.3.mac_stats.tx_frames_1024_1522: 18
    dev.em.3.mac_stats.tso_txd: 0
    dev.em.3.mac_stats.tso_ctx_fail: 0
    dev.em.3.interrupts.asserts: 9
    dev.em.3.interrupts.rx_pkt_timer: 0
    dev.em.3.interrupts.rx_abs_timer: 0
    dev.em.3.interrupts.tx_pkt_timer: 0
    dev.em.3.interrupts.tx_abs_timer: 0
    dev.em.3.interrupts.tx_queue_empty: 0
    dev.em.3.interrupts.tx_queue_min_thresh: 0
    dev.em.3.interrupts.rx_desc_min_thresh: 0
    dev.em.3.interrupts.rx_overrun: 0
    dev.em.4.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.4.%driver: em
    dev.em.4.%location: slot=0 function=0
    dev.em.4.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.4.%parent: pci15
    dev.em.4.nvm: -1
    dev.em.4.debug: -1
    dev.em.4.fc: 3
    dev.em.4.rx_int_delay: 0
    dev.em.4.tx_int_delay: 66
    dev.em.4.rx_abs_int_delay: 66
    dev.em.4.tx_abs_int_delay: 66
    dev.em.4.itr: 488
    dev.em.4.rx_processing_limit: 100
    dev.em.4.eee_control: 1
    dev.em.4.link_irq: 0
    dev.em.4.mbuf_alloc_fail: 0
    dev.em.4.cluster_alloc_fail: 0
    dev.em.4.dropped: 0
    dev.em.4.tx_dma_fail: 0
    dev.em.4.rx_overruns: 0
    dev.em.4.watchdog_timeouts: 0
    dev.em.4.device_control: 1074790984
    dev.em.4.rx_control: 67141658
    dev.em.4.fc_high_water: 18432
    dev.em.4.fc_low_water: 16932
    dev.em.4.queue0.txd_head: 0
    dev.em.4.queue0.txd_tail: 0
    dev.em.4.queue0.tx_irq: 0
    dev.em.4.queue0.no_desc_avail: 0
    dev.em.4.queue0.rxd_head: 0
    dev.em.4.queue0.rxd_tail: 1023
    dev.em.4.queue0.rx_irq: 0
    dev.em.4.mac_stats.excess_coll: 0
    dev.em.4.mac_stats.single_coll: 0
    dev.em.4.mac_stats.multiple_coll: 0
    dev.em.4.mac_stats.late_coll: 0
    dev.em.4.mac_stats.collision_count: 0
    dev.em.4.mac_stats.symbol_errors: 0
    dev.em.4.mac_stats.sequence_errors: 0
    dev.em.4.mac_stats.defer_count: 0
    dev.em.4.mac_stats.missed_packets: 0
    dev.em.4.mac_stats.recv_no_buff: 0
    dev.em.4.mac_stats.recv_undersize: 0
    dev.em.4.mac_stats.recv_fragmented: 0
    dev.em.4.mac_stats.recv_oversize: 0
    dev.em.4.mac_stats.recv_jabber: 0
    dev.em.4.mac_stats.recv_errs: 0
    dev.em.4.mac_stats.crc_errs: 0
    dev.em.4.mac_stats.alignment_errs: 0
    dev.em.4.mac_stats.coll_ext_errs: 0
    dev.em.4.mac_stats.xon_recvd: 0
    dev.em.4.mac_stats.xon_txd: 0
    dev.em.4.mac_stats.xoff_recvd: 0
    dev.em.4.mac_stats.xoff_txd: 0
    dev.em.4.mac_stats.total_pkts_recvd: 0
    dev.em.4.mac_stats.good_pkts_recvd: 0
    dev.em.4.mac_stats.bcast_pkts_recvd: 0
    dev.em.4.mac_stats.mcast_pkts_recvd: 0
    dev.em.4.mac_stats.rx_frames_64: 0
    dev.em.4.mac_stats.rx_frames_65_127: 0
    dev.em.4.mac_stats.rx_frames_128_255: 0
    dev.em.4.mac_stats.rx_frames_256_511: 0
    dev.em.4.mac_stats.rx_frames_512_1023: 0
    dev.em.4.mac_stats.rx_frames_1024_1522: 0
    dev.em.4.mac_stats.good_octets_recvd: 0
    dev.em.4.mac_stats.good_octets_txd: 0
    dev.em.4.mac_stats.total_pkts_txd: 0
    dev.em.4.mac_stats.good_pkts_txd: 0
    dev.em.4.mac_stats.bcast_pkts_txd: 0
    dev.em.4.mac_stats.mcast_pkts_txd: 0
    dev.em.4.mac_stats.tx_frames_64: 0
    dev.em.4.mac_stats.tx_frames_65_127: 0
    dev.em.4.mac_stats.tx_frames_128_255: 0
    dev.em.4.mac_stats.tx_frames_256_511: 0
    dev.em.4.mac_stats.tx_frames_512_1023: 0
    dev.em.4.mac_stats.tx_frames_1024_1522: 0
    dev.em.4.mac_stats.tso_txd: 0
    dev.em.4.mac_stats.tso_ctx_fail: 0
    dev.em.4.interrupts.asserts: 0
    dev.em.4.interrupts.rx_pkt_timer: 0
    dev.em.4.interrupts.rx_abs_timer: 0
    dev.em.4.interrupts.tx_pkt_timer: 0
    dev.em.4.interrupts.tx_abs_timer: 0
    dev.em.4.interrupts.tx_queue_empty: 0
    dev.em.4.interrupts.tx_queue_min_thresh: 0
    dev.em.4.interrupts.rx_desc_min_thresh: 0
    dev.em.4.interrupts.rx_overrun: 0
    dev.em.5.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.5.%driver: em
    dev.em.5.%location: slot=0 function=0
    dev.em.5.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.5.%parent: pci16
    dev.em.5.nvm: -1
    dev.em.5.debug: -1
    dev.em.5.fc: 3
    dev.em.5.rx_int_delay: 0
    dev.em.5.tx_int_delay: 66
    dev.em.5.rx_abs_int_delay: 66
    dev.em.5.tx_abs_int_delay: 66
    dev.em.5.itr: 488
    dev.em.5.rx_processing_limit: 100
    dev.em.5.eee_control: 1
    dev.em.5.link_irq: 14
    dev.em.5.mbuf_alloc_fail: 0
    dev.em.5.cluster_alloc_fail: 0
    dev.em.5.dropped: 0
    dev.em.5.tx_dma_fail: 0
    dev.em.5.rx_overruns: 0
    dev.em.5.watchdog_timeouts: 0
    dev.em.5.device_control: 1477444168
    dev.em.5.rx_control: 67141658
    dev.em.5.fc_high_water: 18432
    dev.em.5.fc_low_water: 16932
    dev.em.5.queue0.txd_head: 908
    dev.em.5.queue0.txd_tail: 908
    dev.em.5.queue0.tx_irq: 35644
    dev.em.5.queue0.no_desc_avail: 0
    dev.em.5.queue0.rxd_head: 913
    dev.em.5.queue0.rxd_tail: 912
    dev.em.5.queue0.rx_irq: 33622
    dev.em.5.mac_stats.excess_coll: 0
    dev.em.5.mac_stats.single_coll: 0
    dev.em.5.mac_stats.multiple_coll: 0
    dev.em.5.mac_stats.late_coll: 0
    dev.em.5.mac_stats.collision_count: 0
    dev.em.5.mac_stats.symbol_errors: 0
    dev.em.5.mac_stats.sequence_errors: 0
    dev.em.5.mac_stats.defer_count: 0
    dev.em.5.mac_stats.missed_packets: 0
    dev.em.5.mac_stats.recv_no_buff: 0
    dev.em.5.mac_stats.recv_undersize: 0
    dev.em.5.mac_stats.recv_fragmented: 0
    dev.em.5.mac_stats.recv_oversize: 0
    dev.em.5.mac_stats.recv_jabber: 0
    dev.em.5.mac_stats.recv_errs: 0
    dev.em.5.mac_stats.crc_errs: 0
    dev.em.5.mac_stats.alignment_errs: 0
    dev.em.5.mac_stats.coll_ext_errs: 0
    dev.em.5.mac_stats.xon_recvd: 0
    dev.em.5.mac_stats.xon_txd: 0
    dev.em.5.mac_stats.xoff_recvd: 0
    dev.em.5.mac_stats.xoff_txd: 0
    dev.em.5.mac_stats.total_pkts_recvd: 33681
    dev.em.5.mac_stats.good_pkts_recvd: 33681
    dev.em.5.mac_stats.bcast_pkts_recvd: 13265
    dev.em.5.mac_stats.mcast_pkts_recvd: 6979
    dev.em.5.mac_stats.rx_frames_64: 13907
    dev.em.5.mac_stats.rx_frames_65_127: 10809
    dev.em.5.mac_stats.rx_frames_128_255: 408
    dev.em.5.mac_stats.rx_frames_256_511: 7079
    dev.em.5.mac_stats.rx_frames_512_1023: 414
    dev.em.5.mac_stats.rx_frames_1024_1522: 1064
    dev.em.5.mac_stats.good_octets_recvd: 6547683
    dev.em.5.mac_stats.good_octets_txd: 5926006
    dev.em.5.mac_stats.total_pkts_txd: 36476
    dev.em.5.mac_stats.good_pkts_txd: 36476
    dev.em.5.mac_stats.bcast_pkts_txd: 8603
    dev.em.5.mac_stats.mcast_pkts_txd: 2211
    dev.em.5.mac_stats.tx_frames_64: 18335
    dev.em.5.mac_stats.tx_frames_65_127: 14216
    dev.em.5.mac_stats.tx_frames_128_255: 870
    dev.em.5.mac_stats.tx_frames_256_511: 719
    dev.em.5.mac_stats.tx_frames_512_1023: 306
    dev.em.5.mac_stats.tx_frames_1024_1522: 2030
    dev.em.5.mac_stats.tso_txd: 0
    dev.em.5.mac_stats.tso_ctx_fail: 0
    dev.em.5.interrupts.asserts: 9
    dev.em.5.interrupts.rx_pkt_timer: 0
    dev.em.5.interrupts.rx_abs_timer: 0
    dev.em.5.interrupts.tx_pkt_timer: 0
    dev.em.5.interrupts.tx_abs_timer: 0
    dev.em.5.interrupts.tx_queue_empty: 0
    dev.em.5.interrupts.tx_queue_min_thresh: 0
    dev.em.5.interrupts.rx_desc_min_thresh: 0
    dev.em.5.interrupts.rx_overrun: 0
    dev.em.6.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.6.%driver: em
    dev.em.6.%location: slot=0 function=0
    dev.em.6.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.6.%parent: pci17
    dev.em.6.nvm: -1
    dev.em.6.debug: -1
    dev.em.6.fc: 3
    dev.em.6.rx_int_delay: 0
    dev.em.6.tx_int_delay: 66
    dev.em.6.rx_abs_int_delay: 66
    dev.em.6.tx_abs_int_delay: 66
    dev.em.6.itr: 488
    dev.em.6.rx_processing_limit: 100
    dev.em.6.eee_control: 1
    dev.em.6.link_irq: 22
    dev.em.6.mbuf_alloc_fail: 0
    dev.em.6.cluster_alloc_fail: 0
    dev.em.6.dropped: 0
    dev.em.6.tx_dma_fail: 0
    dev.em.6.rx_overruns: 0
    dev.em.6.watchdog_timeouts: 0
    dev.em.6.device_control: 1477444168
    dev.em.6.rx_control: 67141658
    dev.em.6.fc_high_water: 18432
    dev.em.6.fc_low_water: 16932
    dev.em.6.queue0.txd_head: 103
    dev.em.6.queue0.txd_tail: 103
    dev.em.6.queue0.tx_irq: 87586
    dev.em.6.queue0.no_desc_avail: 0
    dev.em.6.queue0.rxd_head: 840
    dev.em.6.queue0.rxd_tail: 839
    dev.em.6.queue0.rx_irq: 37860
    dev.em.6.mac_stats.excess_coll: 0
    dev.em.6.mac_stats.single_coll: 0
    dev.em.6.mac_stats.multiple_coll: 0
    dev.em.6.mac_stats.late_coll: 0
    dev.em.6.mac_stats.collision_count: 0
    dev.em.6.mac_stats.symbol_errors: 0
    dev.em.6.mac_stats.sequence_errors: 0
    dev.em.6.mac_stats.defer_count: 0
    dev.em.6.mac_stats.missed_packets: 0
    dev.em.6.mac_stats.recv_no_buff: 0
    dev.em.6.mac_stats.recv_undersize: 0
    dev.em.6.mac_stats.recv_fragmented: 0
    dev.em.6.mac_stats.recv_oversize: 0
    dev.em.6.mac_stats.recv_jabber: 0
    dev.em.6.mac_stats.recv_errs: 0
    dev.em.6.mac_stats.crc_errs: 0
    dev.em.6.mac_stats.alignment_errs: 0
    dev.em.6.mac_stats.coll_ext_errs: 0
    dev.em.6.mac_stats.xon_recvd: 0
    dev.em.6.mac_stats.xon_txd: 0
    dev.em.6.mac_stats.xoff_recvd: 0
    dev.em.6.mac_stats.xoff_txd: 0
    dev.em.6.mac_stats.total_pkts_recvd: 41796
    dev.em.6.mac_stats.good_pkts_recvd: 41796
    dev.em.6.mac_stats.bcast_pkts_recvd: 755
    dev.em.6.mac_stats.mcast_pkts_recvd: 454
    dev.em.6.mac_stats.rx_frames_64: 29124
    dev.em.6.mac_stats.rx_frames_65_127: 8291
    dev.em.6.mac_stats.rx_frames_128_255: 984
    dev.em.6.mac_stats.rx_frames_256_511: 824
    dev.em.6.mac_stats.rx_frames_512_1023: 993
    dev.em.6.mac_stats.rx_frames_1024_1522: 1580
    dev.em.6.mac_stats.good_octets_recvd: 6142890
    dev.em.6.mac_stats.good_octets_txd: 89550641
    dev.em.6.mac_stats.total_pkts_txd: 95436
    dev.em.6.mac_stats.good_pkts_txd: 95436
    dev.em.6.mac_stats.bcast_pkts_txd: 21172
    dev.em.6.mac_stats.mcast_pkts_txd: 8816
    dev.em.6.mac_stats.tx_frames_64: 19702
    dev.em.6.mac_stats.tx_frames_65_127: 9482
    dev.em.6.mac_stats.tx_frames_128_255: 2561
    dev.em.6.mac_stats.tx_frames_256_511: 8128
    dev.em.6.mac_stats.tx_frames_512_1023: 765
    dev.em.6.mac_stats.tx_frames_1024_1522: 54798
    dev.em.6.mac_stats.tso_txd: 0
    dev.em.6.mac_stats.tso_ctx_fail: 0
    dev.em.6.interrupts.asserts: 14
    dev.em.6.interrupts.rx_pkt_timer: 0
    dev.em.6.interrupts.rx_abs_timer: 0
    dev.em.6.interrupts.tx_pkt_timer: 0
    dev.em.6.interrupts.tx_abs_timer: 0
    dev.em.6.interrupts.tx_queue_empty: 0
    dev.em.6.interrupts.tx_queue_min_thresh: 0
    dev.em.6.interrupts.rx_desc_min_thresh: 0
    dev.em.6.interrupts.rx_overrun: 0
    dev.em.7.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
    dev.em.7.%driver: em
    dev.em.7.%location: slot=0 function=0
    dev.em.7.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0x0000 class=0x020000
    dev.em.7.%parent: pci18
    dev.em.7.nvm: -1
    dev.em.7.debug: -1
    dev.em.7.fc: 3
    dev.em.7.rx_int_delay: 0
    dev.em.7.tx_int_delay: 66
    dev.em.7.rx_abs_int_delay: 66
    dev.em.7.tx_abs_int_delay: 66
    dev.em.7.itr: 488
    dev.em.7.rx_processing_limit: 100
    dev.em.7.eee_control: 1
    dev.em.7.link_irq: 0
    dev.em.7.mbuf_alloc_fail: 0
    dev.em.7.cluster_alloc_fail: 0
    dev.em.7.dropped: 0
    dev.em.7.tx_dma_fail: 0
    dev.em.7.rx_overruns: 0
    dev.em.7.watchdog_timeouts: 0
    dev.em.7.device_control: 1074790984
    dev.em.7.rx_control: 67141658
    dev.em.7.fc_high_water: 18432
    dev.em.7.fc_low_water: 16932
    dev.em.7.queue0.txd_head: 0
    dev.em.7.queue0.txd_tail: 0
    dev.em.7.queue0.tx_irq: 0
    dev.em.7.queue0.no_desc_avail: 0
    dev.em.7.queue0.rxd_head: 0
    dev.em.7.queue0.rxd_tail: 1023
    dev.em.7.queue0.rx_irq: 0
    dev.em.7.mac_stats.excess_coll: 0
    dev.em.7.mac_stats.single_coll: 0
    dev.em.7.mac_stats.multiple_coll: 0
    dev.em.7.mac_stats.late_coll: 0
    dev.em.7.mac_stats.collision_count: 0
    dev.em.7.mac_stats.symbol_errors: 0
    dev.em.7.mac_stats.sequence_errors: 0
    dev.em.7.mac_stats.defer_count: 0
    dev.em.7.mac_stats.missed_packets: 0
    dev.em.7.mac_stats.recv_no_buff: 0
    dev.em.7.mac_stats.recv_undersize: 0
    dev.em.7.mac_stats.recv_fragmented: 0
    dev.em.7.mac_stats.recv_oversize: 0
    dev.em.7.mac_stats.recv_jabber: 0
    dev.em.7.mac_stats.recv_errs: 0
    dev.em.7.mac_stats.crc_errs: 0
    dev.em.7.mac_stats.alignment_errs: 0
    dev.em.7.mac_stats.coll_ext_errs: 0
    dev.em.7.mac_stats.xon_recvd: 0
    dev.em.7.mac_stats.xon_txd: 0
    dev.em.7.mac_stats.xoff_recvd: 0
    dev.em.7.mac_stats.xoff_txd: 0
    dev.em.7.mac_stats.total_pkts_recvd: 0
    dev.em.7.mac_stats.good_pkts_recvd: 0
    dev.em.7.mac_stats.bcast_pkts_recvd: 0
    dev.em.7.mac_stats.mcast_pkts_recvd: 0
    dev.em.7.mac_stats.rx_frames_64: 0
    dev.em.7.mac_stats.rx_frames_65_127: 0
    dev.em.7.mac_stats.rx_frames_128_255: 0
    dev.em.7.mac_stats.rx_frames_256_511: 0
    dev.em.7.mac_stats.rx_frames_512_1023: 0
    dev.em.7.mac_stats.rx_frames_1024_1522: 0
    dev.em.7.mac_stats.good_octets_recvd: 0
    dev.em.7.mac_stats.good_octets_txd: 0
    dev.em.7.mac_stats.total_pkts_txd: 0
    dev.em.7.mac_stats.good_pkts_txd: 0
    dev.em.7.mac_stats.bcast_pkts_txd: 0
    dev.em.7.mac_stats.mcast_pkts_txd: 0
    dev.em.7.mac_stats.tx_frames_64: 0
    dev.em.7.mac_stats.tx_frames_65_127: 0
    dev.em.7.mac_stats.tx_frames_128_255: 0
    dev.em.7.mac_stats.tx_frames_256_511: 0
    dev.em.7.mac_stats.tx_frames_512_1023: 0
    dev.em.7.mac_stats.tx_frames_1024_1522: 0
    dev.em.7.mac_stats.tso_txd: 0
    dev.em.7.mac_stats.tso_ctx_fail: 0
    dev.em.7.interrupts.asserts: 0
    dev.em.7.interrupts.rx_pkt_timer: 0
    dev.em.7.interrupts.rx_abs_timer: 0
    dev.em.7.interrupts.tx_pkt_timer: 0
    dev.em.7.interrupts.tx_abs_timer: 0
    dev.em.7.interrupts.tx_queue_empty: 0
    dev.em.7.interrupts.tx_queue_min_thresh: 0
    dev.em.7.interrupts.rx_desc_min_thresh: 0
    dev.em.7.interrupts.rx_overrun: 0
    

  • Netgate Administrator

    I have a box that is quite similar in many ways, same NICs, and I often use it as you do, without a switch. I too have all six em NICs bridged together. However I haven't done so recently as I'm using as a test box with 2.2. Thinking back I'm not sure if I ever had that arrangement with 2.1.1/2.
    Interestingly I have experienced very similar behavior on that box but not on the em interfaces, on its one fxp NIC. That NIC will not negotiate a connection if it is powered up before the machine at the other end. In that case it was a harware (perhaps firmware) level issue where it would fail to negotiate a speed and duplex wiht the NIC at the other end resulting in no link. The link LED would flash intermittently as it tried and failed. It has no problem doing it it the other end is already powered up. I never found a good solution to it but you work around it by manually setting the link speed and duplex. Try that.

    Steve



  • I have played around with nic settings on the connecting PC before, and that didn't seem to make a difference, but I will try that again.

    I looked at some of the differences between the em drivers in 2.1.0 and 2.1.2.
    In 2.1.2 the version is 7.3.8, I think 2.1.0 might be 7.3.2.
    One thing I noticed is that the hw.em.eee_setting (Energy Efficient Ethernet) default might have changed.
    It now seems to default to 1 and I think it defaulted to 0 before.
    However I can't figure out by the somewhat cryptic code if that means it's now enabled or disabled.
    Anyway, I set hw.em.eee_setting=0 in /boot/loader.conf.local and rebooted, but that unfortunately didn't fix it.


  • Netgate Administrator

    Damn that seems like a great suspect. The sysctl had changed after boot?

    In my case you could see the NIC wasn't making a connection as ifconfig showed 'no carrier' I don't belive you're seeing that though.

    Steve



  • After the reboot, sysctl did show eee_setting as 0.

    Edit: I tried changing the link speed and duplex of the connecting PC, and that made no difference.

    There are some tests included in the Windows nic driver that I ran, and during the time the problem occurs, they all succeed except for the connection test.

    ![Cable Test.gif](/public/imported_attachments/1/Cable Test.gif)
    ![Cable Test.gif_thumb](/public/imported_attachments/1/Cable Test.gif_thumb)
    ![Hardware Test.gif](/public/imported_attachments/1/Hardware Test.gif)
    ![Hardware Test.gif_thumb](/public/imported_attachments/1/Hardware Test.gif_thumb)