NIC only works with polling enabled



  • Got a bit of a weird problem, it seems I have a NIC that only works (sort of) when I enable polling. The hardware is the OPNsense 5 port GHz rack edition from applianceshop.eu. Unfortunately the website is down at the moment. EDIT: website is back up now

    It has 5 NICs in total with 3 different types: 1 x re, 2 x rl & 2 x em. The re & rl ones work fine with polling off or on. The em ones only seem to work with polling on but this makes the Webconfigurator and SSH access really slow. Additionally the em nics seem to average 20ms latency on the LAN. I found I can turn on polling just for the em NIC by running "ifconfig em0 polling" which solves the slow Webconfigurator and SSH but this turns back off every time I make a change to that NIC or the system reboots.

    Unfortunately I don't know if this box exhibits the same behavior on v1.2.3 as we have only recently needed to use the em based interfaces and we upgraded the box to 2.0 rc1 i386 ages ago (which has been running flawlessly). I updated it to 2.0RC2 yesterday but the behavior is the same.

    Does anyone have any further troubleshooting steps I can try? Or as a work around, how could I get polling to stay on just for the em NICs? I have been using pfsense for a couple of years now but since I have had very few problems I have no idea how to troubleshoot this particular problem!

    Thanks in advance,

    Paul

    Some additional info:

    dmesg | grep em
    em0: <intel(r) 1000="" pro="" legacy="" network="" connection="" 1.0.3=""> port 0xc880-0xc8bf mem 0xcfce0000-0xcfcfffff,0xcfcc0000-0xcfcdffff irq 11 at device 1.0 on pci2
    em0: [FILTER]
    em1: <intel(r) 1000="" pro="" legacy="" network="" connection="" 1.0.3=""> port 0xc480-0xc4bf mem 0xcfc60000-0xcfc7ffff,0xcfc40000-0xcfc5ffff irq 16 at device 4.0 on pci2
    em1: [FILTER]</intel(r)></intel(r)>
    
    ifconfig em0
    em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=20db <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,polling,vlan_hwcsum,wol_magic>ether 00:03:1d:09:39:84
            inet6 fe80::203:1dff:fe09:3984%em0 prefixlen 64 scopeid 0x1
            inet xx.xx.xx.xx netmask 0xfffffff0 broadcast xx.xx.xx.xx
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (100baseTX <full-duplex>)
            status: active</full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,polling,vlan_hwcsum,wol_magic></up,broadcast,running,simplex,multicast> 
    

  • Netgate Administrator

    Hmm, weird problem. What are the NICs on the em interfaces?

    You could add the polling work around as a shell command. That would only help with rebooting though.

    Steve



  • Is there any way to identify the make/model of the NIC from the command line? This box is at a remote site so I can't look inside.

    I'll give that shell command ago.


  • Netgate Administrator

    
    [2.0-RC2][root@pfsense.fire.box]/root(4): sysctl dev.em.0
    dev.em.0.%desc: Intel(R) PRO/1000 Legacy Network Connection 1.0.3
    dev.em.0.%driver: em
    dev.em.0.%location: slot=1 function=0
    dev.em.0.%pnpinfo: vendor=0x8086 device=0x1075 subvendor=0x8086 subdevice=0x1075 class=0x020000
    
    

    Plus a load of other variables. Change 0 for 1 for the other NIC, though they're probably the same.

    Steve



  • Thanks, both are identical, output below:

    dev.em.0.%desc: Intel(R) PRO/1000 Legacy Network Connection 1.0.3
    dev.em.0.%driver: em
    dev.em.0.%location: slot=1 function=0
    dev.em.0.%pnpinfo: vendor=0x8086 device=0x1076 subvendor=0x8086 subdevice=0x1076 class=0x020000
    dev.em.0.%parent: pci2
    dev.em.0.nvm: -1
    dev.em.0.rx_int_delay: 0
    dev.em.0.tx_int_delay: 66
    dev.em.0.rx_abs_int_delay: 66
    dev.em.0.tx_abs_int_delay: 66
    dev.em.0.rx_processing_limit: 100
    dev.em.0.flow_control: 3
    dev.em.0.mbuf_alloc_fail: 0
    dev.em.0.cluster_alloc_fail: 0
    dev.em.0.dropped: 0
    dev.em.0.tx_dma_fail: 0
    dev.em.0.tx_desc_fail1: 0
    dev.em.0.tx_desc_fail2: 0
    dev.em.0.rx_overruns: 0
    dev.em.0.watchdog_timeouts: 0
    dev.em.0.device_control: 1480327745
    dev.em.0.rx_control: 32770
    dev.em.0.fc_high_water: 47104
    dev.em.0.fc_low_water: 45604
    dev.em.0.fifo_workaround: 0
    dev.em.0.fifo_reset: 0
    dev.em.0.txd_head: 209
    dev.em.0.txd_tail: 209
    dev.em.0.rxd_head: 152
    dev.em.0.rxd_tail: 151
    dev.em.0.mac_stats.excess_coll: 0
    dev.em.0.mac_stats.single_coll: 0
    dev.em.0.mac_stats.multiple_coll: 0
    dev.em.0.mac_stats.late_coll: 0
    dev.em.0.mac_stats.collision_count: 0
    dev.em.0.mac_stats.symbol_errors: 0
    dev.em.0.mac_stats.sequence_errors: 0
    dev.em.0.mac_stats.defer_count: 0
    dev.em.0.mac_stats.missed_packets: 1546
    dev.em.0.mac_stats.recv_no_buff: 302
    dev.em.0.mac_stats.recv_undersize: 0
    dev.em.0.mac_stats.recv_fragmented: 0
    dev.em.0.mac_stats.recv_oversize: 0
    dev.em.0.mac_stats.recv_jabber: 0
    dev.em.0.mac_stats.recv_errs: 0
    dev.em.0.mac_stats.crc_errs: 0
    dev.em.0.mac_stats.alignment_errs: 0
    dev.em.0.mac_stats.coll_ext_errs: 0
    dev.em.0.mac_stats.xon_recvd: 0
    dev.em.0.mac_stats.xon_txd: 2
    dev.em.0.mac_stats.xoff_recvd: 0
    dev.em.0.mac_stats.xoff_txd: 1548
    dev.em.0.mac_stats.total_pkts_recvd: 1155933
    dev.em.0.mac_stats.good_pkts_recvd: 1154387
    dev.em.0.mac_stats.bcast_pkts_recvd: 827665
    dev.em.0.mac_stats.mcast_pkts_recvd: 0
    dev.em.0.mac_stats.rx_frames_64: 584176
    dev.em.0.mac_stats.rx_frames_65_127: 257893
    dev.em.0.mac_stats.rx_frames_128_255: 109244
    dev.em.0.mac_stats.rx_frames_256_511: 3437
    dev.em.0.mac_stats.rx_frames_512_1023: 1107
    dev.em.0.mac_stats.rx_frames_1024_1522: 198530
    dev.em.0.mac_stats.good_octets_recvd: 384175227
    dev.em.0.mac_stats.good_octets_txd: 93827474
    dev.em.0.mac_stats.total_pkts_txd: 422787
    dev.em.0.mac_stats.good_pkts_txd: 421237
    dev.em.0.mac_stats.bcast_pkts_txd: 1812
    dev.em.0.mac_stats.mcast_pkts_txd: 3
    dev.em.0.mac_stats.tx_frames_64: 1364
    dev.em.0.mac_stats.tx_frames_65_127: 178187
    dev.em.0.mac_stats.tx_frames_128_255: 196500
    dev.em.0.mac_stats.tx_frames_256_511: 9443
    dev.em.0.mac_stats.tx_frames_512_1023: 10213
    dev.em.0.mac_stats.tx_frames_1024_1522: 25530
    dev.em.0.mac_stats.tso_txd: 0
    dev.em.0.mac_stats.tso_ctx_fail: 0
    
    

  • Netgate Administrator

    Is that with polling enabled?

    Steve



  • Yeah it was, output looks like this with it disabled:

    dev.em.0.%desc: Intel(R) PRO/1000 Legacy Network Connection 1.0.3
    dev.em.0.%driver: em
    dev.em.0.%location: slot=1 function=0
    dev.em.0.%pnpinfo: vendor=0x8086 device=0x1076 subvendor=0x8086 subdevice=0x1076 class=0x020000
    dev.em.0.%parent: pci2
    dev.em.0.nvm: -1
    dev.em.0.rx_int_delay: 0
    dev.em.0.tx_int_delay: 66
    dev.em.0.rx_abs_int_delay: 66
    dev.em.0.tx_abs_int_delay: 66
    dev.em.0.rx_processing_limit: 100
    dev.em.0.flow_control: 3
    dev.em.0.mbuf_alloc_fail: 0
    dev.em.0.cluster_alloc_fail: 0
    dev.em.0.dropped: 0
    dev.em.0.tx_dma_fail: 0
    dev.em.0.tx_desc_fail1: 0
    dev.em.0.tx_desc_fail2: 0
    dev.em.0.rx_overruns: 0
    dev.em.0.watchdog_timeouts: 0
    dev.em.0.device_control: 1480327745
    dev.em.0.rx_control: 32770
    dev.em.0.fc_high_water: 47104
    dev.em.0.fc_low_water: 45604
    dev.em.0.fifo_workaround: 0
    dev.em.0.fifo_reset: 0
    dev.em.0.txd_head: 122
    dev.em.0.txd_tail: 122
    dev.em.0.rxd_head: 235
    dev.em.0.rxd_tail: 235
    dev.em.0.mac_stats.excess_coll: 0
    dev.em.0.mac_stats.single_coll: 0
    dev.em.0.mac_stats.multiple_coll: 0
    dev.em.0.mac_stats.late_coll: 0
    dev.em.0.mac_stats.collision_count: 0
    dev.em.0.mac_stats.symbol_errors: 0
    dev.em.0.mac_stats.sequence_errors: 0
    dev.em.0.mac_stats.defer_count: 0
    dev.em.0.mac_stats.missed_packets: 1546
    dev.em.0.mac_stats.recv_no_buff: 403
    dev.em.0.mac_stats.recv_undersize: 0
    dev.em.0.mac_stats.recv_fragmented: 0
    dev.em.0.mac_stats.recv_oversize: 0
    dev.em.0.mac_stats.recv_jabber: 0
    dev.em.0.mac_stats.recv_errs: 0
    dev.em.0.mac_stats.crc_errs: 0
    dev.em.0.mac_stats.alignment_errs: 0
    dev.em.0.mac_stats.coll_ext_errs: 0
    dev.em.0.mac_stats.xon_recvd: 0
    dev.em.0.mac_stats.xon_txd: 2
    dev.em.0.mac_stats.xoff_recvd: 0
    dev.em.0.mac_stats.xoff_txd: 1548
    dev.em.0.mac_stats.total_pkts_recvd: 1192738
    dev.em.0.mac_stats.good_pkts_recvd: 1191192
    dev.em.0.mac_stats.bcast_pkts_recvd: 840327
    dev.em.0.mac_stats.mcast_pkts_recvd: 0
    dev.em.0.mac_stats.rx_frames_64: 594209
    dev.em.0.mac_stats.rx_frames_65_127: 272509
    dev.em.0.mac_stats.rx_frames_128_255: 119139
    dev.em.0.mac_stats.rx_frames_256_511: 3692
    dev.em.0.mac_stats.rx_frames_512_1023: 1217
    dev.em.0.mac_stats.rx_frames_1024_1522: 200426
    dev.em.0.mac_stats.good_octets_recvd: 391205396
    dev.em.0.mac_stats.good_octets_txd: 101322876
    dev.em.0.mac_stats.total_pkts_txd: 457538
    dev.em.0.mac_stats.good_pkts_txd: 455988
    dev.em.0.mac_stats.bcast_pkts_txd: 1812
    dev.em.0.mac_stats.mcast_pkts_txd: 3
    dev.em.0.mac_stats.tx_frames_64: 1368
    dev.em.0.mac_stats.tx_frames_65_127: 187744
    dev.em.0.mac_stats.tx_frames_128_255: 218107
    dev.em.0.mac_stats.tx_frames_256_511: 10422
    dev.em.0.mac_stats.tx_frames_512_1023: 11285
    dev.em.0.mac_stats.tx_frames_1024_1522: 27062
    dev.em.0.mac_stats.tso_txd: 0
    dev.em.0.mac_stats.tso_ctx_fail: 0
    
    


  • It looks wrong to me that one NIC on PCI bus 4 uses irq 11 and another NIC on the same bus uses irq 16. (At least such an assignment would be most unlikely on recent Intel hardware where I would expect both to have an irq of 16 or more or both to have 12 or less.)

    Please provide the output of the pfSense shell command dmesg so I can see what other devices and irqs are in use of your box. You might also look to see if there is a more recent BIOS that one currently installed, in particular, a BIOS update addressing irq assignments to PCI devices.



  • All fixed :D Jos from applianceshop.eu contacted me and suggested i try disabling ACPI by adding hint.acpi.0.disabled="1" to /boot/loader.conf.

    Just in case the dmesg is helpful to anyone else, here it is (with acpi disabled)

    Copyright (c) 1992-2010 The FreeBSD Project.
    Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
            The Regents of the University of California. All rights reserved.
    FreeBSD is a registered trademark of The FreeBSD Foundation.
    FreeBSD 8.1-RELEASE-p4 #0: Thu Jun  2 04:44:19 EDT 2011
        sullrich@FreeBSD_8.0_pfSense_2.0-snaps.pfsense.org:/usr/obj.pfSense/usr/pnsesrc/src/sys/pfSense_wrap.8.i386 i386
    Timecounter "i8254" frequency 1193182 Hz quality 0
    CPU: Intel(R) Atom(TM) CPU Z530   @ 1.60GHz (1596.00-MHz 686-class CPU)
      Origin = "GenuineIntel"  Id = 0x106c2  Family = 6  Model = 1c  Stepping = 2
      Features=0xbfe9fbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mcmov,pat,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>Features2=0x40c3bd <sse3,dtes64,mon,ds_cpl,vmx,est,tm2,ssse3,xtpr,pdcm,movbe<br>AMD Features=0x100000 <nx>AMD Features2=0x1 <lahf>TSC: P-state invariant
    real memory  = 1073741824 (1024 MB)
    avail memory = 1026523136 (978 MB)
    MPTable: <intel  =""  poulsbo="">WARNING: Non-uniform processors.
    WARNING: Using suboptimal topology.
    ioapic0: Assuming intbase of 0
    ioapic0 <version 2.0="">irqs 0-23 on motherboard
    wlan: mac acl policy registered
    ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/
    ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in ot/loader.conf.
    module_register_init: MOD_LOAD (ipw_bss_fw, 0xc0708980, 0) error 1
    ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw
    ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 inoot/loader.conf.
    module_register_init: MOD_LOAD (ipw_ibss_fw, 0xc0708a20, 0) error 1
    wpi: You need to read the LICENSE file in /usr/share/doc/legal/intel_wpi/.
    wpi: If you agree with the license, set legal.intel_wpi.license_ack=1 in /boooader.conf.
    module_register_init: MOD_LOAD (wpi_fw, 0xc087b840, 0) error 1
    ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_/.
    ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 /boot/loader.conf.
    module_register_init: MOD_LOAD (ipw_monitor_fw, 0xc0708ac0, 0) error 1
    cryptosoft0: <software crypto="">on motherboard
    padlock0: No ACE support.
    pcib0: <mptable host-pci="" bridge="">pcibus 0 on motherboard
    pci0: <pci bus="">on pcib0
    vgapci0: <vga-compatible display="">port 0xe880-0xe887 mem 0xdff80000-0xdffffffxd0000000-0xd7ffffff,0xdff60000-0xdff7ffff irq 16 at device 2.0 on pci0
    pcib1: <pci-pci bridge="">irq 16 at device 28.0 on pci0
    pci1: <pci bus="">on pcib1
    pcib2: <mptable pci-pci="" bridge="">at device 0.0 on pci1
    pci2: <pci bus="">on pcib2
    em0: <intel(r) 1000="" pro="" legacy="" network="" connection="" 1.0.3="">port 0xc880-0xc8bf m0xcfce0000-0xcfcfffff,0xcfcc0000-0xcfcdffff irq 17 at device 1.0 on pci2
    em0: [FILTER]
    rl0: <realtek 10="" 8139="" 100basetx="">port 0xce00-0xceff mem 0xcfc9fc00-0xcfc9fcffq 18 at device 2.0 on pci2
    miibus0: <mii bus="">on rl0
    rlphy0: <realtek internal="" media="" interface="">PHY 0 on miibus0
    rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    rl0: [ITHREAD]
    rl1: <realtek 10="" 8139="" 100basetx="">port 0xcd00-0xcdff mem 0xcfc9f800-0xcfc9f8ffq 19 at device 3.0 on pci2
    miibus1: <mii bus="">on rl1
    rlphy1: <realtek internal="" media="" interface="">PHY 0 on miibus1
    rlphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    rl1: [ITHREAD]
    em1: <intel(r) 1000="" pro="" legacy="" network="" connection="" 1.0.3="">port 0xc480-0xc4bf m0xcfc60000-0xcfc7ffff,0xcfc40000-0xcfc5ffff irq 16 at device 4.0 on pci2
    em1: [FILTER]
    pcib3: <mptable pci-pci="" bridge="">irq 17 at device 28.1 on pci0
    pci3: <pci bus="">on pcib3
    re0: <realtek 8111="" 8168="" b="" c="" cp="" d="" dp="" e="" pcie="" gigabit="" ethernet="">port 0xde00-0xdemem 0xcfdff000-0xcfdfffff,0xff9fc000-0xff9fffff irq 17 at device 0.0 on pci3
    re0: Using 1 MSI messages
    re0: Chip rev. 0x28000000
    re0: MAC rev. 0x00000000
    miibus2: <mii bus="">on re0
    rgephy0: <rtl8169s 8110s="" 8211b="" media="" interface="">PHY 1 on miibus2
    rgephy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseDX, auto
    re0: [FILTER]
    uhci0: <uhci (generic)="" usb="" controller="">port 0xe480-0xe49f irq 16 at device 29on pci0
    uhci0: [ITHREAD]
    usbus0: <uhci (generic)="" usb="" controller="">on uhci0
    ehci0: <ehci (generic)="" usb="" 2.0="" controller="">mem 0xdff5bc00-0xdff5bfff irq 19 aevice 29.7 on pci0
    ehci0: [ITHREAD]
    usbus1: EHCI version 1.0
    usbus1: <ehci (generic)="" usb="" 2.0="" controller="">on ehci0
    isab0: <pci-isa bridge="">at device 31.0 on pci0
    isa0: <isa bus="">on isab0
    atapci0: <intel sch="" udma100="" controller="">port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x,0xffa0-0xffaf at device 31.1 on pci0
    ata0: <ata 0="" channel="">on atapci0
    ata0: [ITHREAD]
    cpu0 on motherboard
    unknown: <pnp0c01>can't assign resources (memory)
    atrtc0: <at realtime="" clock="">at port 0x70-0x71 irq 8 pnpid PNP0b00 on isa0
    uart0: <16550 or compatible> at port 0x3f8-0x3ff irq 4 flags 0x10 pnpid PNP05on isa0
    uart0: [FILTER]
    uart0: console (9600,n,8,1)
    ppc0: parallel port not found.
    unknown: <pnp0c01>can't assign resources (memory)
    Timecounter "TSC" frequency 1595998260 Hz quality 800
    Timecounters tick every 10.000 msec
    IPsec: Initialized Security Association Processing.
    usbus0: 12Mbps Full Speed USB v1.0
    usbus1: 480Mbps High Speed USB v2.0
    ugen0.1: <intel>at usbus0
    uhub0: <intel 1="" 9="" uhci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usbus0
    ugen1.1: <intel>at usbus1
    uhub1: <intel 1="" 9="" ehci="" root="" hub,="" class="" 0,="" rev="" 2.00="" 1.00,="" addr="">on usbus1
    ad1: 967MB <cf 20090819="" 1gb="">at ata0-slave PIO4
    GEOM: ad1s1: media size does not match label.
    GEOM: ad1s2: media size does not match label.
    Root mount waiting for: usbus1 usbus0
    uhub0: 2 ports with 2 removable, self powered
    Root mount waiting for: usbus1
    uhub1: 4 ports with 4 removable, self powered
    Trying to mount root from ufs:/dev/ufs/pfsense0
    pflog0: promiscuous mode enabled
    em0: link state changed to UP
    rl0: link state changed to UP
    rl1: link state changed to UP
    WARNING: pseudo-random number generator used for IPsec processing
    arp: 10.21.3.22 moved from 58:55:ca:07:0d:59 to 00:23:12:0b:4b:b1 on re0</cf></intel></intel></intel></intel></pnp0c01></at></pnp0c01></ata></intel></isa></pci-isa></ehci></ehci></uhci></uhci></rtl8169s></mii></realtek></pci></mptable></intel(r)></realtek></mii></realtek></realtek></mii></realtek></intel(r)></pci></mptable></pci></pci-pci></vga-compatible></pci></mptable></software></version></intel></lahf></nx></sse3,dtes64,mon,ds_cpl,vmx,est,tm2,ssse3,xtpr,pdcm,movbe<br></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mcmov,pat,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe> 
    

Log in to reply