EM0 High Interrupt, Enabling polling brings down the interface. 2.0.2-RELEASE
-
Hey all,
I am using 2.0.2-RELEASE (i386), and I have a Intel Pro Gigabit card, using the EM driver. When I turn on Polling, from the router, I can send about 7 packets to another host with ping before the connection drops. I think that I am getting a watchdog timeout error. With the polling disabled, it works fine, but generates high interrupts on a gigabit network. Please advise.
[2.0.2-RELEASE][admin@router.*****.com]/root(1): dmesg
Copyright
1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.1-RELEASE-p13 #0: Fri Dec 7 16:51:57 EST 2012
root@snapshots-8_1-i386.builders.pfsense.org:/usr/obj./usr/pfSensesrc/src/sys/pfSense_SMP.8 i386
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Celeron(R) CPU 3.06GHz (3066.79-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf49 Family = f Model = 4 Stepping = 9
Features=0xbfebfbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>Features2=0x651d <sse3,dtes64,mon,ds_cpl,tm2,cnxt-id,cx16,xtpr>AMD Features=0x20000000 <lm>TSC: P-state invariant
real memory = 536870912 (512 MB)
avail memory = 468828160 (447 MB)
ACPI APIC Table: <award awrdacpi="">ioapic0 <version 1.1="">irqs 0-23 on motherboard
wpi: You need to read the LICENSE file in /usr/share/doc/legal/intel_wpi/.
wpi: If you agree with the license, set legal.intel_wpi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (wpi_fw, 0xc098a6c0, 0) error 1
ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_bss_fw, 0xc078b700, 0) error 1
ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_ibss_fw, 0xc078b7a0, 0) error 1
ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_monitor_fw, 0xc078b840, 0) error 1
wlan: mac acl policy registered
kbd1 at kbdmux0
cryptosoft0: <software crypto="">on motherboard
padlock0: No ACE support.
acpi0: <award awrdacpi="">on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, 1def0000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
cpu0: <acpi cpu="">on acpi0
acpi_button0: <power button="">on acpi0
acpi_button1: <sleep button="">on acpi0
pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff,0x480-0x48f,0x1000-0x10df,0x10e0-0x10ff on acpi0
pci0: <acpi pci="" bus="">on pcib0
agp0: <sis 661="" host="" to="" agp="" bridge="">on hostb0
pcib1: <pci-pci bridge="">at device 1.0 on pci0
pci1: <pci bus="">on pcib1
vgapci0: <vga-compatible display="">port 0x9000-0x907f mem 0xd8000000-0xdfffffff,0xe1000000-0xe101ffff at device 0.0 on pci1
isab0: <pci-isa bridge="">at device 2.0 on pci0
isa0: <isa bus="">on isab0
atapci0: <sis 964="" udma133="" controller="">port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x4000-0x400f at device 2.5 on pci0
ata0: <ata 0="" channel="">on atapci0
ata0: [ITHREAD]
ata1: <ata 1="" channel="">on atapci0
ata1: [ITHREAD]
pci0: <multimedia, audio="">at device 2.7 (no driver attached)
ohci0: <sis 5571="" usb="" controller="">mem 0xe1142000-0xe1142fff irq 20 at device 3.0 on pci0
ohci0: [ITHREAD]
usbus0: <sis 5571="" usb="" controller="">on ohci0
ohci1: <sis 5571="" usb="" controller="">mem 0xe1143000-0xe1143fff irq 21 at device 3.1 on pci0
ohci1: [ITHREAD]
usbus1: <sis 5571="" usb="" controller="">on ohci1
ohci2: <sis 5571="" usb="" controller="">mem 0xe1144000-0xe1144fff irq 22 at device 3.2 on pci0
ohci2: [ITHREAD]
usbus2: <sis 5571="" usb="" controller="">on ohci2
ehci0: <ehci (generic)="" usb="" 2.0="" controller="">mem 0xe1140000-0xe1140fff irq 23 at device 3.3 on pci0
ehci0: [ITHREAD]
usbus3: EHCI version 1.0
usbus3: <ehci (generic)="" usb="" 2.0="" controller="">on ehci0
sis0: <sis 10="" 900="" 100basetx="">port 0xa800-0xa8ff mem 0xe1141000-0xe1141fff irq 19 at device 4.0 on pci0
miibus0: <mii bus="">on sis0
rlphy0: <rtl8201l 10="" 100="" media="" interface="">PHY 9 on miibus0
rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
sis0: [ITHREAD]
atapci1: <sis 964="" udma133="" controller="">port 0xac00-0xac07,0xb000-0xb003,0xb400-0xb407,0xb800-0xb803,0xbc00-0xbc0f irq 17 at device 5.0 on pci0
atapci1: [ITHREAD]
ata2: <ata 0="" channel="">on atapci1
ata2: [ITHREAD]
ata3: <ata 1="" channel="">on atapci1
ata3: [ITHREAD]
em0: <intel(r) 1000="" pro="" legacy="" network="" connection="" 1.0.4="">port 0xc000-0xc03f mem 0xe1100000-0xe111ffff,0xe1120000-0xe113ffff irq 18 at device 10.0 on pci0
em0: [FILTER]
acpi_tz0: <thermal zone="">on acpi0
atrtc0: <at realtime="" clock="">port 0x70-0x73 irq 8 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart1: [FILTER]
ppc0: <parallel port="">port 0x378-0x37f,0x778-0x77b irq 7 drq 3 on acpi0
ppc0: Generic chipset (ECP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/16 bytes threshold
ppc0: [ITHREAD]
ppbus0: <parallel port="" bus="">on ppc0
plip0: <plip network="" interface="">on ppbus0
plip0: [ITHREAD]
lpt0: <printer>on ppbus0
lpt0: [ITHREAD]
lpt0: Interrupt-driven port
ppi0: <parallel i="" o="">on ppbus0
atkbdc0: <keyboard controller="" (i8042)="">port 0x60,0x64 irq 1 on acpi0
atkbd0: <at keyboard="">irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
psm0: <ps 2="" mouse="">irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model IntelliMouse Explorer, device ID 4
pmtimer0 on isa0
orm0: <isa option="" roms="">at iomem 0xc0000-0xc7fff,0xd0000-0xd17ff,0xef000-0xeffff pnpid ORM0000 on isa0
sc0: <system console="">at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <generic isa="" vga="">at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
p4tcc0: <cpu frequency="" thermal="" control="">on cpu0
Timecounter "TSC" frequency 3066792481 Hz quality 800
Timecounters tick every 1.000 msec
IPsec: Initialized Security Association Processing.
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
ugen0.1: <sis>at usbus0
uhub0: <sis 1="" 9="" ohci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usbus0
ugen1.1: <sis>at usbus1
uhub1: <sis 1="" 9="" ohci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usbus1
ugen2.1: <sis>at usbus2
uhub2: <sis 1="" 9="" ohci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usbus2
ugen3.1: <sis>at usbus3
uhub3: <sis 1="" 9="" ehci="" root="" hub,="" class="" 0,="" rev="" 2.00="" 1.00,="" addr="">on usbus3
acd0: DMA limited to UDMA33, device found non-ATA66 cable
acd0: DVDR <lite-on dvdrw="" shw-160p6s="" prs2="">at ata1-master UDMA33
ad4: DMA limited to UDMA33, device found non-ATA66 cable
ad4: 78533MB <hitachi hds728080pla380="" pf2oa60a="">at ata2-master UDMA33
Root mount waiting for: usbus3 usbus2 usbus1 usbus0
uhub2: 2 ports with 2 removable, self powered
uhub0: 3 ports with 3 removable, self powered
uhub1: 3 ports with 3 removable, self powered
Root mount waiting for: usbus3
Root mount waiting for: usbus3
Root mount waiting for: usbus3
uhub3: 8 ports with 8 removable, self powered
Trying to mount root from ufs:/dev/ad4s1a
pflog0: promiscuous mode enabled
sis0: link state changed to UP
em0: link state changed to UP
em0: Watchdog timeout – resetting
em0: link state changed to DOWN
em0: link state changed to UP
em0: Watchdog timeout -- resetting
em0: link state changed to DOWN
em0: link state changed to UP
sis0: watchdog timeout
arp: 10.1.10.11 moved from 00:19:d1:e1:7a:1f to 00:40:ca:72:5b:ee on em0
arp: 10.1.10.11 moved from 00:40:ca:72:5b:ee to 00:19:d1:e1:7a:1f on em0[2.0.2-RELEASE][admin@router.*****.com]/root(2): ifconfig -a
sis0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=80008 <vlan_mtu,linkstate>ether 00:16:ec:5b:2d:d7
inet 173.14.10.137 netmask 0xfffffffc broadcast 173.14.10.139
inet6 fe80::216:ecff:fe5b:2dd7%sis0 prefixlen 64 scopeid 0x1
nd6 options=43 <performnud,accept_rtadv>media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=209b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic>ether 00:07:e9:0f:f3:50
inet 10.1.10.1 netmask 0xffffff00 broadcast 10.1.10.255
inet6 fe80::207:e9ff:fe0f:f350%em0 prefixlen 64 scopeid 0x2
nd6 options=43 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
plip0: flags=8810 <pointopoint,simplex,multicast>metric 0 mtu 1500
pfsync0: flags=0<> metric 0 mtu 1460
syncpeer: 224.0.0.240 maxupd: 128 syncok: 1
pflog0: flags=100 <promisc>metric 0 mtu 33200
enc0: flags=0<> metric 0 mtu 1536
lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
nd6 options=43 <performnud,accept_rtadv></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></promisc></pointopoint,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></vlan_mtu,linkstate></up,broadcast,running,simplex,multicast></hitachi></lite-on></sis></sis></sis></sis></sis></sis></sis></sis></cpu></generic></system></isa></ps></at></keyboard></parallel></printer></plip></parallel></parallel></at></thermal></intel(r)></ata></ata></sis></rtl8201l></mii></sis></ehci></ehci></sis></sis></sis></sis></sis></sis></multimedia,></ata></ata></sis></isa></pci-isa></vga-compatible></pci></pci-pci></sis></acpi></acpi></sleep></power></acpi></award></software></version></award ></lm></sse3,dtes64,mon,ds_cpl,tm2,cnxt-id,cx16,xtpr></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe> -
A snapshot build of pfSense 2.1 might work better for you. Snapshot builds of pfSense 2.1 have more up to date device drivers than build of pfSense 2.0.x.
-
I just did the update to the latest snapshot 2.1 there was. Problem is still occuring. On a machine that I have pinging the router…
I am only testing the polling using the command ifconfig em0 polling and ifconfig em0 -polling
64 bytes from 10.1.10.1: icmp_req=399 ttl=64 time=0.204 ms
64 bytes from 10.1.10.1: icmp_req=400 ttl=64 time=0.207 ms
64 bytes from 10.1.10.1: icmp_req=401 ttl=64 time=0.257 ms
64 bytes from 10.1.10.1: icmp_req=402 ttl=64 time=0.105 ms
64 bytes from 10.1.10.1: icmp_req=403 ttl=64 time=0.167 ms
64 bytes from 10.1.10.1: icmp_req=404 ttl=64 time=0.221 msEnable Polling
64 bytes from 10.1.10.1: icmp_req=405 ttl=64 time=11085 ms
64 bytes from 10.1.10.1: icmp_req=406 ttl=64 time=10085 ms
64 bytes from 10.1.10.1: icmp_req=407 ttl=64 time=9085 ms
64 bytes from 10.1.10.1: icmp_req=408 ttl=64 time=8085 ms
64 bytes from 10.1.10.1: icmp_req=409 ttl=64 time=7086 ms
64 bytes from 10.1.10.1: icmp_req=410 ttl=64 time=6086 ms
64 bytes from 10.1.10.1: icmp_req=411 ttl=64 time=5086 ms
64 bytes from 10.1.10.1: icmp_req=412 ttl=64 time=4086 ms
64 bytes from 10.1.10.1: icmp_req=413 ttl=64 time=3086 ms
64 bytes from 10.1.10.1: icmp_req=414 ttl=64 time=2087 ms
64 bytes from 10.1.10.1: icmp_req=415 ttl=64 time=1087 ms
64 bytes from 10.1.10.1: icmp_req=416 ttl=64 time=87.2 msDisable Polling
64 bytes from 10.1.10.1: icmp_req=417 ttl=64 time=0.243 ms
64 bytes from 10.1.10.1: icmp_req=418 ttl=64 time=0.146 ms
64 bytes from 10.1.10.1: icmp_req=419 ttl=64 time=0.164 ms
64 bytes from 10.1.10.1: icmp_req=420 ttl=64 time=0.167 msRe-enable polling
64 bytes from 10.1.10.1: icmp_req=421 ttl=64 time=9365 ms
64 bytes from 10.1.10.1: icmp_req=422 ttl=64 time=8365 ms
64 bytes from 10.1.10.1: icmp_req=423 ttl=64 time=7366 ms
64 bytes from 10.1.10.1: icmp_req=424 ttl=64 time=6366 ms
64 bytes from 10.1.10.1: icmp_req=425 ttl=64 time=5366 ms
64 bytes from 10.1.10.1: icmp_req=426 ttl=64 time=4366 ms
64 bytes from 10.1.10.1: icmp_req=427 ttl=64 time=3367 ms
64 bytes from 10.1.10.1: icmp_req=428 ttl=64 time=2367 ms
64 bytes from 10.1.10.1: icmp_req=429 ttl=64 time=1367 ms
64 bytes from 10.1.10.1: icmp_req=430 ttl=64 time=368 msDisable Polling.
64 bytes from 10.1.10.1: icmp_req=431 ttl=64 time=0.133 ms
64 bytes from 10.1.10.1: icmp_req=432 ttl=64 time=0.240 ms
64 bytes from 10.1.10.1: icmp_req=433 ttl=64 time=0.154 ms
64 bytes from 10.1.10.1: icmp_req=434 ttl=64 time=0.239 ms
64 bytes from 10.1.10.1: icmp_req=435 ttl=64 time=0.145 ms
64 bytes from 10.1.10.1: icmp_req=436 ttl=64 time=0.305 ms
64 bytes from 10.1.10.1: icmp_req=437 ttl=64 time=0.111 ms -
Problem is still occuring.
What problem? Latency when polling is enabled? High interrupt load?
You haven't presented any evidence of high interrupt load. Please post output of pfSense shell command```
vmstat -i -
All that is seen below is without polling. Enabling polling drops the connections, and will not work.
[2.1-BETA1][admin@router.*****.com]/root(31): vmstat -i
interrupt total rate
irq1: atkbd0 374 0
irq15: ata1 68 0
irq17: atapci1 100178 30
irq18: em0 774085 237
irq19: sis0 439173 134
cpu0: timer 6521961 1999
Total 7835839 2402Problem is incredibly high latency with polling enabled.
[2.1-BETA1][admin@router.*****.com]/root(32): top -SH
last pid: 87256; load averages: 1.43, 0.77, 0.41 up 0+01:01:42 15:43:49
128 processes: 4 running, 104 sleeping, 20 waiting
CPU: 0.0% user, 0.0% nice, 100% system, 0.0% interrupt, 0.0% idle
Mem: 76M Active, 257M Inact, 92M Wired, 12M Cache, 56M Buf, 13M Free
Swap: 1024M Total, 1024M FreePID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND
0 root -68 0 0K 64K - 2:03 62.26% kernel{em0 taskq}IPerf
pca@PCA-Linux ~ $ iperf -c 10.1.10.1 -d -i 2 -t 50
–----------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
Client connecting to 10.1.10.1, TCP port 5001
TCP window size: 169 KByte (default)[ 5] local 10.1.10.121 port 35687 connected with 10.1.10.1 port 5001
[ 4] local 10.1.10.121 port 5001 connected with 10.1.10.1 port 50877
[ ID] Interval Transfer Bandwidth
[ 5] 0.0- 2.0 sec 78.9 MBytes 331 Mbits/sec
[ 4] 0.0- 2.0 sec 47.3 MBytes 198 Mbits/sec
[ 5] 2.0- 4.0 sec 87.6 MBytes 368 Mbits/sec
[ 4] 2.0- 4.0 sec 30.4 MBytes 128 Mbits/sec
[ 5] 4.0- 6.0 sec 82.0 MBytes 344 Mbits/sec
[ 4] 4.0- 6.0 sec 39.9 MBytes 168 Mbits/sec
[ 5] 6.0- 8.0 sec 81.5 MBytes 342 Mbits/sec
[ 4] 6.0- 8.0 sec 40.6 MBytes 170 Mbits/sec
[ 5] 8.0-10.0 sec 101 MBytes 423 Mbits/sec
[ 4] 8.0-10.0 sec 23.9 MBytes 100 Mbits/sec
[ 5] 10.0-12.0 sec 79.6 MBytes 334 Mbits/sec
[ 4] 10.0-12.0 sec 45.4 MBytes 190 Mbits/sec
[ 5] 12.0-14.0 sec 90.0 MBytes 377 Mbits/sec
[ 4] 12.0-14.0 sec 35.3 MBytes 148 Mbits/sec
[ 5] 14.0-16.0 sec 68.0 MBytes 285 Mbits/sec
[ 4] 14.0-16.0 sec 56.9 MBytes 239 Mbits/sec
[ 5] 16.0-18.0 sec 77.8 MBytes 326 Mbits/sec
[ 4] 16.0-18.0 sec 46.8 MBytes 196 Mbits/sec
[ 5] 18.0-20.0 sec 90.2 MBytes 379 Mbits/sec
[ 4] 18.0-20.0 sec 35.5 MBytes 149 Mbits/sec
[ 5] 20.0-22.0 sec 91.1 MBytes 382 Mbits/sec
[ 4] 20.0-22.0 sec 35.1 MBytes 147 Mbits/sec
[ 5] 22.0-24.0 sec 92.5 MBytes 388 Mbits/sec
[ 4] 22.0-24.0 sec 33.9 MBytes 142 Mbits/sec
[ 5] 24.0-26.0 sec 89.9 MBytes 377 Mbits/sec
[ 4] 24.0-26.0 sec 36.6 MBytes 154 Mbits/sec
[ 5] 26.0-28.0 sec 91.9 MBytes 385 Mbits/sec
[ 4] 26.0-28.0 sec 34.4 MBytes 144 Mbits/sec
[ 5] 28.0-30.0 sec 92.6 MBytes 388 Mbits/sec
[ 4] 28.0-30.0 sec 33.8 MBytes 142 Mbits/sec
[ 5] 30.0-32.0 sec 89.6 MBytes 376 Mbits/sec
[ 4] 30.0-32.0 sec 36.2 MBytes 152 Mbits/sec
[ 5] 32.0-34.0 sec 87.9 MBytes 369 Mbits/sec
[ 4] 32.0-34.0 sec 38.7 MBytes 162 Mbits/sec
[ 5] 34.0-36.0 sec 82.1 MBytes 344 Mbits/sec
[ 4] 34.0-36.0 sec 44.5 MBytes 187 Mbits/sec
[ 5] 36.0-38.0 sec 88.9 MBytes 373 Mbits/sec
[ 4] 36.0-38.0 sec 37.3 MBytes 156 Mbits/sec
[ 5] 38.0-40.0 sec 83.0 MBytes 348 Mbits/sec
[ 4] 38.0-40.0 sec 42.9 MBytes 180 Mbits/sec
[ 5] 40.0-42.0 sec 85.4 MBytes 358 Mbits/sec
[ 4] 40.0-42.0 sec 40.5 MBytes 170 Mbits/sec
[ 5] 42.0-44.0 sec 83.0 MBytes 348 Mbits/sec
[ 4] 42.0-44.0 sec 41.5 MBytes 174 Mbits/sec
[ 5] 44.0-46.0 sec 89.6 MBytes 376 Mbits/sec
[ 4] 44.0-46.0 sec 32.9 MBytes 138 Mbits/sec
[ 5] 46.0-48.0 sec 86.5 MBytes 363 Mbits/sec
[ 4] 46.0-48.0 sec 39.9 MBytes 167 Mbits/sec
[ 5] 48.0-50.0 sec 78.9 MBytes 331 Mbits/sec
[ 5] 0.0-50.0 sec 2.10 GBytes 361 Mbits/sec
[ 4] 48.0-50.0 sec 47.9 MBytes 201 Mbits/sec
[ 4] 0.0-50.0 sec 978 MBytes 164 Mbits/sec -
Problem is incredibly high latency with polling enabled.
Don't use polling then! (I will comment later on polling.)
[2.1-BETA1][admin@router.*****.com]/root(32): top -SH
last pid: 87256; load averages: 1.43, 0.77, 0.41 up 0+01:01:42 15:43:49
128 processes: 4 running, 104 sleeping, 20 waiting
CPU: 0.0% user, 0.0% nice, 100% system, 0.0% interrupt, 0.0% idle
Mem: 76M Active, 257M Inact, 92M Wired, 12M Cache, 56M Buf, 13M Free
Swap: 1024M Total, 1024M FreePID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND
0 root -68 0 0K 64K - 2:03 62.26% kernel{em0 taskq}What was going on when you took this snapshot? The iperf run?
Your vmstat output doesn't show excessive interrupt rates, but the rates shown are averaged since startup, not "instantaneous" rates. The top output shows a fully loaded CPU, most of the load apparently in em0 taskq which might be quite reasonable, depending on what the system was supposed to be doing. You have a single CPU, a 3GHz Celeron.
If you want something to happen on the polling issue you most likely will have to report it to FreeBSD, say on web page: []http://www.freebsd.org/send-pr.html](http://www.freebsd.org/send-pr.html[url) and provide the output of pfSense shell command```
pciconf -lI don't know the precise meaning of the iperf numbers reported but my gut feel is that they are possibly low for the type of CPU you have spending 100% in system "mode" (as reported by top). I am guessing you have an iperf server installed on your pfSense box, and that the iperf report segment > [ 5] 8.0-10.0 sec 101 MBytes 423 Mbits/sec > [ 4] 8.0-10.0 sec 23.9 MBytes 100 Mbits/sec means that in the interval 8 to 10 seconds from start of test, one stream sent and received at 423Mbps and another stream sent and received at 100Mbps which would make for a total of 2 x (423+100) Mbps or about 1Gbps processed by the box which would pretty much saturate a PCI bus (might be relevant because there are PCI em devices). A bit more context would be helpful in interpreting the numbers you have provided. -
Yeah, the testing is using IPerf. I can get far better numbers with the same test on a linux machine running the IPerf server, with identical hardware. I have re-ran the tests on both to make sure. Also changed to a single thread.
IPerf on Server
pca@PCA-Linux ~ $ iperf -c 10.1.10.71 -i 2 -t 50
–----------------------------------------------------------
Client connecting to 10.1.10.71, TCP port 5001
TCP window size: 22.9 KByte (default)[ 3] local 10.1.10.121 port 34919 connected with 10.1.10.71 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 2.0 sec 205 MBytes 860 Mbits/sec
[ 3] 2.0- 4.0 sec 217 MBytes 911 Mbits/sec
[ 3] 4.0- 6.0 sec 219 MBytes 919 Mbits/sec
[ 3] 6.0- 8.0 sec 217 MBytes 911 Mbits/sec
[ 3] 8.0-10.0 sec 218 MBytes 915 Mbits/sec
[ 3] 10.0-12.0 sec 218 MBytes 916 Mbits/sec
[ 3] 12.0-14.0 sec 218 MBytes 915 Mbits/sec
[ 3] 14.0-16.0 sec 218 MBytes 914 Mbits/sec
[ 3] 16.0-18.0 sec 218 MBytes 914 Mbits/sec
[ 3] 18.0-20.0 sec 218 MBytes 915 Mbits/sec
[ 3] 20.0-22.0 sec 219 MBytes 918 Mbits/sec
[ 3] 22.0-24.0 sec 219 MBytes 917 Mbits/sec
[ 3] 24.0-26.0 sec 217 MBytes 912 Mbits/sec
[ 3] 26.0-28.0 sec 218 MBytes 914 Mbits/sec
[ 3] 28.0-30.0 sec 218 MBytes 915 Mbits/sec
[ 3] 30.0-32.0 sec 219 MBytes 920 Mbits/sec
[ 3] 32.0-34.0 sec 218 MBytes 913 Mbits/sec
[ 3] 34.0-36.0 sec 218 MBytes 913 Mbits/sec
[ 3] 36.0-38.0 sec 219 MBytes 920 Mbits/sec
[ 3] 38.0-40.0 sec 217 MBytes 910 Mbits/sec
[ 3] 40.0-42.0 sec 218 MBytes 915 Mbits/sec
[ 3] 42.0-44.0 sec 219 MBytes 920 Mbits/sec
[ 3] 44.0-46.0 sec 217 MBytes 910 Mbits/sec
[ 3] 46.0-48.0 sec 218 MBytes 915 Mbits/sec
[ 3] 48.0-50.0 sec 218 MBytes 916 Mbits/sec
[ 3] 0.0-50.0 sec 5.31 GBytes 913 Mbits/sec
pca@PCA-Linux ~ $Top on Linux Box.
root@cloud:~# top -SH
top - 16:50:12 up 9 days, 1:03, 2 users, load average: 0.00, 0.00, 0.00
Tasks: 130 total, 1 running, 129 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.7%us, 11.3%sy, 0.0%ni, 86.4%id, 0.0%wa, 0.0%hi, 1.7%si, 0.0%st
Mem: 1553960k total, 336700k used, 1217260k free, 23408k buffers
Swap: 3034104k total, 31976k used, 3002128k free, 131940k cachedPID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6774 root 20 0 27848 1064 908 S 13.9 0.1 0:01.86 iperfIPerf on Router
pca@PCA-Linux ~ $ iperf -c 10.1.10.1 -i 2 -t 50
–----------------------------------------------------------
Client connecting to 10.1.10.1, TCP port 5001
TCP window size: 22.9 KByte (default)[ 3] local 10.1.10.121 port 36129 connected with 10.1.10.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 2.0 sec 120 MBytes 502 Mbits/sec
[ 3] 2.0- 4.0 sec 120 MBytes 505 Mbits/sec
[ 3] 4.0- 6.0 sec 116 MBytes 485 Mbits/sec
[ 3] 6.0- 8.0 sec 117 MBytes 490 Mbits/sec
[ 3] 8.0-10.0 sec 117 MBytes 489 Mbits/sec
[ 3] 10.0-12.0 sec 117 MBytes 490 Mbits/sec
[ 3] 12.0-14.0 sec 120 MBytes 503 Mbits/sec
[ 3] 14.0-16.0 sec 120 MBytes 501 Mbits/sec
[ 3] 16.0-18.0 sec 118 MBytes 497 Mbits/sec
[ 3] 18.0-20.0 sec 119 MBytes 498 Mbits/sec
[ 3] 20.0-22.0 sec 120 MBytes 504 Mbits/sec
[ 3] 22.0-24.0 sec 120 MBytes 504 Mbits/sec
[ 3] 24.0-26.0 sec 120 MBytes 502 Mbits/sec
[ 3] 26.0-28.0 sec 121 MBytes 506 Mbits/sec
[ 3] 28.0-30.0 sec 121 MBytes 506 Mbits/sec
[ 3] 30.0-32.0 sec 123 MBytes 516 Mbits/sec
[ 3] 32.0-34.0 sec 122 MBytes 513 Mbits/sec
[ 3] 34.0-36.0 sec 123 MBytes 516 Mbits/sec
[ 3] 36.0-38.0 sec 123 MBytes 515 Mbits/sec
[ 3] 38.0-40.0 sec 124 MBytes 522 Mbits/sec
[ 3] 40.0-42.0 sec 124 MBytes 522 Mbits/sec
[ 3] 42.0-44.0 sec 125 MBytes 524 Mbits/sec
[ 3] 44.0-46.0 sec 125 MBytes 523 Mbits/sec
[ 3] 46.0-48.0 sec 124 MBytes 522 Mbits/sec
[ 3] 48.0-50.0 sec 124 MBytes 522 Mbits/sec
[ 3] 0.0-50.0 sec 2.95 GBytes 507 Mbits/sec
pca@PCA-Linux ~ $Router TOP during test:
last pid: 44285; load averages: 0.84, 0.62, 0.30 up 0+01:57:56 16:40:03
127 processes: 4 running, 103 sleeping, 20 waiting
CPU: 6.1% user, 0.0% nice, 93.9% system, 0.0% interrupt, 0.0% idle
Mem: 104M Active, 236M Inact, 81M Wired, 16M Cache, 56M Buf, 14M Free
Swap: 1024M Total, 1024M FreePID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND
0 root -68 0 0K 64K - 4:48 67.29% kernel{em0 taskq} -
Anything at all? This router is separating servers from the local network, and I cannot continue to use this router with the sub par performance. I would like to be able to, but not being able to utilize a gigabit network is a bit frustrating.
-
Anything at all?
A few things.
1. As I pointed out in a previous reply, it is possible your iperf performance on your pfSense box is limited by the hardware. Please post the output of pfSense command```
pciconf -l -v2\. Your description doesn't describe clearly the difference in configuration of the tests involving the Linux server and pfSense. Is exactly the same hardware used to run Linux and pfSense? Are the interfaces exactly the same down to the exact same bus type and NIC chipset? 3\. iperf is a useful benchmark but you shouldn't extrapolate iperf performance to packet forwarding performance of a router because iperf moves data between user space and kernel space (an operation well known to be high overhead operation) while routers forward packets between interfaces WITHOUT moving data between kernel and user space. @cncking2000: > This router is separating servers from the local network, I suggest the first thing would be to verify the hardware you have chosen is actually capable of moving data at gigabit speeds between two Gigabit interfaces. If the interface to the servers and the interface to the "local network" are both on the same "standard" PCI bus then it is a physical impossibility to get sustained gigabit speeds between the interfaces. Your pfSense startup output mentions AGP, SiS and doesn't mention SATA all of which suggests to me that the box is of a generation unlikely to have any more than a single standard PCI bus. -
You're expecting gigabit wire speed from the sounds of it. Not going to get it with a Celeron. You're wanting Mustang performance out of a Pinto. The diff in cache between a Celeron and an equivalent Pentium 4 makes for a significant performance difference as a firewall. Commercial firewalls that will push a gigabit wire speed are upwards of $10K USD, and you're trying to use a box that probably wouldn't sell on ebay for $50.
Ditch the sis NIC and Celeron proc and have nothing on a PCI bus, either PCI-X or PCI-e only. Don't use polling.
-
So, What you are suggesting is that I try a HT 3.2Ghz P4 and a 4x PCI-E Dual Intel network card in the 16x PCI-E slot since I have an onboard video in this other box? I can't convince management to spend much on this, but it sure the hell beats using the old Linksys. This facility is too large for a simple router.
-
So, What you are suggesting is that I try a HT 3.2Ghz P4 and a 4x PCI-E Dual Intel network card in the 16x PCI-E slot since I have an onboard video in this other box?
And run a more appropriate benchmark.