High cpu usage with packages enabled - squid, snort,etc

[messerchmidt]

hi everyone

using an old core 2 e6420  on a gigabyte p35 with 4gb ddr2, nvidia 210 pcie graphics, dual intel pcie pro/1000 nics, and a 250gb hd. i am the only user here. internet is cable @ 250/20 with the isp modem in bridge mode.

using pfsense 2.2.1 x64 with the following plugins.

1. squid3 dev
2. squidguard dev
3. HVAP (disabled as it gives me issues)
4. pfblockerng
5. snort
6. bandwidthd
7. sarg
8. Service Watchdog
9. iperf

i have squid3 running as a http (not https ) transparent proxy. setup wdap using the online guide and urlresolver instead of the forwarder.

everything is working fine except that the cou stays at 50%+ load as in this picture, even when there is no traffic. Also, the squid3 package turns off every so often. I installed that Service Watchdog to restart it.

squid 3 has 30,000mb for cache in a /cache folder as per the one of the online guides.

Any idea what pluggin is causing this cpu load?

[pwnell]

I'd just log in to bash (ssh) and perform a "top"

[messerchmidt]

@pwnell:

I'd just log in to bash (ssh) and perform a "top"

can you be more specific? ssh login with putty I can do - what is a top? what is the exact command?

[messerchmidt]

did a google search of top and pfsense, and found this: https://forum.pfsense.org/index.php?topic=43339.0

went diagnostics - system activity and here it is:

last pid: 18925;  load averages:  1.18,  1.25,  1.20  up 0+08:14:31    19:12:29
157 processes: 5 running, 112 sleeping, 40 waiting

Mem: 95M Active, 437M Inact, 249M Wired, 948K Cache, 136M Buf, 3154M Free
Swap:

PID USERNAME PRI NICE  SIZE    RES STATE  C  TIME    WCPU COMMAND
  17 root    -16 ki-1    0K    16K CPU1    1 493:54 100.00% [idlepoll]
  11 root    155 ki31    0K    32K RUN    0 264:45  68.65% [idle{idle: cpu0}]
  11 root    155 ki31    0K    32K RUN    1 193:18  28.17% [idle{idle: cpu1}]
78363 root      22    0  224M 33480K piperd  0  0:00  0.20% php-fpm: pool lighty (php-fpm)
6865 root      20    0  771M  398M bpf    0  1:38  0.00% /usr/local/bin/snort -R 12483 -D -q –supp
    0 root    -16    0    0K  224K swapin  0  0:49  0.00% [kernel{swapper}]
  12 root    -60    -    0K  640K WAIT    0  0:13  0.00% [intr{swi4: clock}]
43228 root      20    0 54892K  8796K kqread  0  0:08  0.00% /usr/local/sbin/lighttpd -f /var/etc/light
    5 root    -16    -    0K    16K pftm    0  0:08  0.00% [pf purge]
32917 root      20    0 16812K  2660K bpf    0  0:04  0.00% /usr/local/sbin/filterlog -i pflog0 -p /va
  276 root      20    0  224M 23916K kqread  0  0:04  0.00% php-fpm: master process (/usr/local/lib/ph
  21 root      16    -    0K    16K syncer  0  0:03  0.00% [syncer]
45523 unbound  20    0 55728K 31732K kqread  1  0:03  0.00% /usr/local/sbin/unbound -c /var/unbound/un
22681 root      20    0 49772K 14808K nanslp  0  0:03  0.00% /usr/local/bin/barnyard2 -r 12483 -f snort
80311 root      20    0 14664K  2400K select  0  0:03  0.00% /usr/sbin/syslogd -s -c -c -l /var/dhcpd/v
38431 root      20    0 12464K  2232K select  0  0:02  0.00% /usr/local/sbin/apinger -c /var/etc/apinge
  15 root    -16    -    0K    16K -      0  0:02  0.00% [rand_harvestq]
63378 root      52  20 17144K  2488K wait    1  0:02  0.00% /bin/sh /var/db/rrd/updaterrd.sh

I have "Enable device polling" on as my network cards support it - could that be the cause? how do i get idlepoll working properly?

[messerchmidt]

disabled idlepol and rebooted, cpu usage is back to zero.

is there a way to get it working without it sucking so much cpu power? seems to speed things up by negating irqs.

POST disabling idlepolling and rebooting

last pid: 27923;  load averages:  0.86,  0.76,  0.34  up 0+00:02:07    19:20:12
159 processes: 3 running, 116 sleeping, 40 waiting

Mem: 427M Active, 57M Inact, 232M Wired, 972K Cache, 96M Buf, 3218M Free
Swap:

PID USERNAME PRI NICE  SIZE    RES STATE  C  TIME    WCPU COMMAND
  11 root    155 ki31    0K    32K CPU0    0  1:44  95.36% [idle{idle: cpu0}]
  11 root    155 ki31    0K    32K RUN    1  1:26  95.26% [idle{idle: cpu1}]
59214 root      39    0  224M 40040K piperd  1  0:00  0.59% php-fpm: pool lighty (php-fpm)
8632 proxy    52    0 73644K 13072K nanslp  1  0:00  0.39% /usr/local/sbin/squid -f /usr/pbi/squid-am
    0 root    -16    0    0K  224K swapin  0  0:49  0.00% [kernel{swapper}]
    4 root    -16    -    0K    32K -      0  0:00  0.00% [cam{doneq0}]
  12 root    -60    -    0K  640K WAIT    0  0:00  0.00% [intr{swi4: clock}]
    4 root    -16    -    0K    32K -      0  0:00  0.00% [cam{scanner}]
43555 root      20    0 50796K  7424K kqread  0  0:00  0.00% /usr/local/sbin/lighttpd -f /var/etc/light
71515 root      20    0  763M  360M bpf    0  0:00  0.00% /usr/local/bin/snort -R 12483 -D -q –supp
45428 unbound  20    0 43440K 21796K kqread  0  0:00  0.00% /usr/local/sbin/unbound -c /var/unbound/un
    5 root    -16    -    0K    16K pftm    0  0:00  0.00% [pf purge]
  12 root    -92    -    0K  640K WAIT    0  0:00  0.00% [intr{irq256: em0:rx 0}]
  54 root      -8    -    0K    16K mdwait  1  0:00  0.00% [md1]
  12 root    -92    -    0K  640K WAIT    1  0:00  0.00% [intr{irq259: em1:rx 0}]
33182 root      20    0 16812K  2408K bpf    1  0:00  0.00% /usr/local/sbin/filterlog -i pflog0 -p /va
95254 root      20    0 28172K 18076K select  1  0:00  0.00% /usr/local/sbin/ntpd -g -c /var/etc/ntpd.c
  12 root    -88    -    0K  640K WAIT    0  0:00  0.00% [intr{irq262: ahci1:ch}]

[doktornotor]

What? You enabled the horrible Device polling "feature" in System: Advanced: Networking? Kindly do not touch any Network Interfaces defaults there unless you absolutely know what you are doing. (I requested this item to be removed multiple times to no avail, allegedly it is "useful" for someone. The only use without exception for anyone who ever touched that polling checkbox was a CPU burn-in test.)

[messerchmidt]

ahh ok thanks for the reply. i thought it would speed things up. guess not.