Upgraded from 2.3.2_1 to 2.3.3_1 > High CPU usage
-
I have recently upgraded pfSense from 2.3.2_1 to 2.3.3_1 and now I am experiencing high CPU usage.
CPU is frequently on 100% and load averages shows numbers couple hundreds high (up to 300) even though CPU has 4 cores - Supermicro A1SRM-2558F (Intel Atom C2558) with 8 GB RAM and 40 GB SSD. Internet connection: VDSL 30/5, dual WAN (2x PPPoE 30/5).On 2.3.2_1 max load was ~85% (with 30 Mb/s P2P download, active Snort and some other packages) in idle it was 5-15%. On 2.3.2_1 and all previous versions I never saw load averages higher than ~4.
On 2.3.3_1 it goes to 100% (load average shows numbers up to 300), WebGUI stops responding and even internet connection is down for couple of seconds.From SSH to pfSense and top -aSH I saw, that most CPU power was used by process /usr/local/sbin/filterdns -p /var/run/filterdns.pid. In fact, there was 15-20 of those processes.
Additional thing that I noticed was, that on 2.3.3_1 Snort was blocking IPs on whitelist.Any ideas what can I try, before I revert back to 2.3.2_1 or try reinstall 2.3.3_1?
top -aSH from SSH:
last pid: 6507; load averages: 114.86, 38.44, 14.80 up 0+00:05:27 22:31:02 1672 processes:13 running, 1620 sleeping, 39 waiting CPU: 95.6% user, 0.1% nice, 4.3% system, 0.1% interrupt, 0.0% idle Mem: 1129M Active, 80M Inact, 353M Wired, 640M Buf, 6326M Free Swap: 16G Total, 16G Free Message from syslogd@pfsense at Mar 18 22:27:56 ... TIME WCPU COMMAND 11se php-fpm[186155 ki31 0K 64K RUN 0 3:34 14.06% [idle{idle: cpu0}] 11 root 155 ki31 0K 64K RUN 3 2:57 7.08% [idle{idle: cpu3}] 66886 root 73 0 16868K 3168K RUN 1 0:01 5.27% /sbin/pfctl -nf /tmp/rules.test.packages 96161 root 34 0 733M 314M nanslp 2 0:00 4.98% /usr/local/bin/snort -R 23410 -D -q --suppress-conf 96161 root 34 0 733M 314M bpf 1 0:00 4.98% /usr/local/bin/snort -R 23410 -D -q --suppress-conf 1285 root 52 0 271M 40040K piperd 2 0:00 3.27% /usr/local/bin/php-cgi -f /usr/local/pkg/servicewat 51538 root 52 0 427M 132M umtxn 0 0:01 2.88% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 26 0 427M 132M uwait 3 0:01 2.10% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 52 0 427M 132M umtxn 1 0:01 1.46% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 21 0 427M 132M umtxn 3 0:00 1.27% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 21 0 427M 132M umtxn 2 0:00 1.27% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 42805 root 52 0 272M 42744K sysctl 3 0:01 1.17% php-fpm: pool nginx (php-fpm) 51538 root 52 0 427M 132M umtxn 0 0:01 1.17% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 99134 root 8024 72 0 75868114.86, 38.44, 14.80 0:00 1.17% /usr/local/bin/snort -R 401 up 0+00:05:27pr22:31:02 51538 root 6 21 0 1627M 132M 1 zombie, 39 waiting.17% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 7216891.1per 0.90 0 2 8.0K 7796K C 0.0 2 0:02 1.07% top -aSH 51538 root 21 0 427M 132M um1xn 1 0500 1.07% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 52 0 427M 132M uwait 1 0:01 0.98% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 21 0 427M 132M umtxn 1 0:00 0.98% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 21 0 427M 132M umtxn 0 0:00 0.98% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 21 0 427M 132M umtxn 1 0:00 0.98% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 21 0 427M 132M umtxn 1 0:00 0.98% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 21 0 427M 132M umtxn 3 0:00 0.58% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 21 0 427M 132M umtxn 3 0:00 0.88% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 91538 root 20 0 427M 132M umtxn 0 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 1538 root 20 0 427M 132M RUN 3 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 20 0 427M 133M umtxn 3 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 21 0 427M 133M umtxn 1 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 51538 root 21 0 427M 133M umtxn 3 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 72168 gasper 21 0 25952K 7796K CPU2 2 0:02 1.37% top -aSH 99134 72 75868K 18508K RUN 0 3 bin/snort -R 40157 -D -q --suppress-conf 51538 21 427M 133M umtxn 0 1.27% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 5 21 3 2 0 1.2 42805 52 272M 42743 1 php-fpm: pool nginx (php-fpm) 51538 root 1 427M 133M umtxn 1 0 /usr/local/sbin/filterdns -p /var/run/filterdns.pid 5 52 3M uwait 1 0.98 5 21 3M umtxn 0 5 3 0 5 3 1 5 3 5 3 0 5 3 5 3 1 5 1 3 3 8 52404 52 272M 37923K piperd 0 php-fpm: pool nginx (php-fpm) 5 0 3 3 5 0 3 5 3 1last pid: 11749; load averages: 0.25, 22.31, 32.43 up 0+01:44:01 00:09:36 155 processes: 5 running, 111 sleeping, 39 waiting CPU: 0.0% user, 0.0% nice, 0.0% system, 3.2% interrupt, 96.8% idle Mem: 23M Active, 98M Inact, 304M Wired, 508K Cache, 761M Buf, 7462M Free Swap: 16G Total, 16G Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root 155 ki31 0K 64K CPU3 3 76:15 100.00% [idle{idle: cpu3}] 11 root 155 ki31 0K 64K CPU1 1 74:44 100.00% [idle{idle: cpu1}] 11 root 155 ki31 0K 64K RUN 2 74:44 99.17% [idle{idle: cpu2}] 11 root 155 ki31 0K 64K CPU0 0 72:48 90.77% [idle{idle: cpu0}] 12 root -92 - 0K 624K WAIT 0 5:45 7.76% [intr{irq271: igb3:que}] 12 root -92 - 0K 624K WAIT 0 1:16 1.86% [intr{irq256: igb0:que}] 12 root -92 - 0K 624K WAIT 1 0:44 0.10% [intr{irq257: igb0:que}] 98484 root 20 0 14508K 2296K select 2 1:13 0.00% /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/lo 0 root -16 - 0K 432K swapin 3 0:56 0.00% [kernel{swapper}] 12 root -88 - 0K 624K WAIT 3 0:51 0.00% [intr{irq277: ahci1}] 12 root -92 - 0K 624K WAIT 3 0:45 0.00% [intr{irq259: igb0:que}] 12 root -92 - 0K 624K WAIT 2 0:40 0.00% [intr{irq258: igb0:que}] 12 root -60 - 0K 624K WAIT 2 0:34 0.00% [intr{swi4: clock}] 0 root -92 - 0K 432K - 0 0:25 0.00% [kernel{dummynet}] 18 root 21 - 0K 16K psleep 3 0:20 0.00% [bufdaemon] 12 root -92 - 0K 624K WAIT 1 0:07 0.00% [intr{irq272: igb3:que}] 12 root -92 - 0K 624K WAIT 3 0:06 0.00% [intr{irq274: igb3:que}] 12 root -92 - 0K 624K WAIT 2 0:05 0.00% [intr{irq273: igb3:que}] 5 root -16 - 0K 16K pftm 2 0:04 0.00% [pf purge] 14410 root 20 0 16676K 2236K bpf 0 0:04 0.00% /usr/local/sbin/filterlog -i pflog0 -p /var/run/fil 61870 www 20 0 35684K 11112K kqread 3 0:03 0.00% /usr/local/sbin/haproxy -f /var/etc/haproxy/haproxy 15 root -16 - 0K 16K - 3 0:02 0.00% [rand_harvestq] 12 root -92 - 0K 624K WAIT 0 0:02 0.00% [intr{irq266: igb2:que}] 84816 root 52 20 17000K 2544K wait 1 0:02 0.00% /bin/sh /var/db/rrd/updaterrd.sh 74441 dhcpd 20 0 24840K 13656K select 3 0:02 0.00% /usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chr 54328 root 20 0 23204K 2376K nanslp 0 0:01 0.00% [dpinger{dpinger}] 54222 root 20 0 23204K 2376K nanslp 1 0:01 0.00% [dpinger{dpinger}] 54661 root 20 0 19108K 2304K nanslp 2 0:01 0.00% [dpinger{dpinger}] 68094 root 20 0 39144K 7392K kqread 3 0:01 0.00% nginx: worker process (nginx)Just got top -aSH from high CPU usage:
last pid: 93085; load averages: 274.32, 229.63, 159.74 up 0+02:07:12 00:32:47 1655 processes:29 running, 1587 sleeping, 39 waiting CPU: 97.3% user, 0.0% nice, 2.1% system, 0.6% interrupt, 0.0% idle Mem: 162M Active, 103M Inact, 336M Wired, 368K Cache, 826M Buf, 7287M Free Swap: 16G Total, 16G Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 98484 root 24 0 14508K 2296K RUN 0 1:24 4.59% /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/lo 44891 root 21 0 691M 170M umtxn 3 0:00 1.76% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 3 0:00 1.66% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 3 0:00 1.56% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 2 0:00 1.37% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 3 0:00 1.27% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 11763 gasper 20 0 25952K 9504K CPU3 3 0:08 1.17% top -aSH 44891 root 21 0 691M 170M umtxn 3 0:00 1.17% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 1 0:00 1.17% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 2 0:00 0.98% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 3 0:00 0.88% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 3 0:00 0.88% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 2 0:00 0.88% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 1 0:01 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 1 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 0 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 1 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 0 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 1 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 3 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 2 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 2 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 2 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 1 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 1 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 2 0:00 0.78% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 3 0:01 0.68% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 20 0 691M 170M umtxn 2 0:00 0.68% /usr/local/sbin/filterdns -p /var/run/filterdns.pid 44891 root 21 0 691M 170M umtxn 1 0:00 0.68% /usr/local/sbin/filterdns -p /var/run/filterdns.pid
Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.