High CPU Usage after upgrading to 22.05

Mike-moon

@mike-moon said in High CPU Usage after upgrading to 22.05:

Hi all, I have applied the fix referenced above but I still have 30-40% CPU usage at an i3 with 3.7GHz. With the previous version 22.01 the CPU usage was typically below 10%.
Could there be an additional issue?
Thanks, Mike

Sorry, I made a mistake when applying the fix: I removed the bracket between the two quotes but did not replace that bracket by a space as required. After correcting this the CPU usage jumped down and the core temperatures decreased about 10 degrees. So everything is fine.
Mike

tohil

Hi,

Upgraded the first firewall to 22.05, but I'm not experiencing high cpu load by pfblockerng.
has this issue already been fixed in a new package release?

/usr/local/pkg/pfblockerng/pfblockerng.inc on line 4139 at my install:

 foreach ($list_type as $ip_type => $vtype) {
                        if (!empty($config['installedpackages'][$ip_type]['config'])) {
                                foreach ($config['installedpackages'][$ip_type]['config'] as $key => $list) {
                                        if (!is_array($list)) {
                                                $list = array();
                                        }
                                        if (!is_array($list['row'])) {
                                                $list['row'] = array();
                                        }

                                        if ($vtype == '_v4') {
                                                $list['vtype'] = '_v4';
                                        } else {
                                                $list['vtype'] = '_v6';
                                        }

                                        // Collect list array key location
                                        $list['key'] = "{$key}";

                                        // If only the 'customlist' is defined. Remove the 'List row' data.
                                        if (empty($list['row'][0]['url'])) {
                                                unset($list['row']);
                                        }

fireodo

@tohil said in High CPU Usage after upgrading to 22.05:

/usr/local/pkg/pfblockerng/pfblockerng.inc on line 4139 at my install:

Maybe your Texteditor is counting wrong:

Here at line 4139 is "$r = explode(' ', $result, 2);"

tohil

@fireodo said in High CPU Usage after upgrading to 22.05:

$r = explode(' ', $result, 2)

Hi

I'm using vi and jump to:4139

I even cannot find that line....
vi /usr/local/pkg/pfblockerng/pfblockerng.inc

fireodo

@tohil said in High CPU Usage after upgrading to 22.05:

@fireodo said in High CPU Usage after upgrading to 22.05:

$r = explode(' ', $result, 2)

Hi

I'm using vi and jump to:4139

I even cannot find that line....
vi /usr/local/pkg/pfblockerng/pfblockerng.inc
If you trust me:
pfblockerng.txt
You have to change the .txt to .inc
Changes are allready made - you can replace the file directly.

tohil

@fireodo
Okay, now its going kind of weird. I've compared my file (putty.log) with your text...
it seems like the file version is not the same....
alt text

fireodo

@tohil
Hmmm,

I have here pfblocker 3.1.0_4 devel and this pfblocker.inc file

tohil

@fireodo
I have pfBlocker 2.1.4_27

Is this issue just with dev version?
Checked the bugtracker entry:

I'm running this on 22.01-Release and this was CPU load was not occurring prior to 3.1.0.4.

So everyone not using pfBlockerNG higher than 3.1.04 will be affected...

fireodo

@tohil said in High CPU Usage after upgrading to 22.05:

@fireodo
I have pfBlocker 2.1.4_27

Is this issue just with dev version?

Ooooh - in this case the file is NOT for this old Version!!! But the problem is the same the line number must be in this old version different - I suggest to upgrade to the devel version wich is actually maintained! But its in the end your decision!

Edit: If you find in your pfblocker .inc

$r = explode(')', $result, 2);

then you should replace it with:

$r = explode(' ', $result, 2);

tohil

@fireodo
https://forum.netgate.com/topic/156604/pfblockerng-vs-pfblockerng-devel/7

https://forum.netgate.com/topic/172036/solved-pfblocker-stable-vs-devel/3

It seems I have to update and go to the devel version on all my installs...

fireodo

@tohil said in High CPU Usage after upgrading to 22.05:

@fireodo
https://forum.netgate.com/topic/156604/pfblockerng-vs-pfblockerng-devel/7

It seems I have to update and go to the devel version on all my installs...

That would (from my point of view) be a good move ...
(as far I recall, there should be no problems and all your settings are preserved - but BACKUP is allways recommended )

johnpoz

@mike-moon said in High CPU Usage after upgrading to 22.05:

I removed the bracket between the two quotes but did not replace that bracket by a space as required

I made the same mistake doing the patch by hand - hehehe

Yup what a difference.

I was thinking that "fix" didn't seem to do much.. Then as you can see from graph after putting in the space between the 's cpu util dropped off to normal, and yup did see a couple of degrees difference in the temp as well.

FrankZappa

@johnpoz Johnpoz, can you give a "how to for idiots" on changing the script?
I've never changed the script file and not sure how to do it e.g. What software (Putty?/Text Editor, etc) and where do I go to change it.

Sorry, I'm a newbie at this.

tohil

@frankzappa
Hi

connect to your pfsense via putty.

enter the following

vi /usr/local/pkg/pfblockerng/pfblockerng.inc

Then press "ESC" followed by ":" enter line number 4139

cursor jumps to the line.

go to the part to change, like the e of explode. press ESX followed by x. this removes char by char. delete all behind the e until then ;

$r = ;

then copy the new part

explode(' ', $result, 2)

Press ESC and I

then paste it with right click.

ESC / wq to save

reboot

hope this helps

sretalla

@frankzappa

Just do it from the pfSense GUI.

In the Diagnostics menu, go to Edit File

Type (or copy/paste) /usr/local/pkg/pfblockerng/pfblockerng.inc into the "Path of file to be edited" box and click "Load"

Type 4139 in the "Go to line #" box just below that and to the right, then click the button.

Make the edit (change the '?' to ' ' ensuring to make a space between the single quotes)

Click the "Save" button.

FrankZappa

@sretalla Thanks folks, that seemed to work (using Edit File from GUI). Didn't know there was a GUI option to do that (although I'm familiar with Putty as well). Thanks for the help.

FrankZappa

@frankzappa Update: Everything working Great! CPU usage is way down as well as temps.. This was an awesome fix!
As an aside: Holy Cow! You have to be pretty good at coding to find that error. Some smart dudes figured that one out. I'm not one of them!!!

GPinzone

I noticed the "pfBlockerNG DNSBL service" was stopped after rebooting. I was able to start it from the dashboard. Just thought I'd mention it.

BTW, a reboot was required to get the CPU down. Restarting the pfBlocker services wasn't enough.

Edit: I think I spoke too soon:

CPU Activity

last pid:  9211;  load averages:  1.54,  0.92,  0.51  up 0+00:18:07    12:10:32
526 threads:   6 running, 490 sleeping, 30 waiting
CPU: 10.2% user,  0.1% nice, 10.1% system,  0.4% interrupt, 79.1% idle
Mem: 487M Active, 149M Inact, 476M Wired, 2701M Free
ARC: 185M Total, 57M MFU, 123M MRU, 565K Anon, 1030K Header, 4060K Other
     73M Compressed, 205M Uncompressed, 2.82:1 Ratio

  PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
44959 root        102    0    30M    19M RUN      0   1:59  96.58% /usr/local/sbin/lighttpd_pfb -f /var/unbound/pfb_dnsbl_lighty.conf
   11 root        155 ki31     0B    32K RUN      0  14:20  31.59% [idle{idle: cpu0}]
45117 root         42    0    60M    41M RUN      0   0:35  30.47% /usr/local/bin/php -f /usr/local/pkg/pfblockerng/pfblockerng.inc index
   11 root        155 ki31     0B    32K RUN      1  14:20  24.37% [idle{idle: cpu1}]
   12 root        -84    -     0B   480K WAIT     0   0:08   4.59% [intr{irq16: uart0+}]
    0 root        -76    -     0B   528K -        1   0:08   1.56% [kernel{if_io_tqg_1}]
    0 root        -76    -     0B   528K -        0   0:06   1.27% [kernel{if_io_tqg_0}]
   23 root        -16    -     0B    16K mmcsd    1   0:02   0.88% [mmcsd0: mmc/sd card]
45086 root         20    0    61M    41M piperd   0   0:01   0.29% /usr/local/bin/php -f /usr/local/pkg/pfblockerng/pfblockerng.inc dnsbl
12762 root         20    0   415M   360M bpf      0   0:06   0.20% /usr/local/bin/snort -R _51743 -D -q --suppress-config-log --daq pcap --daq-mode passive --treat-drop-as-alert -l /var/log/snort/snort_ix351743 --pid-path /var/run --nolock-pidfile --no-interface-pidfile -G 51743 -c /usr/local/etc/snort/snort_51743_ix3/snort.conf -i ix3{snort}
   32 root        -16    -     0B  5088K -        0   0:01   0.20% [zpool-pfSense{zio_write_issue_hig}]
  383 root         22    0   132M    46M piperd   1   0:06   0.10% php-fpm: pool nginx (php-fpm)
    0 root        -16    -     0B   528K swapin   0   0:24   0.00% [kernel{swapper}]
68621 root         20    0    29M  9232K kqread   0   0:08   0.00% nginx: worker process (nginx)
68836 root         20    0    28M  8616K kqread   1   0:08   0.00% nginx: worker process (nginx)
  382 root         52    0   132M    46M accept   1   0:06   0.00% php-fpm: pool nginx (php-fpm)
60893 unbound      20    0    90M    70M kqread   1   0:05   0.00% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound}
35420 root         52    0   132M    45M accept   0   0:04   0.00% php-fpm: pool nginx (php-fpm)
    0 root        -76    -     0B   528K -        1   0:02   0.00% [kernel{if_config_tqg_0}]
   32 root        -12    -     0B  5088K -        0   0:02   0.00% [zpool-pfSense{zio_write_issue}]
60893 unbound      20    0    90M    70M kqread   0   0:01   0.00% /usr/local/sbin/unbound -c /var/unbound/unbound.conf{unbound}
   14 root         -8    -     0B    48K -        0   0:01   0.00% [geom{g_up}]
45749 root         23    0    61M    41M piperd   1   0:01   0.00% /usr/local/bin/php -f /usr/local/pkg/pfblockerng/pfblockerng.inc queries
   32 root         -8    -     0B  5088K tx->tx   0   0:00   0.00% [zpool-pfSense{txg_thread_enter}]
   32 root        -16    -     0B  5088K -        1   0:00   0.00% [zpool-pfSense{zio_write_issue_hig}]
   12 root        -72    -     0B   480K WAIT     1   0:00   0.00% [intr{swi1: netisr 0}]
   12 root        -60    -     0B   480K WAIT     1   0:00   0.00% [intr{swi4: clock (0)}]
   32 root        -16    -     0B  5088K -        0   0:00   0.00% [zpool-pfSense{zio_write_intr_high}]
   32 root        -16    -     0B  5088K -        0   0:00   0.00% [zpool-pfSense{zio_ioctl_intr}]
   19 root        -16    -     0B    16K pftm     0   0:00   0.00% [pf purge]

lohphat

The patch doesn't seem to be working for me. I have a vanilla pfBlockerNG-devel config and I start getting failed DNS lookups within an hour of starting the service. The patch is applied.

EDIT: The DNS failures are caused by a bug in unbound. There are work-arounds (disable IPv6 or tell unbound to serve expired records). Check out the "Slow DNS after upgrading to 22.05 thread).

wc2l

I have also done this. It worked for a while. This morning I'm seeing a high CPU usage 80 - 100% pretty steady. I have edited line 4139 (done through the GUI.. Checked again this morning.

Not sure what is causing the issue. Is there some suggestions? Where do we look to figure out the cause??

Sorry, I'm still learning about this stuff!!