Very high CPU usage every 15 minutes
-
Hi,
I have one pfsense box that is alarming CPU usage every 15 minutes. This pfsense has:
openvpn
pfblocker
squid
dual wan
a few VLANsIt is a small company, around 20 users. Following the official pfsense cpu troubleshooting guide I could narrow the issue down to:
/sbin/pfctl -o basic -f /tmp/rules.debug
Image attached.
Can you please help me understand what may be causing this? I compared the CPU metrics with other pfsense boxes and this one is having a huge CPU spike, very different from the normal behavior:
Thanks in advance!
-
pfBlocker can stuff a lot of rules into your ruleset depending on what lists and options you have selected. I would focus on that first. Disable it and see if anything changes.
What are the hardware specs?
-
I uninstalled pfblocker and it got back to normal, then I installed it again and the issue is back.
Now: how to see what in pfblocker is causing this? I can see that it started not long ago, so something was changed which I do not recall.
-
No idea; I don't use pfB. Try asking this in the pfBlocker forum.
-
I don't see this on mine. Are you updating every 15 minutes (I wouldn't think so)?
Using the latest devel version here.Then again, I feel if I've paid for the CPU, I may as well use it. It's not like it's completely pegging it (45%).
-
I just looked at mine...there is no way to update every 15mins; however, I also using the latest devel version. I wish I could use more of my CPU (only 2%) and RAM ( 18% of 16GB).
-
@NollipfSense You can, but it would require editing the cron task.
-
How large is the rules.debug file?
What CPU is that?
That looks pretty extreme.
Steve
-
@stephenw10 said in Very high CPU usage every 15 minutes:
How large is the rules.debug file?
What CPU is that?
That looks pretty extreme.
Steve
Yes, it is pretty bad, looking forward to fix it.
Where is this file located? /tmp? If yes it is currently 38K
38 -rw-r--r-- 1 root wheel 38K Sep 15 20:45 /tmp/rules.debug -
CPU Type Intel(R) Celeron(R) CPU G470 @ 2.00GHz
AES-NI CPU Crypto: NoThanks!
-
@ViniciusBr Are you using a ramdisk or hard drive for /tmp?
-
Same HD, one for everything... this started to happen not long ago.
Before all were ok.
-
Hmm, how long does it take to reload the filter? I have a 35k ruleset here and it does use ~100% of one CPU core while it reloads but it takes maybe 1s to do it.
You have something that is triggering the filter reload every 15mins. Check the system logs.
Steve
-
@stephenw10 said in Very high CPU usage every 15 minutes:
Hmm, how long does it take to reload the filter? I have a 35k ruleset here and it does use ~100% of one CPU core while it reloads but it takes maybe 1s to do it.
How can I check how long it is taking to reload?
About the logs: cannot find anything useful there.
-
Go to Status > Filter Reload, hit the button there. If you have top open in a console at the time you can see how it actually behaves.
Steve
-
@stephenw10 said in Very high CPU usage every 15 minutes:
Go to Status > Filter Reload, hit the button there. If you have top open in a console at the time you can see how it actually behaves.
Steve
Ok, so that is the way of reproducing the issue!
Got the same high CPU usage, just not sure how to troubleshoot from there.
-
Does it show any errors on the reload page? Does it take more than a few seconds?
Something is triggering that reload, probably a result of something else being updated like an alias perhaps. I'd be amased if nothing shows in the system log though. Can we see it?
Steve
-
@stephenw10 said in Very high CPU usage every 15 minutes:
Does it show any errors on the reload page? Does it take more than a few seconds?
Something is triggering that reload, probably a result of something else being updated like an alias perhaps. I'd be amased if nothing shows in the system log though. Can we see it?
Steve
It is taking around 25 seconds to complete.
-
Hmm, no errors that jump out there. It's quite a long list though. 25s seems waaay longer than I might expect. It may be resolving a load of things.
I meant the system log though. Covering at least, say, 30mins so we can see a reload cycle.
Steve
-
I don't see anything strange:
