PFsense 2.3.2 on Esxi 4.1 - SPIKE CPU Usage
-
Running 2.3.2 and 2.2.6 release on VMware ESXI 4.1 ML350 HP hardware.
2.2.6 has a nice smooth response, has squid running, pfblocker, show a cpu spike when needed and necessary. (Bottom graph, spiked when it should have)PLEASE SEE the attached machine images.
What can be causing this?
I'm thinking that this is FREEBSD thing and not pfsense.
Can anyone help?
I want to upgrade the 2.2.6 machine, but I don't want to until I can see why this machine is spiking like it is…
(And NO, there is no one on the web-interface watching this, all console are disconnected, so it not a user driven thing)Any suggestions where to look?
Thanks
-
What does top say from console? And yes, you need to upgrade both pfSense and ESXi. I can't believe you're running on 4.1 still.
-
@KOM:
What does top say from console? And yes, you need to upgrade both pfSense and ESXi. I can't believe you're running on 4.1 still.
Yes, there are still many many 4.1 machines living a long life….... In production. These however are not in a critical environ.
TOP output below doesn't show -what I can see- anything unusual.
Have been using 226 since it came out, and pfsense back to 1.3 years ago. Never had a problems with cpu usage on these 4.1 esxi units. On average, these 2 firewalls have low usage all the time, but as you can see the spikes, which occur 1 per min? .. is unusual.
Be happy to collect stats to determine what this is....
Now that its off hours I am going to start stopping services, like pfblocker, snort, etc. and disable the nics and see what it does.
Good plan?
-
Killed every service running, disconnected all nics, (thru vmware, and then in pfsense. )
Still a 'spiker…. '
I really think its a freebsd thing... what else goes 'once per minute' what other timer pops are there.?
-
Take a look at the crontab so see if ti's firing something every minute. Does pfSense monitoring show a corresponding CPU spike that matches the VMware spike?
-
@KOM:
Take a look at the crontab so see if ti's firing something every minute. Does pfSense monitoring show a corresponding CPU spike that matches the VMware spike?
Yes they do,, great idea looking at that….
And its a little different because the time samples are not exactly the same, but look at the graph from pfsense below... yes VMware and pfsense indicate the same SPIKEY CPU ...
-
Here are two graphs from the same hour….
The difference is that the PFsense graph is at 1 min intervals and the vmware graph is at (i think 20 sec intervals. )
Still you can see that the USER Util and the system Util have those spikes.... and its only me, there are no other people logged in the pfsense. No other web interfaces active, no active main console.I guess I'm saying that 2.2.6 had less overall (AVG) cpu utilization, than the 2.3.2 install.
Same platform, and this WAS a 2.2.6 machine that I upgraded thru the gui to 2.3.2, it was not a fresh install.
I have a snapshot of the old machine, rolled back, and its avg cpu was 3-5%....
Average now is 18%..... but because of the spikes, they drive the avg higher.
So it just 'looks' like its using more cpu to VMware? .... no it really is? ...hanging head, shaking back and forth, thinking........I could have opened a deli, and served samagizzes... ... :-\
-
Temporarily disable both snort and ntopng and see if the problem persists. Your top output showed a mainly idle system. You might want to run it for awhile an watch it for processes that spike it.
-
@KOM:
Temporarily disable both snort and ntopng and see if the problem persists. Your top output showed a mainly idle system. You might want to run it for awhile an watch it for processes that spike it.
Disabled all services that I had running….........
No change.
Doing a deep dive... -
The only thing I see now using TOP from the pfsense console and setting the update to 1 sec.. – is PFCTL using 10% cpu , then gone..... .01% ... then 8-12%-- then .01 % ---
It's definitely PFCTL doing it. TOP is showing that PFCTL goes to 15%-18%-27%, then 0%
This is coinciding with the VMWARE graph on the host hypervisor.
hmmmmm -
Another poster recommended doing TOP -SH for a while and then look…. Left it running for 45min?
Here's the output showing a spike of pfctl at 11.96% and a screenshot right after that not showing pfctl.
Otherwise... nothing I see here....Hope this helps --
An additional screenshot of the DASHboard showing one of those spikes. 51%
Most of the time the dash cpu is 1-4%...
-
Anything in your System or Gateways logs during the spike?