2.4.5.a.20200110.1421 and earlier: High CPU usage from pfctl
-
Read the thread. All the info is here. At least read from https://forum.netgate.com/post/908806 down.
-
@jimp said in 2.4.5.a.20200110.1421 and earlier: High CPU usage from pfctl:
Read the thread. All the info is here. At least read from https://forum.netgate.com/post/908806 down.
Instead of replying arrogantly, why not assuming I've just done that 5 times and try to give a more "to the point" answer?
Why do you think I am asking if there's any development over this situation? Maybe because of your post stating that you're "assessing the next steps" (the same you just gave me a link to go and read again...).Reducing my system to a single core installation is not doable (done that... performance went to the ground) and unfortunately reducing the tables to 65000 entries makes pfBlockerNG go wild and start throwing errors.
So... Expecting any development over this situation? is there any kind of patch we can do? can we "override" the setting that exists that causes this problem?
-
While terse, my response was not "arrogant". It's not arrogant to expect people to read a thread with all of the information (and links to more information). Far too often people pop in and expect others to do the leg work for them when all of the information is here. If you'd read from my linked post down and followed the links, you'd have all your answers. I just skimmed them and checked. The info is here, and in the links (Like https://redmine.pfsense.org/issues/10414 linked in https://forum.netgate.com/post/909130).
There are multiple comments with suggested workarounds and how to enact them.
-
Working in IT as I do, I understand your point and where you're coming from... However would have been much nicer to start with a: "Have you read the thread already? Did you notice that we've already pointed to possible workarounds?"
To which I'd reply that I've seen the workarounds and that none of them work for me (unfortunately I must add) at least without having another impact....Thanks for the link to the redmine part, I did not notice that one previously, apologies!
I've noticed that there's a patch for the kernel and good results are visible... Do we know or have an idea when that's coming out?
Thanks for your time.
-
We don't all have time to be nice, especially when there is no indication of what the person posting has done. If you had included the additional info about exactly what you had tried in your first comment, that would have been even more helpful. We're not mind readers.
The workarounds do work, at a possible performance penalty (hurts different deployments worse than others). The main workaround is reducing the CPU cores to 1, which is mentioned several times, and that will work 100% of the time for everyone. If that was too much of a performance hit, then you will need to disable all the large tables, move it to hardware with faster single cores, or go back to 2.4.4-p3.
No ETA on 2.4.5-p1 other than "Soon" (as in Weeks, not months). Still some testing left to do on other issues being rolled into 2.4.5-p1 to address other issues discovered in the release.
-
@ghosterius said in 2.4.5.a.20200110.1421 and earlier: High CPU usage from pfctl:
Is there any development on this situation? Are we having some 2.4.5_p1 coming up soon to solve this?
I have the pfSense running on an Hyper-V and there's absolutely nothing I can do on the pfSense without having an huge impact (outage, traffic gone, website and console unresponsive).I've attempted reverting back to 2.4.4_p3 but... unfortunately you guys removed the image available for me to reinstall it so... oops!
Sure. Just reduce to one virtual CPU. EOF
-
So glad I found this thread.
I have had this with my 2.4.5 install using proxmox.
Now reverted to 1 core for the time being which seems to just about cope.
Thread followed to watch out for 2.4.5-p1.
Thanks for debugging this guys! -
https://redmine.pfsense.org/projects/pfsense/roadmap#2.4.5-p1 looks very soon.
-Rico
-
No ETA on 2.4.5-p1 other than "Soon" (as in Weeks, not months)
looks very soon.
now you have done it..
it seems that @techpro2004 is still away ... I'm sure he will come back to ask.. when ... "Soon" (as in Weeks, not months)! -
I bet he can't be far away since at least you now pinged him.
-Rico
-
I love that I am very new to this forum but still know exaclty what you are talking about.
-
Appears to affect physical hardware as well. I have been investigating unexplained CARP state changes between a HA pair with Netgate C2758 hardware. The CARP state change is always preceded by a filter reload. Thanks @jimp and team for tracking this down. Looking forward to 2.4.5-p1 to fix.
-
@mjh_ca looks like the last open bug was merged yesterday if I am reading the redmine right. Can't be too long :) Then I can add all my cpu core back!
-
@tomahhunt said in 2.4.5.a.20200110.1421 and earlier: High CPU usage from pfctl:
@mjh_ca looks like the last open bug was merged yesterday if I am reading the redmine right. Can't be too long :) Then I can add all my cpu core back!
There are 4 more hidden bugs, so hopefully they're done soon!
-
@psylenced said in 2.4.5.a.20200110.1421 and earlier: High CPU usage from pfctl:
There are 4 more hidden bugs, so hopefully they're done soon!
3 of those are just administrivia things like updating the docs, release notes, and blog. Just one "bug" left and it should be solved just waiting for internal confirmation. Main thing we're waiting on now is internal testing of the release images.
-
@jimp said in 2.4.5.a.20200110.1421 and earlier: High CPU usage from pfctl:
Main thing we're waiting on now is internal testing of the release images.
Any way we could help with testing an "RC" kinda version?
-
Not in this case since there aren't many changes and we were able to confirm the original issue and the fix internally. We're not going to have a long RC period and if testing goes well, it should be out next week sometime.
If it was going to be in RC for a while we might have made public snapshots but in this case a short cycle is warranted.
-
Thought so but wanted to offer the help nonetheless :)
-
https://forum.netgate.com/topic/154337/pfsense-2-4-5-release-p1-now-available
-
2.4.5-p1 did not fix the issue for me. Or it is a different issue with very similar symptoms?Netgate C2758 hardware, HA configuration. LAGG to switches. Configuration has been rock solid for years and unchanged, since upgrading to 2.4.5 I have had issues with unexpected CARP failovers. Thought the L2 switches had gone bad so I replaced them with Cisco switches, no improvement.Correction - the 2.4.5-p1 high CPU fix does fix my issue.
Somehow both units were incorrectly in "Persistent CARP Maintenance Mode" (likely that way before the upgrade). Taking them out of maintenance mode, and the upgrade to 2.4.5-p1 for high CPU fix, seems to have resolved my CARP state change issues. Thank you Netgate!
