PfBlockerNG high CPU

[serbus]

Hello!

This might give more insight into how to deal with the shield/amplitude or other similar devices/services...

https://developers.amplitude.com/docs/http-api-v2

Pfb will probably block the link, but it is interesting...at least as long as you are the vampire and not the "donor".

John

[lunaticfringe80]

@Gertjan said in PfBlockerNG high CPU:

@lunaticfringe80 said in PfBlockerNG high CPU:

I don't want to be forced to sacrifice logging to solve this

You don't need to : see what I proposed above.

Your suggestion above was quite literally to disable logging. See the word just to the left of what you suggested I switch? Also, I'm reading both sinkholed and web server.

@serbus Your suggestion to use a custom block page that sends a 404 appears to be working. So far so good.

[Gertjan]

@lunaticfringe80 said in PfBlockerNG high CPU:

Also, I'm reading both sinkholed and web server.

Important note : 'English' is not my native language. More somewhat between second and third.
I'm probably confounding "Black hole" with "Sink hole"'.

And you're right : When you put the/a DNSBL in own it's DNDBL feed file - a manually created feed - prioritize it and then set Logging to "Disabled" for this DNSBL file then there are no more logs. Because the device will not access the network, because it can't use a obtained 0.0.0.0 address.
This is where I consider that 0.0.0.0 is a black hole (== nothing comes in (into pfSense) because nothing comes out (of the device).
Logging won't work at that moment for very obvious reasons.

If you keep the VIP as the "Sink hole" then the device will "hit" the pfBlockerNG-devel web server - and leave a log trace at that moment.
With all the related consequences ..... as the device won't take no for an answer == what created the problem raised in this thread.

[Spacecase]

I know I'm a bit late to the party, but I had a similar issue. I'm posting this to document the cause and the solution.

In my case, I correlated the high CPU load to an unusually high amount of LAN traffic. I isolated it to my Windows-10 desktop machine. Further isolation using the Windows 'Resource Monitor' app identified the activity on the LAN causing the high CPU load; it was Malwarebytes attempting to send its telemetry back to the mothership. It was doggedly persistent. Once I allowed the traffic, all was back to normal. Unfortunately, the clever 404 PHP solution in post 75 didn't work.

[inline]

@Spacecase I can confirm the issue, I had one machine that was trying to reach out to MalwareBytes telemetry and killing the CPU on my pfSense box.

Once I disabled telemetry, issue resolved. I turned it on again and saw the issue, turned it off again and problem gone.

[RVAIBC]

Problem:
High CPU attributed to pfBlockerNG.
Issue was traced to a single user with Malwarebytes by examining the pfBlockerNG DNSBL.LOG

Solution:
Put ".malwarebytes.com" in the DNSBL Whitelist.

Effect:
CPU returned to its usual value of about 5% (J1900)
from 33% (one core almost fully used)

[provels]

@RVAIBC
Worked for me, thank you!

[sdd]

Thanks everyone for the helpful post.

I wanted to share my story here for the next person from the future that finds it.

This process was previously taking 80% of the CPU, without ever letting up:

78554 root        20    0  9016K  5576K kqread  0   0:02   0.78%  /usr/local/sbin/lighttpd_pfb -f /var/unbound/pfb_dnsbl_lighty.conf

I was concerned because that kept the temps on my SG-3100 higher than I was comfortable with.

The key insight for me was that lighttpd_pfb is probably pfBlocker's web server that sends out the "blocked" web page, so something on my network must be spamming it.

Root Cause
A device on my local network occasionally tries to phone home to some.service.com (that's the fake FQDN I'll use). It doesn't do this all the time, but when the attempt fails due to DNSBL, it re-tries repeatedly, causing the CPU spike.

Solution
I whitelisted the domain in DNSBL, similar to what others have done. However, I'm not simply giving in to a app's persistent whining to reach a service that I'd prefer it didn't. So, I created a host alias for some.service.com and blocked it in the firewall.

Now the DNS queries go through, and then the request is rejected. pfBlocker's web server is not needed for this, so the CPU load is back to normal.

[BBcan177]

@lunaticfringe80
Just create a new DNSBL Group, and add that domain to the custom list, with Logging set to disabled, and Priority as Primary. Force Reload to apply.

The new Unbound Python mode is better situated to handle this condition.

[dpseattle]

@bbcan177 AWESOME tip to create a new dnsbl group. Amazon devices were constantly calling api.amplitude.com. Followed your instructions and pfsense down to <10% cpu and <.5 load. THANK YOU!

[sdd]

@bbcan177 An awesome tip, thanks! I wrote my previous post after I first solved the issue and was ready to shelve it for awhile, but I just revisited it and did what you said. More or less the same behavior, but it's much easier!

[chrcoluk]

Hit the same issue myself, everything ran fine for years, but two things happened.

Letting neighbour use my network currently, as they got no broadband, and they have a TV that is absolutely unreal in terms of DNS traffic, hence recently all me doing stuff on pfblockerng.

Decided to change pfblockerng cron from hourly to daily as I had nothing updating more often than daily anyway.

This combination seems to have unsettled the pfblockerng web server, I wouldnt personally call this a sinkhole as its a webserver responding to requests, a sinkhole is a null route like replying with 0.0.0.0.

Obvious solution is to stop using the VIP filtering, if that keeps all the dnsbl logging then no issue, but I read in the thread VIP, stats only accrue from VIP traffic.

I see a ton of states in fin wait, so looking to see if the time outs can be reduced, seeing also if the web server is actually caching content or fetching its index from storage every time.

I see its configured with 4096 bit keys, over kill for this sort of thing and also a top end EC.

The index.php seems to be deliberatly configured to not cache, but I can see why, as its used for logging stuff, which would break if cached by the client, I think I will just move some stuff of the web server.