V 3.2.0 with pfsense 23.01 RC 20230202

tcw

@jimp's patch just got applied to an updated pfBlockerNG v. 3.2.0_1. (Thanks!) That appears to have been the only change.

Gertjan

Using the latest pfSense RC :

23.01-RC (amd64)
built on Wed Feb 08 06:11:39 UTC 2023

TLD Whitelist selected.
I'm here :

UPDATE PROCESS START [ v3.2.0_1 ] [ 02/8/23 11:13:01 ]

===[  DNSBL Process  ]================================================

Loading DNSBL Statistics... completed
Missing DNSBL stats and/or Unbound DNSBL files - Rebuilding

Loading DNSBL SafeSearch...  enabled
Loading DNSBL Whitelist... completed
Blacklist database(s) ... exists.

[ StevenBlack_ADs ]		 Downloading update .. 200 OK.
 Whitelist: 15.taboola.com|aax-eu.amazon-adsystem.com|adsafeprotected.com|am-match.taboola.com| ..... snipped
 Orig.    Unique     # Dups     # White    # TOP1M    Final                
 ----------------------------------------------------------------------
 177888   177888     0          97         0          177791               
 ----------------------------------------------------------------------

------------------------------------------------------------------------
Assembling DNSBL database...... completed [ 02/8/23 11:13:13 ]
TLD:
TLD analysis.. completed [ 02/8/23 11:13:17 ]
TLD finalize..

and I understand why :

The /tmp/dnsbl_tld_remove file - the list with TLDs to remove is 37000+ lines.
The /var/unbound/pfb_py_data.txt.raw file 133608 lines

[edit]
From what I make of this : each of the 37000+ lines is checked (grepped) with every line in the 133608 file.
So, 37000 times 133608 'greps' to be executed.
That's huge ....

And I have only one dnsbl feed - with "133608" dnsbl entries.
[end edit]

I copied both files to /root/ and repeated the command 'on the command line'.
This command is great to max out one core, 100 %, and it will take minutes if not hours to complete.

pfblockerng-devel does this with PHP handling the return (output). That will make things even worse.

49 degrees and rising. Of to the kitchen, looking for some eggs.

I guess not using (unchecking) Wildcard Blocking (TLD) is the best option right now.

emikaadeo

@gertjan said in V 3.2.0 with pfsense 23.01 RC 20230202:

I guess not using TLD Whitelisting is the best option right now.

I'm not using TLD Whitelist
My DNSBL Mode is set to "Unbound python mode" and as pfBlockerNG states: "TLD Whitelist is not utilized for Unbound python mode! Use DNSBL Whitelist instead."
The main problem is when Wildcard Blocking (TLD) is enabled.

Gertjan

@emikaadeo
You're right :

That's the one :

petrt3522

I believe mine is now having the same issue with my upgrade to 23.01-Final. I manually did a reload and it's at 20 minutes, stuck on "TLD finalize."

I did have an error: On it's first boot I got a banner about this extensive error: https://pastebin.com/aj8q4Mjw than that, It seems to work fine and appears to be passing traffic across 2 VLAN and 1 WAN.

4NVXr3wHBnQYsHwE

This happened to me today as well and likewise disabling Wildcard Blocking (TLD) worked around it. grep was stuck at 100% CPU utilization for several minutes otherwise.

OpIT GmbH

Andy Fix for this? Except disabling Wildcard TLD blocking

SteveITS

@opit-gmbh said in V 3.2.0 with pfsense 23.01 RC 20230202:

Andy Fix for this? Except disabling Wildcard TLD blocking

Not yet: https://www.patreon.com/posts/pfblockerng-v3-2-78781333

emikaadeo

@steveits @jmontleon @OpIT-GmbH
It is now fixed with 3.2.0_3 version :)
https://forum.netgate.com/post/1088962

OpIT GmbH

Yes, working now. THX