TLD Domain count exceeded, prn getting through filter
-
Using production Version 2.1.4_28.
The DNSBL feature "Enable TLD" works great for smaller blacklists. Essentially, it blocks subdomains so like for a porn blacklist badpornsite.com gets blocked, but with TLD www.badpornsite.com also gets blocked even though not specifically on the blacklist. For those using larger blacklists like the decent free ones found at UT1:
http://dsi.ut-capitole.fr/blacklists/index_en.php
however the error ** TLD Domain count exceeded. [ 2500000 ] All subsequent Domains listed as-is ** will probably be in the pfBlockerNG logs. The UT1 porn blacklist alone I think has more than 3 million entries (far exceeding the 2500000 pfBlockerNG limit for routers with 8 gig or more of RAM. So I've been trying to find a way to fix this and found suggestions like to add more RAM, or adjust this or that none of which work. Adding more RAM can work up to the 2500000 limit, but after that no. So one suggestion from way back was to modify the pfblockerng.inc file max Domain count (/usr/local/pkg/pfblockerng/pfblockerng.inc). In the current version this is located around lines 2849-2858 and look like this:// Determine max Domain count available for DNSBL TLD analysis (Avoid Unbound memory exhaustion) $pfs_memory = (round(get_single_sysctl('hw.physmem') / (1024*1024)) ?: 1000); $pfb['pfs_mem'] = array( '0' => '100000', '1500' => '150000', '2000' => '200000', '2500' => '250000', '3000' => '400000', '4000' => '600000', '5000' => '1000000', '6000' => '1500000', '7000' => '2000000', '8000' => '2500000');I have 48 Gig RAM on my system, so I changed the '8000' => '2500000' to a larger number like 6 million instead of the 2.5 million. This appears to be working -- I no longer have the exceeded errors and prn subdomains are now being blocked where before they were not.
Somewhere I read this can be dangerous as if the system runs out of RAM it will crash and then crash again on reboot, requiring a config reload without the modded file to get back up again. So I am wondering if anyone else has tried this and/or has any other thoughts?
-
One side note about this -- on my system using 48 Gig RAM, I typically am using about 25% of the RAM using minimal Suricata rules plus pfBlockerNG, and when the filter lists for pfBlockerNG reload RAM usage is higher, maybe 30-50%.
What that tells me is that most packaged routers today are not coming with near enough memory capability!
-
@supertechie
PfblockerNG 2.1.4_28 is an old unsupported version that has not been updated in years. The package maintainer has recommended that everyone update to the to the latest version of pfBlockerNG-devel 3.1.0_7.
Don't let the "-devel" throw you as it is very stable and been in use for 2-3 years or more.After you update to the latest version, if you are still having issues come back to the forum and someone will be glad to help you.
-
Actually the old version has been actively updated -- I think the last update to it was in September. But yea I should probably try the devel version. I notice the same file in the devel version has fixes for this issue.
Thanks much!
-
@supertechie the update on the legacy version I suspect was to address the RCE
https://nvd.nist.gov/vuln/detail/CVE-2022-31814 -
-
@Unoptanio
Just looking at your screenshots I don't think upping your firewall Maximum Table Entries would help. But you need more physical memory -- 8 Gig is not near enough to turn on all the "toys". I recommend at least 32 Gig these days.
