Resurrecting this thread for two reasons:
1.) Because this is where I landed when newly confronted with the topic using pfBlockerNG-devel 3.2.10 on pfSense CE 2.8.1-RELEASE; and
2.) to confirm that the 'issue' and 'fix' here continue to be viable despite the TLD analysis function being considerably modified since the last post in September 2024.
Current function @ L7255 of /usr/local/pkg/pfblockerng/pfblockerng.inc:
// Determine max Domain count available for DNSBL TLD analysis (Avoid Unbound memory exhaustion)
$pfs_memory = (round(get_single_sysctl('hw.physmem') / (1024*1024)) ?: 1000);
$pfb['pfs_mem'] = [
'0' => '100000',
'1500' => '150000',
'2000' => '200000',
'2500' => '250000',
'3000' => '400000',
'4000' => '600000',
'5000' => '1000000',
'6000' => '1500000',
'7000' => '2000000',
'8000' => '2500000',
'12000' => '3000000',
'16000' => '4000000',
'32000' => '8000000'
];
if ($pfb['dnsbl_py_blacklist']) {
array_walk($pfb['pfs_mem'], function (&$value) {
$value = $value * 3;
});
}
foreach ($pfb['pfs_mem'] as $pfb_mem => $domain_max) {
if ($pfs_memory >= $pfb_mem) {
$pfb['domain_max_cnt'] = $domain_max;
}
}
On a system with 32 GB of RAM attempting to 'analyze' over 24M but less than 27M domains, the line "'32000' => '8000000'" was changed to "'32000' => '9000000'" (i.e., permitting a maximum number of 27M domains to be 'analyzed') in order for the function to complete successfully.
Analyzing (and then subsequently loading) precisely this number of domains...
Original Matches Removed Final
----------------------------------------
24270656 21017552 6463516 17807140
-----------------------------------------
...results in Unbound's stable operational consumption of ~6 GB of RAM and any/all pfBlockerNG 'Reload' options consuming as much as ~6 GB of RAM, concurrently. Therefore one should only attempt this DNSBL hack if they're confident that their system has at least 13 GB of memory 'headroom' (taking into account normal system operation and any other resource-consuming, installed packages).
