PHP Memory Allocation Error in pfBlockerNG DNSBL

[SteveITS]

@mpfrench How big is that list? It is probably larger than PHP can fit into memory given the PHP memory limit.

Looks like that can be changed by editing code if that's your desire. Presumably, will need to be done after updates.
https://forum.netgate.com/topic/142702/allowed-memory-size-exhausted/5

[mpfrench]

The error message indicated that less than 128 MB additional space was needed. My dashboard indicates that about 500 MB is available. So it looks as though a change to File:
/usr/local/pkg/pfblockerng/pfblockerng.inc to increase the maximum size by 128 MB would be all that is required.

I took a look at that file but did not see a way to make that change. Perhaps a developer could do it but not me.

[SteveITS]

@mpfrench There’s a Diagnostics/Edit File menu item.

Might be better to optimize the list. is there a smaller one or better way to accomplish your goal?

[SteveITS]

@mpfrench saw you other post discussing DNSBL. You could forward DNS to OpenDNS Family Shield or similar.

[mpfrench]

Steve, thanks for your suggestions. I am familiar with the OpenDNS family shield and Cloudflare's version (https://blog.cloudflare.com/introducing-1-1-1-1-for-families/) but neither one tells us what they specifically block.

The reason I'm trying to get my 1100 to do the job is to know precisely what I'm blocking.

Since it appears the the PHP memory limit needs to be increased by only 128MB, that would not break the unit. Unfortunately, I don't know how to accomplish this code change.
Mike

[SteveITS]

@mpfrench I'm not necessarily advocating this and as I said you'll probably have to do it every upgrade, but:

Load the file, edit, and Save. Change to 192M or 256M or whatever works. I'd guess you'll have to reboot or at least restart PHP-FPM from the console.

[mpfrench]

Steve, your suggestion worked! I changed the 128 to 256 and the UT1 list loading did not cause a memory error message as it did before. However, the update process did not load a part of the database as the log excerpt that follows indicates:

UPDATE PROCESS START [ v3.2.0_3 ] [ 04/3/23 10:13:56 ]

===[ DNSBL Process ]================================================

Loading DNSBL Statistics... completed
Loading DNSBL SafeSearch... disabled
Loading DNSBL Whitelist... completed
Blacklist database(s) ... exists.

[ UT1_adult ] Downloading update ..
Killed

Orig. Unique # Dups # White # TOP1M Final

4504390 0 0 0 0 0

[ UT1_chat ] Downloading update [ 04/3/23 10:19:47 ] ..

Whitelist: laffer.sourceforge.net
Orig. Unique # Dups # White # TOP1M Final

---

167 167 0 1 0 166

[ UT1_child ] Downloading update [ 04/3/23 10:19:51 ] ..

Orig. Unique # Dups # White # TOP1M Final

72 72 0 0 0 72

As you can see, the Adult data was killed but it does say why. I'll wait for a day or two before flagging this as a problem. The problem may be on the downloading site.

I appreciate your help! Why doesn't the base configuration file use the 256M value? I can't be the only person who ran into trouble using the base configuration of config.inc.
Mike

[SteveITS]

@mpfrench Per the code it uses 512 MB for amd64 CPUs. I would guess Netgate assumes ARM has/should use less memory which is true for the 1100 and 3100 but not the 2100.

[mpfrench]

Steve, I'm not sure that I understand your last comment regarding a permanent change to 256M. I have the Netgate 1100, the least capable device Netgate sells. The 256M minimum should run fine on more capable units.

I think the developers should make this change permanent.
Mike

[SteveITS]

@mpfrench I think you'll need to edit the config.inc file after each pfSense upgrade. They are probably trying to be as safe as possible. It all depends on what is being read in to memory...I use pfBlocker but smaller lists so don't have a problem. I've been told not to run a RAM disk on 3100s either but as long as the logging volume is low the RAM usage is low so it's all relative.