pfBlockerNG-devel 2.2.5_30 "Cannot allocate memory..."

Co6aka

"Cannot allocate memory..." when updating/reloading IP tables. Device is the SG-3100 with 2GB RAM...

I gradually increased "Firewall Maximum Table Entries" from 400,000 to 9,000,000 with no effect. I then sub-divided the tables (spread feeds across many tables) until the errors stopped. However, today, I started getting an error with one of the sub-divided tables that contains only one feed; the old feed contained 36k entries and the new feed contains 43k entries. So, now I'm stuck. On the Dashboard, memory usage (with 9,000,000 max) indicates approximately "30% of 2028 MiB" so I don't see how actual memory consumption could be the problem. From the log:

pfSense Table Stats

table-entries hard limit 9000000
Table Usage Count 207361

Error message:
12:22:53 There were error(s) loading the rules: /tmp/rules.debug:37: cannot define table pfB_BAD_1Wa_v4: Cannot allocate memory - The line in question reads [37]: table <pfB_BAD_1Wa_v4> persist file "/var/db/aliastables/pfB_BAD_1Wa_v4.txt"

I was also getting the similar IPv6 bogons error until I disabled IPv6. I just tested this again - re-enabled IPv6 - and get the error message...

13:40:35 There were error(s) loading the rules: /tmp/rules.debug:19: cannot define table bogonsv6: Cannot allocate memory - The line in question reads [19]: table <bogonsv6> persist file "/etc/bogonsv6"

I've gone through all of the reboots (soft and hard) with and without pfB enabled. I haven't yet removed pfB and then re-installed/configured though.

Seems to me this is likely a memory allocation bug in pfB.

You can increase that beyond 9 million. You will, most likely, have to reboot to make that work. There isn't an issue with pfblocker but there is an issue with how tables are loaded into pf with pfctl. You will most likely notice latency and packet loss when the tables are loaded. It will only happen when something changes causing the filters to reload.

This is a 2.4.5 only issue.

Co6aka

@jwj said in pfBlockerNG-devel 2.2.5_30 "Cannot allocate memory...":

This is a 2.4.5 only issue.

Currently I'm on 2.2.5_30, and until it was released, the previous devel version.

I'll increase "Firewall Maximum Table Entries" to 100,000,000 and see what happens, but I don't see how there can't be an issue of some kind when... "table-entries hard limit 9000000,
Table Usage Count 207361" which is a difference of 8,792,639 entries. The total of my raw feeds is probably under 1,000,000 entries. And the IPv6 bogons issue remains.

Co6aka

"Firewall Maximum Table Entries" is now 100,000,000. The reboot is automatically offered.

No errors upon boot, so I presume this is the right "direction" to go, but 100-million max table entries to support only 207372 resulting entries? Maybe it's 100MB rather than 100M table entries now? Anyway, memory usage is still 30% in Dashboard. Later, I'll try re-combining a divided table (move all feeds back into one table) and see what happens. Also, I'll re-enable IPv6 and see what happens with the bogons.

RonpfS

Look at https://forum.netgate.com/topic/151690/increased-memory-and-cpu-spikes-causing-latency-outage-with-2-4-5/108

Co6aka

This is with "Firewall Maximum Table Entries" set to 100,000,000...

21:23:48 There were error(s) loading the rules: /tmp/rules.debug:39: cannot define table pfB_BAD_1W_IN_v4: Cannot allocate memory - The line in question reads [39]: table <pfB_BAD_1W_IN_v4> persist file "/var/db/aliastables/pfB_BAD_1W_IN_v4.txt"

Previously the table had 113,795 lines/records, so apparently adding a few more broke it.

Should I increase to 200,000,000 or just go right to 1,000,000,000 entries max?

LinuxTracker

@Co6aka I'm in the same boat as you. Upgraded from 2.4.4 to 2.4.5. Wound up with "Cannot allocate memory" errors & only the firewall could access the internet. Uninstalling/reinstalling pfBlocker_NG gets the LAN back online (I know it isn't a pfBlocker issue).

My table entries were at 20 million before upgrading - because I have a lot of lists and some of them are massive (each list does have a purpose). I think I worked up to 60M entries before setting this aside for the night.

I haven't tried breaking apart my lists into smaller aliases. After reading the relevant posts here and on Reddit, it doesn't seem likely to help. It'd still be the same number of IPs that need allocation.

(wild guess coming) Unless the issue is that the structures holding my massive aliases are buckling under the load. But, heck. I don't know.

I'm going to sleep on it. Maybe tomorrow I'll puzzle out where I should be looking for clues. Otherwise, I'll have to check into rolling back - wait for bigger brains to set our world right (yet) again.

Edit: box has 4GB RAM

Q: How do I calculate Firewall Maximum Table Entries (assume 100MB in aliastables dir)

Edit.2: I haven't been able to find a fix. Going to roll back.
and
I'm fairly impressed w/ the difficulty of locating a download link for
pfSense-CE-2.4.4-RELEASE-p3-amd64.iso.gz
Not giving up!

Edit3: Found a copy of 2.4.4 on Linuxtracker.org (not affiliated).
Installed a fresh copy. Restored from the backup I made using 2.4.5 (because, you know) and that worked just fine. Everything came right up; no issues at all.

I'm all good again. I'm also scared of upgrading any of my boxes to 2.4.5 but what can you do.
I still appreciate all the work that goes into this.