PFBlockerNG 2.1.1_2 Memory Errors

[Gerard64]

Thanks for the tip and help!
I changed the file like you wrote and i run the command in a putty terminal on the router now it hangs like 5 minutes on:

[2.3.2-RELEASE][root@router.xxxxx.xxx]/root: php /usr/local/www/pfblockerng/pfblockerng.php dc
Country code update Start [ 08/03/16 22:51:59 ]
Converting MaxMind Country databases for pfBlockerNG.
Processing ISO IPv4 Continent/Country Data
Processing ISO IPv6 Continent/Country Data

The prompt is gone it seems its waiting for something.

[Gerard64]

After a long wait it outputs:

[2.3.2-RELEASE][root@router.xxxxx.xxx]/root: php /usr/local/www/pfblockerng/pfblockerng.php dc
Country code update Start [ 08/03/16 22:51:59 ]
Converting MaxMind Country databases for pfBlockerNG.
Processing ISO IPv4 Continent/Country Data
Processing ISO IPv6 Continent/Country Data
Creating pfBlockerNG Continent XML files
IPv4 Africa
IPv6 Africa
IPv4 Antarctica
IPv6 Antarctica
IPv4 Asia
IPv6 Asia
IPv4 Europe
IPv6 Europe

Fatal error: Allowed memory size of 314572800 bytes exhausted (tried to allocate 72 bytes) in /usr/local/www/pfblockerng/pfblockerng.php on line 728

Call Stack:
    0.0075    422488  1. {main}() /usr/local/www/pfblockerng/pfblockerng.php:0
  603.9766  12588288  2. pfblockerng_get_countries() /usr/local/www/pfblockerng/pfblockerng.php:105
  612.1411  44556872  3. explode() /usr/local/www/pfblockerng/pfblockerng.php:728

PHP ERROR: Type: 1, File: /usr/local/www/pfblockerng/pfblockerng.php, Line: 728, Message: Allowed memory size of 314572800 bytes exhausted (tried to allocate 72 bytes)[2.3.2-RELEASE][root@router.xxxxxxx.xxx]/root:

[RonpfS]

Yes it takes 5+minutes to complete

Try increasing to 400M, 500M up to 750M

[Gerard64]

Thanks  RonpfS ..!
After I set it to 500M it worked.

[Qinn]

@Gé:

Thanks  RonpfS ..!
After I set it to 500M it worked.

Thanks @RonpfS setting the memory_limit on line 236 to 300M fixed it, strange that @Gé needed 500M?

[Perforado]

For me i needed even 500M didn't work :/ 640M worked for me.

[RonpfS]

@Perforado:

For me i needed even more as 500M didn't work :/ 768M worked for me.

Which version of pfsense?
x386 or amd64 ?
how much ram ?

[Perforado]

CPU: Intel(R) Atom(TM) CPU  C2558  @ 2.40GHz (2400.06-MHz K8-class CPU)
RAM: 8192MB

pfSense 2.3.3.a.20160803.1621
pfBlockerNG 2.1.1_2

[RonpfS]

Well on amd64 the default is 512MB.

/var in Ramdisk ? getting full ?

[RonpfS]

@oswoldy:

Ok, while the php /usr/local/www/pfblockerng/pfblockerng.php dc command works, the cron jobs are still crashing and giving errors, I am currently at 2GB limit and climbing.

If you run the dc command, it changes the GeoIP database, you have to run a force reload after the dc complete.

Also check diagnostic system activity to see if there isn't something unusual.
Look at the System log for hints on the failure mode.
An can you post the portion of the pfblockerng.log file where you have failure.

[RonpfS]

@oswoldy:

@RonpfS:

@oswoldy:

Ok, while the php /usr/local/www/pfblockerng/pfblockerng.php dc command works, the cron jobs are still crashing and giving errors, I am currently at 2GB limit and climbing.

If you run the dc command, it changes the GeoIP database, you have to run a force reload after the dc complete.

Also check diagnostic system activity to see if there isn't something unusual.
Look at the System log for hints on the failure mode.
An can you post the portion of the pfblockerng.log file where you have failure.

Ok, I ran the dc command, followed by a force reload, no different. If I remove pfB_NAmerica_v4 then it works fine, crash report is:

Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p5
FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016    root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[04-Aug-2016 10:29:24 Europe/London] PHP Fatal error:  Allowed memory size of 524288000 bytes exhausted (tried to allocate 20 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3875
[04-Aug-2016 10:29:24 Europe/London] PHP Stack trace:
[04-Aug-2016 10:29:24 Europe/London] PHP  1. {main}() /usr/local/www/pfblockerng/pfblockerng.php:0
[04-Aug-2016 10:29:24 Europe/London] PHP  2. sync_package_pfblockerng() /usr/local/www/pfblockerng/pfblockerng.php:87
[04-Aug-2016 10:29:24 Europe/London] PHP  3. file() /usr/local/pkg/pfblockerng/pfblockerng.inc:3875

I have reduced the limit back to 500MB as increasing it wasnt making a difference, the dc command still completes but not a force reload or CRON.

Once the dc command complete, the MaxMind database is created, so you do not need to re run it.

What about the pfblockerng.log ? What are the symptoms ?
Maybe raise it to 640M or 768M?

[wiz561]

I started a thread up over in the pfblocker posting and just letting everybody know that I'm also experiencing very similar memory issues.

Thismorning when I logged in, I also had a pfsense crash report with the following:

					Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p5
FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[04-Aug-2016 00:18:40 America/Chicago] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3868
[04-Aug-2016 00:18:40 America/Chicago] PHP Stack trace:
[04-Aug-2016 00:18:40 America/Chicago] PHP   1\. {main}() /etc/rc.start_packages:0
[04-Aug-2016 00:18:40 America/Chicago] PHP   2\. sync_package() /etc/rc.start_packages:90
[04-Aug-2016 00:18:40 America/Chicago] PHP   3\. eval() /etc/inc/pkg-utils.inc:631
[04-Aug-2016 00:18:40 America/Chicago] PHP   4\. sync_package_pfblockerng() /etc/inc/pkg-utils.inc(631) : eval()'d code:3
[04-Aug-2016 00:18:40 America/Chicago] PHP   5\. array_merge() /usr/local/pkg/pfblockerng/pfblockerng.inc:3868

Filename: /var/crash/minfree
2048

[lucasrca]

How I solved my problem:

My pfSense config:

• Version: 2.3.2 (amd64), running on VMWare 6
  – Snort
  -- pfBlockerNG
  -- OpenVPN
  -- Open-vm-tools
  -- DHCP Relay
  -- Quagga OSPFd with another 2 pfSense.
• ~3000 users simultaneously
• 2 x 100 Mbit uplinks
• 16 GB RAM
• 80 GB SAS
• CPU Type: Intel(R) Xeon(R) CPU E5-4620 v2 @ 2.60GHz
• 16 CPUs: 8 package(s) x 2 core(s)
• 8 Interfaces, including WAN
• routing, filtering and relaying dhcp to 16 branches over MPLS, WiMax and fiber
• Load balance and failover
• QoS with Traffic shaper

1. Updated Firewall Maximum Table Entries: 4000000 -> 8000000
   1.1) Reboot
2. Edited /usr/local/pkg/pfblockerng/pfblockerng.inc and set memory limit to 500M
3. Executed php /usr/local/www/pfblockerng/pfblockerng.php dc
4. It's alive.

Thanks to all involved.

[Mithrondil]

POST-INSTALL script failed
Message from GeoIP-1.6.9:
GeoIP does not ship with the actual data files. You must download
them yourself! Please run:

/usr/local/bin/geoipupdate.sh

[RonpfS]

@wiz561:

I started a thread up over in the pfblocker posting and just letting everybody know that I'm also experiencing very similar memory issues.

Thismorning when I logged in, I also had a pfsense crash report with the following:

					Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p5
FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[04-Aug-2016 00:18:40 America/Chicago] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3868
[04-Aug-2016 00:18:40 America/Chicago] PHP Stack trace:
[04-Aug-2016 00:18:40 America/Chicago] PHP   1\. {main}() /etc/rc.start_packages:0
[04-Aug-2016 00:18:40 America/Chicago] PHP   2\. sync_package() /etc/rc.start_packages:90
[04-Aug-2016 00:18:40 America/Chicago] PHP   3\. eval() /etc/inc/pkg-utils.inc:631
[04-Aug-2016 00:18:40 America/Chicago] PHP   4\. sync_package_pfblockerng() /etc/inc/pkg-utils.inc(631) : eval()'d code:3
[04-Aug-2016 00:18:40 America/Chicago] PHP   5\. array_merge() /usr/local/pkg/pfblockerng/pfblockerng.inc:3868

Filename: /var/crash/minfree
2048

The php memory should be 512M by default, so setting it in the inc file to 400M might not help.
Did it fail without any fix to the inc file? Can you try setting the limit to 640M or 768M?
What do you have for Firewall Maximum Table Entries?

[RonpfS]

@Mithrondil:

POST-INSTALL script failed
Message from GeoIP-1.6.9:
GeoIP does not ship with the actual data files. You must download
them yourself! Please run:

/usr/local/bin/geoipupdate.sh

You will need to run  php /usr/local/www/pfblockerng/pfblockerng.php dc from the shell.
Once it succeeds, you should be able to install, it may need a reboot.

[RonpfS]

For those with failed installation. Verify that /var isn't full. The MaxMind database is huge so if you are using a RAM Disk, it might eat up memory that is needed for the pfblockerng.php.

If disk space is running low, BBcan177 suggest to delete the /var/db/pfblockerng/deny and /var/db/pfblockerng/original folders before installation to free some disk space. This means it will need to redownload all IP feeds after installation.

Check pfblockerng.log, the system log, Dashboard for crash report, Status Monitoring System Memory.

Post relevant debug info here.

[wiz561]

@RonpfS:

@wiz561:

I started a thread up over in the pfblocker posting and just letting everybody know that I'm also experiencing very similar memory issues.

Thismorning when I logged in, I also had a pfsense crash report with the following:

					Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p5
FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[04-Aug-2016 00:18:40 America/Chicago] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3868
[04-Aug-2016 00:18:40 America/Chicago] PHP Stack trace:
[04-Aug-2016 00:18:40 America/Chicago] PHP   1\. {main}() /etc/rc.start_packages:0
[04-Aug-2016 00:18:40 America/Chicago] PHP   2\. sync_package() /etc/rc.start_packages:90
[04-Aug-2016 00:18:40 America/Chicago] PHP   3\. eval() /etc/inc/pkg-utils.inc:631
[04-Aug-2016 00:18:40 America/Chicago] PHP   4\. sync_package_pfblockerng() /etc/inc/pkg-utils.inc(631) : eval()'d code:3
[04-Aug-2016 00:18:40 America/Chicago] PHP   5\. array_merge() /usr/local/pkg/pfblockerng/pfblockerng.inc:3868

Filename: /var/crash/minfree
2048

The php memory should be 512M by default, so setting it in the inc file to 400M might not help.
Did it fail without any fix to the inc file? Can you try setting the limit to 640M or 768M?
What do you have for Firewall Maximum Table Entries?

I'm going to have to do a bit of work this weekend on this and see more.  I can try it and see what happens, but I'm getting concerned about reaching my 2gb memory limit.  I can add more memory, but that requires me to go to the store and I'm kinda lazy…..and cheap.  :)

Interesting that I too have a similar setup to the one user above.  I'm running it on vmware esxi with Snort (disabled) and OpenVPN.  Granted, I don't have nearly the hardware or setup, but it's interesting that we're both running vmware.

[wiz561]

OK…  Now maybe I'll try what others suggested...  :)

I wiped and reinstalled pfsense tonight and pfblockng is still coming back with that crash and memory errors.  I know others said to adjust the memory but I thought I would give this a try.  Unfortunately, it failed.

I also had issues with php-fpm having high utilization....so I'm hoping that the wipe/reinstall fixed the issue with that.  Time will tell.

[RonpfS]

What about posting pfblockerNG, system log, crash report, screen shot of system activity, etc, so we can see what is happening on your setup?

The crash report you posted earlier tells me you have under 400MB defined.

PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted

Did you raise the Firewall Maximum Table Entries ?