Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    PFBlockerNG 2.1.1_2 Memory Errors

    Scheduled Pinned Locked Moved pfBlockerNG
    61 Posts 17 Posters 65.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • RonpfSR
      RonpfS
      last edited by

      @rajl:

      I'm having constant "out of memory" errors related to the latest release of PFBlockerNG.  An example error message reads:

      
      /usr/local/www/pfblockerng/pfblockerng.php: PHP ERROR: Type: 1, File: /usr/local/pkg/pfblockerng/pfblockerng.inc, Line: 3874, Message: Allowed memory size of 536870912 bytes exhausted (tried to allocate 72 bytes)
      
      

      I have a really simple PFBlockerNG ruleset – block all incoming connections except for IPv4 and IPv6 connections from the United States + the ad-blocker built into PFBlocker.

      I've seen other people on this forum report similar problems, but their solutions don't seem to work for me.  For example, I have already increased the setting for Firewall Maximum Table Entries from 2000000 to 4000000 (see e.g. https://forum.pfsense.org/index.php?topic=102470.msg643960#msg643960) but continue to get memory errors.

      Any other ideas?

      This is also related to the MaxMind db issue. But fixing this in that part of the code isn't so simple as BBcan177 is away at the moment.

      Check the selection of countries in the GeoIP tabs, someone reported that the selection was inverted at some point.

      2.4.5-RELEASE-p1 (amd64)
      Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
      Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

      1 Reply Last reply Reply Quote 0
      • R
        rajl
        last edited by

        What should I be looking for?  I only have one country selected (United States) and I did invert the rule (i.e. block all inbound connections except from U.S.).  Are there any solutions other than to change the maximum number of table entries?  I have  8GB of RAM installed in my PFSense box, so the idea of running out of memory is a bit flumoxing as by all measures, I should have more than enough.

        Thanks

        1 Reply Last reply Reply Quote 0
        • RonpfSR
          RonpfS
          last edited by

          That was just to make sure that nothing got crooked in the config. One users reported that the selection was inverted somehow on his setup.

          The only fix will come from BBcan177, but for now is doesn't have affordable Internet access somewhere in the middle of the sea. :(

          2.4.5-RELEASE-p1 (amd64)
          Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
          Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

          1 Reply Last reply Reply Quote 0
          • RonpfSR
            RonpfS
            last edited by

            The problem seems to be related to i386 installation.
            The fix is to edit /usr/local/pkg/pfblockerng/pfblockerng.inc  (Diagnostics/Edit File)
            go to line 236

            change```
            pfb_global();

            // DNSBL Lighttpd HTTPS Daemon (Scans Lighttpd dnsbl_error.log for requested https domain names)
            if (isset($argv[1]) && $argv[1] == 'dnsbl') {

            to```
            pfb_global();
            ini_set('memory_limit', '300M');
            
            // DNSBL Lighttpd HTTPS Daemon (Scans Lighttpd dnsbl_error.log for requested https domain names)
            if (isset($argv[1]) && $argv[1] == 'dnsbl') {
            

            I tested with 128M, 200M and it fails
            256M succeeds, but BBcan177 recommends using 300M. You might need more depending on your system.

            So for i386, the default is 128M so very low, it should be 256M or more depending on you setup.
            For amd64 system, the default is 512M, so you might need to bump it to 640M or 768M.

            after the modifying the file, go to shell and run

            php /usr/local/www/pfblockerng/pfblockerng.php dc

            Once this succeed, you do not need to re-run the dc command. The GeoIP db will change next month.

            2.4.5-RELEASE-p1 (amd64)
            Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
            Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

            1 Reply Last reply Reply Quote 0
            • G
              Gerard64
              last edited by

              Thanks for the tip and help!
              I changed the file like you wrote and i run the command in a putty terminal on the router now it hangs like 5 minutes on:

              [2.3.2-RELEASE][root@router.xxxxx.xxx]/root: php /usr/local/www/pfblockerng/pfblockerng.php dc
              Country code update Start [ 08/03/16 22:51:59 ]
              Converting MaxMind Country databases for pfBlockerNG.
              Processing ISO IPv4 Continent/Country Data
              Processing ISO IPv6 Continent/Country Data

              The prompt is gone it seems its waiting for something.

              1 Reply Last reply Reply Quote 0
              • G
                Gerard64
                last edited by

                After a long wait it outputs:

                [2.3.2-RELEASE][root@router.xxxxx.xxx]/root: php /usr/local/www/pfblockerng/pfblockerng.php dc
                Country code update Start [ 08/03/16 22:51:59 ]
                Converting MaxMind Country databases for pfBlockerNG.
                Processing ISO IPv4 Continent/Country Data
                Processing ISO IPv6 Continent/Country Data
                Creating pfBlockerNG Continent XML files
                IPv4 Africa
                IPv6 Africa
                IPv4 Antarctica
                IPv6 Antarctica
                IPv4 Asia
                IPv6 Asia
                IPv4 Europe
                IPv6 Europe

                Fatal error: Allowed memory size of 314572800 bytes exhausted (tried to allocate 72 bytes) in /usr/local/www/pfblockerng/pfblockerng.php on line 728

                Call Stack:
                    0.0075    422488  1. {main}() /usr/local/www/pfblockerng/pfblockerng.php:0
                  603.9766  12588288  2. pfblockerng_get_countries() /usr/local/www/pfblockerng/pfblockerng.php:105
                  612.1411  44556872  3. explode() /usr/local/www/pfblockerng/pfblockerng.php:728

                PHP ERROR: Type: 1, File: /usr/local/www/pfblockerng/pfblockerng.php, Line: 728, Message: Allowed memory size of 314572800 bytes exhausted (tried to allocate 72 bytes)[2.3.2-RELEASE][root@router.xxxxxxx.xxx]/root:

                1 Reply Last reply Reply Quote 0
                • RonpfSR
                  RonpfS
                  last edited by

                  Yes it takes 5+minutes to complete

                  Try increasing to 400M, 500M up to 750M

                  2.4.5-RELEASE-p1 (amd64)
                  Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                  Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                  1 Reply Last reply Reply Quote 0
                  • G
                    Gerard64
                    last edited by

                    Thanks  RonpfS ..!
                    After I set it to 500M it worked.

                    1 Reply Last reply Reply Quote 0
                    • QinnQ
                      Qinn
                      last edited by

                      @Gé:

                      Thanks  RonpfS ..!
                      After I set it to 500M it worked.

                      Thanks @RonpfS setting the memory_limit on line 236 to 300M fixed it, strange that @Gé needed 500M?

                      Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                      Firmware: Latest-stable-pfSense CE (amd64)
                      Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                      1 Reply Last reply Reply Quote 0
                      • PerforadoP
                        Perforado Rebel Alliance
                        last edited by

                        For me i needed even 500M didn't work :/ 640M worked for me.

                        1 Reply Last reply Reply Quote 0
                        • RonpfSR
                          RonpfS
                          last edited by

                          @Perforado:

                          For me i needed even more as 500M didn't work :/ 768M worked for me.

                          Which version of pfsense?
                          x386 or amd64 ?
                          how much ram ?

                          2.4.5-RELEASE-p1 (amd64)
                          Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                          Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                          1 Reply Last reply Reply Quote 0
                          • PerforadoP
                            Perforado Rebel Alliance
                            last edited by

                            CPU: Intel(R) Atom(TM) CPU  C2558  @ 2.40GHz (2400.06-MHz K8-class CPU)
                            RAM: 8192MB

                            pfSense 2.3.3.a.20160803.1621
                            pfBlockerNG 2.1.1_2

                            1 Reply Last reply Reply Quote 0
                            • RonpfSR
                              RonpfS
                              last edited by

                              Well on amd64 the default is 512MB.

                              /var in Ramdisk ? getting full ?

                              2.4.5-RELEASE-p1 (amd64)
                              Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                              Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                              1 Reply Last reply Reply Quote 0
                              • RonpfSR
                                RonpfS
                                last edited by

                                @oswoldy:

                                Ok, while the php /usr/local/www/pfblockerng/pfblockerng.php dc command works, the cron jobs are still crashing and giving errors, I am currently at 2GB limit and climbing.

                                If you run the dc command, it changes the GeoIP database, you have to run a force reload after the dc complete.

                                Also check diagnostic system activity to see if there isn't something unusual.
                                Look at the System log for hints on the failure mode.
                                An can you post the portion of the pfblockerng.log file where you have failure.

                                2.4.5-RELEASE-p1 (amd64)
                                Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                                Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                                1 Reply Last reply Reply Quote 0
                                • RonpfSR
                                  RonpfS
                                  last edited by

                                  @oswoldy:

                                  @RonpfS:

                                  @oswoldy:

                                  Ok, while the php /usr/local/www/pfblockerng/pfblockerng.php dc command works, the cron jobs are still crashing and giving errors, I am currently at 2GB limit and climbing.

                                  If you run the dc command, it changes the GeoIP database, you have to run a force reload after the dc complete.

                                  Also check diagnostic system activity to see if there isn't something unusual.
                                  Look at the System log for hints on the failure mode.
                                  An can you post the portion of the pfblockerng.log file where you have failure.

                                  Ok, I ran the dc command, followed by a force reload, no different. If I remove pfB_NAmerica_v4 then it works fine, crash report is:

                                  Crash report begins.  Anonymous machine information:

                                  amd64
                                  10.3-RELEASE-p5
                                  FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016    root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

                                  Crash report details:

                                  PHP Errors:
                                  [04-Aug-2016 10:29:24 Europe/London] PHP Fatal error:  Allowed memory size of 524288000 bytes exhausted (tried to allocate 20 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3875
                                  [04-Aug-2016 10:29:24 Europe/London] PHP Stack trace:
                                  [04-Aug-2016 10:29:24 Europe/London] PHP  1. {main}() /usr/local/www/pfblockerng/pfblockerng.php:0
                                  [04-Aug-2016 10:29:24 Europe/London] PHP  2. sync_package_pfblockerng() /usr/local/www/pfblockerng/pfblockerng.php:87
                                  [04-Aug-2016 10:29:24 Europe/London] PHP  3. file() /usr/local/pkg/pfblockerng/pfblockerng.inc:3875

                                  I have reduced the limit back to 500MB as increasing it wasnt making a difference, the dc command still completes but not a force reload or CRON.

                                  Once the dc command complete, the MaxMind database is created, so you do not need to re run it.

                                  What about the pfblockerng.log ? What are the symptoms ?
                                  Maybe raise it to 640M or 768M?

                                  2.4.5-RELEASE-p1 (amd64)
                                  Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                                  Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                                  1 Reply Last reply Reply Quote 0
                                  • W
                                    wiz561
                                    last edited by

                                    I started a thread up over in the pfblocker posting and just letting everybody know that I'm also experiencing very similar memory issues.

                                    Thismorning when I logged in, I also had a pfsense crash report with the following:

                                    					Crash report begins.  Anonymous machine information:
                                    
                                    amd64
                                    10.3-RELEASE-p5
                                    FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense
                                    
                                    Crash report details:
                                    
                                    PHP Errors:
                                    [04-Aug-2016 00:18:40 America/Chicago] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3868
                                    [04-Aug-2016 00:18:40 America/Chicago] PHP Stack trace:
                                    [04-Aug-2016 00:18:40 America/Chicago] PHP   1\. {main}() /etc/rc.start_packages:0
                                    [04-Aug-2016 00:18:40 America/Chicago] PHP   2\. sync_package() /etc/rc.start_packages:90
                                    [04-Aug-2016 00:18:40 America/Chicago] PHP   3\. eval() /etc/inc/pkg-utils.inc:631
                                    [04-Aug-2016 00:18:40 America/Chicago] PHP   4\. sync_package_pfblockerng() /etc/inc/pkg-utils.inc(631) : eval()'d code:3
                                    [04-Aug-2016 00:18:40 America/Chicago] PHP   5\. array_merge() /usr/local/pkg/pfblockerng/pfblockerng.inc:3868
                                    
                                    Filename: /var/crash/minfree
                                    2048
                                    
                                    1 Reply Last reply Reply Quote 0
                                    • L
                                      lucasrca
                                      last edited by

                                      How I solved my problem:

                                      My pfSense config:

                                      • Version: 2.3.2 (amd64), running on VMWare 6
                                        – Snort
                                        -- pfBlockerNG
                                        -- OpenVPN
                                        -- Open-vm-tools
                                        -- DHCP Relay
                                        -- Quagga OSPFd with another 2 pfSense.
                                      • ~3000 users simultaneously
                                      • 2 x 100 Mbit uplinks
                                      • 16 GB RAM
                                      • 80 GB SAS
                                      • CPU Type: Intel(R) Xeon(R) CPU E5-4620 v2 @ 2.60GHz
                                      • 16 CPUs: 8 package(s) x 2 core(s)
                                      • 8 Interfaces, including WAN
                                      • routing, filtering and relaying dhcp to 16 branches over MPLS, WiMax and fiber
                                      • Load balance and failover
                                      • QoS with Traffic shaper
                                      1. Updated Firewall Maximum Table Entries: 4000000 -> 8000000
                                        1.1) Reboot
                                      2. Edited /usr/local/pkg/pfblockerng/pfblockerng.inc and set memory limit to 500M
                                      3. Executed php /usr/local/www/pfblockerng/pfblockerng.php dc
                                      4. It's alive.

                                      Thanks to all involved.

                                      This a UNIX country. On a quiet night, you can hear Windows rebooting.

                                      1 Reply Last reply Reply Quote 0
                                      • M
                                        Mithrondil
                                        last edited by

                                        POST-INSTALL script failed
                                        Message from GeoIP-1.6.9:
                                        GeoIP does not ship with the actual data files. You must download
                                        them yourself! Please run:

                                        /usr/local/bin/geoipupdate.sh

                                        1 Reply Last reply Reply Quote 0
                                        • RonpfSR
                                          RonpfS
                                          last edited by

                                          @wiz561:

                                          I started a thread up over in the pfblocker posting and just letting everybody know that I'm also experiencing very similar memory issues.

                                          Thismorning when I logged in, I also had a pfsense crash report with the following:

                                          					Crash report begins.  Anonymous machine information:
                                          
                                          amd64
                                          10.3-RELEASE-p5
                                          FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense
                                          
                                          Crash report details:
                                          
                                          PHP Errors:
                                          [04-Aug-2016 00:18:40 America/Chicago] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3868
                                          [04-Aug-2016 00:18:40 America/Chicago] PHP Stack trace:
                                          [04-Aug-2016 00:18:40 America/Chicago] PHP   1\. {main}() /etc/rc.start_packages:0
                                          [04-Aug-2016 00:18:40 America/Chicago] PHP   2\. sync_package() /etc/rc.start_packages:90
                                          [04-Aug-2016 00:18:40 America/Chicago] PHP   3\. eval() /etc/inc/pkg-utils.inc:631
                                          [04-Aug-2016 00:18:40 America/Chicago] PHP   4\. sync_package_pfblockerng() /etc/inc/pkg-utils.inc(631) : eval()'d code:3
                                          [04-Aug-2016 00:18:40 America/Chicago] PHP   5\. array_merge() /usr/local/pkg/pfblockerng/pfblockerng.inc:3868
                                          
                                          Filename: /var/crash/minfree
                                          2048
                                          

                                          The php memory should be 512M by default, so setting it in the inc file to 400M might not help.
                                          Did it fail without any fix to the inc file? Can you try setting the limit to 640M or 768M?
                                          What do you have for Firewall Maximum Table Entries?

                                          2.4.5-RELEASE-p1 (amd64)
                                          Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                                          Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                                          1 Reply Last reply Reply Quote 0
                                          • RonpfSR
                                            RonpfS
                                            last edited by

                                            @Mithrondil:

                                            POST-INSTALL script failed
                                            Message from GeoIP-1.6.9:
                                            GeoIP does not ship with the actual data files. You must download
                                            them yourself! Please run:

                                            /usr/local/bin/geoipupdate.sh

                                            You will need to run  php /usr/local/www/pfblockerng/pfblockerng.php dc from the shell.
                                            Once it succeeds, you should be able to install, it may need a reboot.

                                            2.4.5-RELEASE-p1 (amd64)
                                            Intel Core2 Quad CPU Q8400 @ 2.66GHz 8GB
                                            Backup 0.5_5, Bandwidthd 0.7.4_4, Cron 0.3.7_5, pfBlockerNG-devel 3.0.0_16, Status_Traffic_Totals 2.3.1_1, System_Patches 1.2_5

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.