Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    PFBlockerNG 2.1.1_2 Memory Errors

    pfBlockerNG
    17
    61
    20254
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • W
      wiz561 last edited by

      @RonpfS:

      @wiz561:

      I started a thread up over in the pfblocker posting and just letting everybody know that I'm also experiencing very similar memory issues.

      Thismorning when I logged in, I also had a pfsense crash report with the following:

      					Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p5
FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[04-Aug-2016 00:18:40 America/Chicago] PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3868
[04-Aug-2016 00:18:40 America/Chicago] PHP Stack trace:
[04-Aug-2016 00:18:40 America/Chicago] PHP   1\. {main}() /etc/rc.start_packages:0
[04-Aug-2016 00:18:40 America/Chicago] PHP   2\. sync_package() /etc/rc.start_packages:90
[04-Aug-2016 00:18:40 America/Chicago] PHP   3\. eval() /etc/inc/pkg-utils.inc:631
[04-Aug-2016 00:18:40 America/Chicago] PHP   4\. sync_package_pfblockerng() /etc/inc/pkg-utils.inc(631) : eval()'d code:3
[04-Aug-2016 00:18:40 America/Chicago] PHP   5\. array_merge() /usr/local/pkg/pfblockerng/pfblockerng.inc:3868

Filename: /var/crash/minfree
2048

      The php memory should be 512M by default, so setting it in the inc file to 400M might not help.
      Did it fail without any fix to the inc file? Can you try setting the limit to 640M or 768M?
      What do you have for Firewall Maximum Table Entries?

      I'm going to have to do a bit of work this weekend on this and see more.  I can try it and see what happens, but I'm getting concerned about reaching my 2gb memory limit.  I can add more memory, but that requires me to go to the store and I'm kinda lazy…..and cheap.  :)

      Interesting that I too have a similar setup to the one user above.  I'm running it on vmware esxi with Snort (disabled) and OpenVPN.  Granted, I don't have nearly the hardware or setup, but it's interesting that we're both running vmware.

      1 Reply Last reply Reply Quote 0
      • W
        wiz561 last edited by

        OK…  Now maybe I'll try what others suggested...  :)

        I wiped and reinstalled pfsense tonight and pfblockng is still coming back with that crash and memory errors.  I know others said to adjust the memory but I thought I would give this a try.  Unfortunately, it failed.

        I also had issues with php-fpm having high utilization....so I'm hoping that the wipe/reinstall fixed the issue with that.  Time will tell.

        1 Reply Last reply Reply Quote 0
        • RonpfS
          RonpfS last edited by

          What about posting pfblockerNG, system log, crash report, screen shot of system activity, etc, so we can see what is happening on your setup?

          The crash report you posted earlier tells me you have under 400MB defined.

          PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted

          Did you raise the Firewall Maximum Table Entries ?

          1 Reply Last reply Reply Quote 0
          • Perforado
            Perforado Rebel Alliance last edited by

            Temporary Fix for

            php /usr/local/www/pfblockerng/pfblockerng.php update

            Failing with memory exhaustion:

            edit /usr/local/pkg/pfblockerng/pfblockerng.inc as discussed above:
            …
            pfb_global();
            ini_set('memory_limit', '640M');
            ...

            cp /etc/rc.php_ini_setup /etc/rc.php_ini_setup.BACKUP
            cp /usr/local/etc/php.ini /usr/local/etc/php.ini.BACKUP
            perl -pi -e 's/536870912/671088640/g' /etc/rc.php_ini_setup /usr/local/etc/php.ini

            512 * 1024 * 1024 -> 536870912
            640 * 1024 * 1024 -> 671088640 works for me. maybe your setup needs more :)

            1 Reply Last reply Reply Quote 0
            • RonpfS
              RonpfS last edited by

              @Perforado:

              Temporary Fix for

              php /usr/local/www/pfblockerng/pfblockerng.php update

              Failing with memory exhaustion:

              cp /etc/rc.php_ini_setup /etc/rc.php_ini_setup.BACKUP
              cp /usr/local/etc/php.ini /usr/local/etc/php.ini.BACKUP
              perl -pi -e 's/536870912/671088640/g' /etc/rc.php_ini_setup /usr/local/etc/php.ini

              512 * 1024 * 1024 -> 536870912
              640 * 1024 * 1024 -> 671088640 works for me. maybe your setup needs more :)

              There is probably a setting (memory_limit?) we can configure in the System/Advanced/System Tunables that will do that for you. But it may require a reboot.

              The fix for the inc file is specific to pfblocker and shouldn't be needed once BBcan177 change the code.

              1 Reply Last reply Reply Quote 0
              • Perforado
                Perforado Rebel Alliance last edited by

                /var/db/aliastables:
                -rw-r–r--  1 root  wheel    351450 Aug  5 10:56 pfB_Top_v4.txt
                -rw-r--r--  1 root  wheel  30690970 Aug  5 11:00 pfB_Top_v6.txt

                cat pfB_Top_v6.txt | wc -l
                1421351

                1.4mio entries? That can't be right?

                cat pfB_Top_v4.txt | wc -l
                  22410

                Could this be the the root cause of all this?

                1 Reply Last reply Reply Quote 0
                • RonpfS
                  RonpfS last edited by

                  @Perforado:

                  /var/db/aliastables:
                  -rw-r–r--  1 root  wheel    351450 Aug  5 10:56 pfB_Top_v4.txt
                  -rw-r--r--  1 root  wheel  30690970 Aug  5 11:00 pfB_Top_v6.txt

                  cat pfB_Top_v6.txt | wc -l
                  1421351

                  1.4mio entries? That can't be right?

                  cat pfB_Top_v4.txt | wc -l
                    22410

                  Could this be the the root cause of all this?

                  What was the size of pfB_Top_v6.txt before the MaxMind db change ?  I do not use these table, so I can't compare.

                  On option BBcan177 mentioned was that he might need to aggregate the table  to shrink them.

                  1 Reply Last reply Reply Quote 0
                  • Perforado
                    Perforado Rebel Alliance last edited by

                    pfB_Top_v6 was about 13000-ish before as far as i recall.

                    1 Reply Last reply Reply Quote 0
                    • W
                      wiz561 last edited by

                      @RonpfS:

                      What about posting pfblockerNG, system log, crash report, screen shot of system activity, etc, so we can see what is happening on your setup?

                      The crash report you posted earlier tells me you have under 400MB defined.

                      PHP Fatal error:  Allowed memory size of 402653184 bytes exhausted

                      Did you raise the Firewall Maximum Table Entries ?

                      I've been busy the past couple of days and blowing it away and restoring the config was pretty simple.  I'm going to try to work on this a bit more this week and take the suggestions of what others have posted to see if it fixes it.  I am going to guess that changing the memory settings around will help, but I also need to buy more memory for my system.

                      1 Reply Last reply Reply Quote 0
                      • R
                        Rickinfl last edited by

                        Hi,

                        I've been reading this forum and trying to figure out if there is a fix for this or not. I really didn't see anyone say "This is the fix" with instructions.

                        Can someone point me in the right direction?
                        Has anyone contacted the package creator?
                        Why hasn't anyone pulled this package from being install if there is issues with it?

                        They should pull this package if its not working. It pretty much killed my pfsense box and I had to remove it.

                        Sorry for being so direct. But I just had to shut down my firewall ports to my websites and I'm trying to get this fixed as soon as possible so I can bring them back online.

                        Thanks,
                        Rick

                        1 Reply Last reply Reply Quote 0
                        • D
                          duanes last edited by

                          Me Too…..
                          (NOTE - malloc failure still shows 512mb of ram.  My mem limit seems to be ignored)

                          I'm using
                          pfb_global();
                          ini_set('memory_limit', '4096M');

                          Still fails when updating with....

                          amd64
                          10.3-RELEASE-p5
                          FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016    root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

                          Crash report details:

                          PHP Errors:
                          [08-Aug-2016 07:47:51 America/Chicago] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3876
                          [08-Aug-2016 07:47:51 America/Chicago] PHP Stack trace:
                          [08-Aug-2016 07:47:51 America/Chicago] PHP  1. {main}() /usr/local/www/pfblockerng/pfblockerng.php:0
                          [08-Aug-2016 07:47:51 America/Chicago] PHP  2. sync_package_pfblockerng() /usr/local/www/pfblockerng/pfblockerng.php:87
                          [08-Aug-2016 07:47:51 America/Chicago] PHP  3. file() /usr/local/pkg/pfblockerng/pfblockerng.inc:3876

                          1 Reply Last reply Reply Quote 0
                          • Perforado
                            Perforado Rebel Alliance last edited by

                            All the php-scripts spawned by the gui are constrained by the memory-limit set by suhosin. Which is 512MB.

                            That's what my perl-one-liner was for. I even increased the limit to 4GB for testing today …

                            1 Reply Last reply Reply Quote 0
                            • P
                              paftdunk last edited by

                              @lucasrca:

                              How I solved my problem:…

                              1. Updated Firewall Maximum Table Entries: 4000000 -> 8000000

                              This ended up being the missing link for me. My default was 2M. When Perforado mentioned the count in  /var/db/aliastables/ I checked mine: 4.4M in those lists alone. I bumped my max table entries in System / Advanced / Firewall & NAT from 2M to 10M and pfblockerng started working again.

                              1 Reply Last reply Reply Quote 0
                              • RonpfS
                                RonpfS last edited by

                                @Perforado:

                                All the php-scripts spawned by the gui are constrained by the memory-limit set by suhosin. Which is 512MB.

                                That's what my perl-one-liner was for. I even increased the limit to 4GB for testing today …

                                For those wandering about suhosin, it is defined in /usr/local/etc/php.ini

                                ; File generated from /etc/rc.php_ini_setup
output_buffering = "0"
expose_php = Off
implicit_flush = true
magic_quotes_gpc = Off
max_execution_time = 900
request_terminate_timeout = 900
max_input_time = 1800
max_input_vars = 5000
register_argc_argv = On
register_long_arrays = Off
variables_order = "GPCS"
file_uploads = On
upload_tmp_dir = /tmp
upload_max_filesize = 200M
post_max_size = 200M
html_errors = Off
zlib.output_compression = Off
zlib.output_compression_level = 1
include_path = ".:/etc/inc:/usr/local/www:/usr/local/captiveportal:/usr/local/pkg:/usr/local/www/classes:/usr/local/www/classes/Form"
display_startup_errors=on
display_errors=on
log_errors=on
error_log=/tmp/PHP_errors.log
extension_dir=/usr/local/lib/php/20131226/
date.timezone="America/New_York"
session.hash_bits_per_character = 5
session.hash_function = 1

; Extensions

; opcache Settings
opcache.enabled="1"
opcache.enable_cli="0"
opcache.memory_consumption="50"

[suhosin]
suhosin.get.max_array_index_length = 256
suhosin.get.max_vars = 5000
suhosin.get.max_value_length = 500000
suhosin.post.max_array_index_length = 256
suhosin.post.max_vars = 5000
suhosin.post.max_value_length = 500000
suhosin.request.max_array_index_length = 256
suhosin.request.max_vars = 5000
suhosin.request.max_value_length = 500000
suhosin.memory_limit = 536870912

                                Setting 'memory_limit' in the inc file is maxed by the suhosin.memory_limit.

                                The memory_limit is defined in /etc/inc/config.inc

                                // Set memory limit to 512M on amd64.
if ($ARCH == "amd64") {
	ini_set("memory_limit", "512M");
} else {
	ini_set("memory_limit", "128M");
}

                                So for those using many IPV6 GeoIP table on amd64, they probably have to change /usr/local/etc/php.ini, /etc/rc.php_ini_setup, pfblockerng.inc as well as the Firewall Maximum Table Entries

                                The php memory issues are not specific to pfBlockerNG, backup crashes when backup size is too big, the Diagnostics/Tables will crash when viewing huge table, etc.

                                1 Reply Last reply Reply Quote 0
                                • R
                                  richcj10gmail.com last edited by

                                  I am having similar issues + NAT / routing was not working at all.

                                  error:

                                  
					Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p5
FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[09-Aug-2016 11:27:29 America/New_York] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3875
[09-Aug-2016 11:27:29 America/New_York] PHP Stack trace:
[09-Aug-2016 11:27:29 America/New_York] PHP   1\. {main}() /etc/rc.start_packages:0
[09-Aug-2016 11:27:29 America/New_York] PHP   2\. sync_package() /etc/rc.start_packages:90
[09-Aug-2016 11:27:29 America/New_York] PHP   3\. eval() /etc/inc/pkg-utils.inc:631
[09-Aug-2016 11:27:29 America/New_York] PHP   4\. sync_package_pfblockerng() /etc/inc/pkg-utils.inc(631) : eval()'d code:3
[09-Aug-2016 11:27:29 America/New_York] PHP   5\. file() /usr/local/pkg/pfblockerng/pfblockerng.inc:3875

                                  I changed the fire wall rule # to 8000000 and added  ini_set("memory_limit", "768M");
                                  I still see the error above. But I at lest have routing back.

                                  1 Reply Last reply Reply Quote 0
                                  • RonpfS
                                    RonpfS last edited by

                                    @richcj10@gmail.com:

                                    I am having similar issues + NAT / routing was not working at all.

                                    error:

                                    
					Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p5
FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[09-Aug-2016 11:27:29 America/New_York] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3875
[09-Aug-2016 11:27:29 America/New_York] PHP Stack trace:
[09-Aug-2016 11:27:29 America/New_York] PHP   1\. {main}() /etc/rc.start_packages:0
[09-Aug-2016 11:27:29 America/New_York] PHP   2\. sync_package() /etc/rc.start_packages:90
[09-Aug-2016 11:27:29 America/New_York] PHP   3\. eval() /etc/inc/pkg-utils.inc:631
[09-Aug-2016 11:27:29 America/New_York] PHP   4\. sync_package_pfblockerng() /etc/inc/pkg-utils.inc(631) : eval()'d code:3
[09-Aug-2016 11:27:29 America/New_York] PHP   5\. file() /usr/local/pkg/pfblockerng/pfblockerng.inc:3875

                                    I changed the fire wall rule # to 8000000 and added  ini_set("memory_limit", "768M");
                                    I still see the error above. But I at lest have routing back.

                                    Did you fix  /usr/local/etc/php.ini, /etc/rc.php_ini_setup ?

                                    1 Reply Last reply Reply Quote 0
                                    • M
                                      marian78 last edited by

                                      same problem:

                                      Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 8388608 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3867 Call Stack: 0.0001 245984 1\. {main}() /usr/local/www/pkg_edit.php:0 0.4857 2721392 2\. eval('global $pfb; $pfb['save'] = TRUE; sync_package_pfblockerng();') /usr/local/www/pkg_edit.php:253 0.4857 2722328 3\. sync_package_pfblockerng() /usr/local/www/pkg_edit.php(253) : eval()'d code:3 3.9159 5449944 4\. file() /usr/local/pkg/pfblockerng/pfblockerng.inc:3867 PHP ERROR: Type: 1, File: /usr/local/pkg/pfblockerng/pfblockerng.inc, Line: 3867, Message: Allowed memory size of 134217728 bytes exhausted (tried to allocate 8388608 bytes)

                                      Crash report begins.  Anonymous machine information:

                                      amd64
                                      10.3-RELEASE-p5
                                      FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016    root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

                                      Crash report details:

                                      PHP Errors:
                                      [10-Aug-2016 12:50:00 Europe/Bratislava] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 8388608 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3867
                                      [10-Aug-2016 12:50:00 Europe/Bratislava] PHP Stack trace:
                                      [10-Aug-2016 12:50:00 Europe/Bratislava] PHP  1. {main}() /usr/local/www/pkg_edit.php:0
                                      [10-Aug-2016 12:50:00 Europe/Bratislava] PHP  2. eval() /usr/local/www/pkg_edit.php:253
                                      [10-Aug-2016 12:50:00 Europe/Bratislava] PHP  3. sync_package_pfblockerng() /usr/local/www/pkg_edit.php(253) : eval()'d code:3
                                      [10-Aug-2016 12:50:00 Europe/Bratislava] PHP  4. file() /usr/local/pkg/pfblockerng/pfblockerng.inc:3867
                                      [10-Aug-2016 12:54:33 Europe/Bratislava] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 8388608 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3867
                                      [10-Aug-2016 12:54:33 Europe/Bratislava] PHP Stack trace:
                                      [10-Aug-2016 12:54:33 Europe/Bratislava] PHP  1. {main}() /usr/local/www/pkg_edit.php:0
                                      [10-Aug-2016 12:54:33 Europe/Bratislava] PHP  2. eval() /usr/local/www/pkg_edit.php:253
                                      [10-Aug-2016 12:54:33 Europe/Bratislava] PHP  3. sync_package_pfblockerng() /usr/local/www/pkg_edit.php(253) : eval()'d code:3
                                      [10-Aug-2016 12:54:33 Europe/Bratislava] PHP  4. file() /usr/local/pkg/pfblockerng/pfblockerng.inc:3867

                                      1 Reply Last reply Reply Quote 0
                                      • R
                                        richcj10gmail.com last edited by

                                        @RonpfS:

                                        @richcj10@gmail.com:

                                        I am having similar issues + NAT / routing was not working at all.

                                        error:

                                        
					Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p5
FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[09-Aug-2016 11:27:29 America/New_York] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3875
[09-Aug-2016 11:27:29 America/New_York] PHP Stack trace:
[09-Aug-2016 11:27:29 America/New_York] PHP   1\. {main}() /etc/rc.start_packages:0
[09-Aug-2016 11:27:29 America/New_York] PHP   2\. sync_package() /etc/rc.start_packages:90
[09-Aug-2016 11:27:29 America/New_York] PHP   3\. eval() /etc/inc/pkg-utils.inc:631
[09-Aug-2016 11:27:29 America/New_York] PHP   4\. sync_package_pfblockerng() /etc/inc/pkg-utils.inc(631) : eval()'d code:3
[09-Aug-2016 11:27:29 America/New_York] PHP   5\. file() /usr/local/pkg/pfblockerng/pfblockerng.inc:3875

                                        I changed the fire wall rule # to 8000000 and added  ini_set("memory_limit", "768M");
                                        I still see the error above. But I at lest have routing back.

                                        Did you fix  /usr/local/etc/php.ini, /etc/rc.php_ini_setup ?

                                        What fix?

                                        1 Reply Last reply Reply Quote 0
                                        • RonpfS
                                          RonpfS last edited by

                                          @richcj10@gmail.com:

                                          @RonpfS:

                                          @richcj10@gmail.com:

                                          I am having similar issues + NAT / routing was not working at all.

                                          error:

                                          
					Crash report begins.  Anonymous machine information:

amd64
10.3-RELEASE-p5
FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016     root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[09-Aug-2016 11:27:29 America/New_York] PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 72 bytes) in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 3875
[09-Aug-2016 11:27:29 America/New_York] PHP Stack trace:
[09-Aug-2016 11:27:29 America/New_York] PHP   1\. {main}() /etc/rc.start_packages:0
[09-Aug-2016 11:27:29 America/New_York] PHP   2\. sync_package() /etc/rc.start_packages:90
[09-Aug-2016 11:27:29 America/New_York] PHP   3\. eval() /etc/inc/pkg-utils.inc:631
[09-Aug-2016 11:27:29 America/New_York] PHP   4\. sync_package_pfblockerng() /etc/inc/pkg-utils.inc(631) : eval()'d code:3
[09-Aug-2016 11:27:29 America/New_York] PHP   5\. file() /usr/local/pkg/pfblockerng/pfblockerng.inc:3875

                                          I changed the fire wall rule # to 8000000 and added  ini_set("memory_limit", "768M");
                                          I still see the error above. But I at lest have routing back.

                                          Did you fix  /usr/local/etc/php.ini, /etc/rc.php_ini_setup ?

                                          What fix?

                                          @Perforado:

                                          Temporary Fix for

                                          php /usr/local/www/pfblockerng/pfblockerng.php update

                                          Failing with memory exhaustion:

                                          edit /usr/local/pkg/pfblockerng/pfblockerng.inc as discussed above:
                                          …
                                          pfb_global();
                                          ini_set('memory_limit', '640M');
                                          ...

                                          cp /etc/rc.php_ini_setup /etc/rc.php_ini_setup.BACKUP
                                          cp /usr/local/etc/php.ini /usr/local/etc/php.ini.BACKUP
                                          perl -pi -e 's/536870912/671088640/g' /etc/rc.php_ini_setup /usr/local/etc/php.ini

                                          512 * 1024 * 1024 -> 536870912
                                          640 * 1024 * 1024 -> 671088640 works for me. maybe your setup needs more :)

                                          You can edit the files or do the perl command

                                          1 Reply Last reply Reply Quote 0
                                          • C
                                            chrlan last edited by

                                            Hello!

                                            I also had this problem. Editing the files as described earlier (including The memory_limit defined in /etc/inc/config.inc where I increased the amd64 limit to 640M)  was part of the solution for me.
                                            The last thing for me was to increase the Firewall Maximum Table Entries: 2000000 -> 10000000. When using pfBlockerNG the total of firewall rules are about 4500000 rows now. Reloading those rules requires about 9000000 table entries since that the new rules are loaded before the old ones are deleted ending up with temporary 9000000 million rows in the table.
                                            If you have lesser than 9000000 you will get an out-of-memory error message plus that some rules are not loaded correctly.

                                            1 Reply Last reply Reply Quote 0
                                            • RonpfS
                                              RonpfS last edited by

                                              https://forum.pfsense.org/index.php?topic=102470.msg645219#msg645219

                                              1 Reply Last reply Reply Quote 0
                                              • J
                                                JorgeOliveira last edited by

                                                I've submitted a PR to pfSense's GitHub repo:
                                                https://github.com/pfsense/pfsense/pull/3101

                                                After that, the following changes suggested by @Perforado on the package could be implemented and should work.
                                                @Perforado:

                                                Temporary Fix for

                                                php /usr/local/www/pfblockerng/pfblockerng.php update

                                                Failing with memory exhaustion:

                                                edit /usr/local/pkg/pfblockerng/pfblockerng.inc as discussed above:
                                                …
                                                pfb_global();
                                                ini_set('memory_limit', '640M');
                                                ...

                                                1 Reply Last reply Reply Quote 0
                                                • RonpfS
                                                  RonpfS last edited by

                                                  I don't think you need that PR, if you are following the pfBlockerNG v2.0 w/DNSBL thread, you should have known that we are working on a fix that shouldn't require modifying memory_limit or php.ini.

                                                  1 Reply Last reply Reply Quote 0
                                                  • P
                                                    paftdunk last edited by

                                                    @RonpfS:

                                                    I don't think you need that PR, if you are following the pfBlockerNG v2.0 w/DNSBL thread, you should have known that we are working on a fix that shouldn't require modifying memory_limit or php.ini.

                                                    A 55 page, nine month old thread introducing the feature seems like a weird place to have active commentary on fixing the bug reported in this one.

                                                    1 Reply Last reply Reply Quote 0
                                                    • RonpfS
                                                      RonpfS last edited by

                                                      This thread is about fixing memory issues introduced by MaxMind database changes.

                                                      The post about "Blocking the world" is about people configuration that trigger the bug.
                                                      It is normally useless to Block the world and the comment is in the right place in the pfBlockerNG w/DNSBL thread.

                                                      The thread has 55 pages, because pfBlockerNG is not trivial to configure.
                                                      Since when the size of a thread make it irrelevant to read it?

                                                      1 Reply Last reply Reply Quote 0
                                                      • J
                                                        JohnH last edited by

                                                        Following the following instructions I have trashed my pfSense install and receive the following stack trace on boot:

                                                        Fatal Error: Allowed memory size of 262144 bytes exhausted (tried to allocate 49152 bytes) in /etc/inc/interfaces.inc on line 568

                                                        Call Stack:
                                                              0.0002  219464    1. {main}() /etc/rc.conf_mount_ro:0
                                                              0.0012  259568    2: require_once('etc/inc/config.inc') /etc/rc.conf_mount_ro:55
                                                              0.0043  366712    3. require_once('/etc/inc/notices.inc') /etc/inc/config.inc:56
                                                              0.0050  396632    4. require_once('/etc/inc/functions.inc') /etc/inc/notices.inc:56

                                                        No options work

                                                        I followed the post below, and then ran the geoip download which completed successfully. Then while running the update from pfBlocker it froze, and the network went down. Now the DHCP does not assign addresses to the network and I cannot access the box except going directly to it with KBB and monitor.

                                                        Reverting changes and rebooting does not resolve. Running any option from the terminal returns the above error. If only the devs hadn't removed the script for manual backups and recovery….

                                                        @RonpfS:

                                                        @Perforado:

                                                        All the php-scripts spawned by the gui are constrained by the memory-limit set by suhosin. Which is 512MB.

                                                        That's what my perl-one-liner was for. I even increased the limit to 4GB for testing today …

                                                        For those wandering about suhosin, it is defined in /usr/local/etc/php.ini

                                                        ; File generated from /etc/rc.php_ini_setup
output_buffering = "0"
expose_php = Off
implicit_flush = true
magic_quotes_gpc = Off
max_execution_time = 900
request_terminate_timeout = 900
max_input_time = 1800
max_input_vars = 5000
register_argc_argv = On
register_long_arrays = Off
variables_order = "GPCS"
file_uploads = On
upload_tmp_dir = /tmp
upload_max_filesize = 200M
post_max_size = 200M
html_errors = Off
zlib.output_compression = Off
zlib.output_compression_level = 1
include_path = ".:/etc/inc:/usr/local/www:/usr/local/captiveportal:/usr/local/pkg:/usr/local/www/classes:/usr/local/www/classes/Form"
display_startup_errors=on
display_errors=on
log_errors=on
error_log=/tmp/PHP_errors.log
extension_dir=/usr/local/lib/php/20131226/
date.timezone="America/New_York"
session.hash_bits_per_character = 5
session.hash_function = 1

; Extensions

; opcache Settings
opcache.enabled="1"
opcache.enable_cli="0"
opcache.memory_consumption="50"

[suhosin]
suhosin.get.max_array_index_length = 256
suhosin.get.max_vars = 5000
suhosin.get.max_value_length = 500000
suhosin.post.max_array_index_length = 256
suhosin.post.max_vars = 5000
suhosin.post.max_value_length = 500000
suhosin.request.max_array_index_length = 256
suhosin.request.max_vars = 5000
suhosin.request.max_value_length = 500000
suhosin.memory_limit = 536870912

                                                        Setting 'memory_limit' in the inc file is maxed by the suhosin.memory_limit.

                                                        The memory_limit is defined in /etc/inc/config.inc

                                                        // Set memory limit to 512M on amd64.
if ($ARCH == "amd64") {
	ini_set("memory_limit", "512M");
} else {
	ini_set("memory_limit", "128M");
}

                                                        So for those using many IPV6 GeoIP table on amd64, they probably have to change /usr/local/etc/php.ini, /etc/rc.php_ini_setup, pfblockerng.inc as well as the Firewall Maximum Table Entries

                                                        The php memory issues are not specific to pfBlockerNG, backup crashes when backup size is too big, the Diagnostics/Tables will crash when viewing huge table, etc.

                                                        This is a AsRock Q1900M w/quad core celeron J1900, 8GB RAM, 1TB HDD, 1 intel dual GbE, 1 intel single GbE cards.

                                                        1 Reply Last reply Reply Quote 0
                                                        • RonpfS
                                                          RonpfS last edited by

                                                          So what modification did you do.

                                                          On i386, only the pfblockerng.inc fix is normally required.

                                                          The config.inc was just stated for referenced. If you modified it, maybe you forgot to put the "M" in 256M? (262144 bytes exhausted)

                                                          The /etc/rc.php_ini_setup &  /usr/local/etc/php.ini fix is for amd64 that need more than 512MB.

                                                          1 Reply Last reply Reply Quote 0
                                                          • J
                                                            JohnH last edited by

                                                            Thanks Ron…a little M goes a long way to making the system work. 2048 instead of 2048M.

                                                            1 Reply Last reply Reply Quote 0
                                                            • RonpfS
                                                              RonpfS last edited by

                                                              @JohnH:

                                                              Thanks Ron…a little M goes a long way to making the system work. 2048 instead of 2048M.

                                                              And where did you made the modifications ? 2048M in config.inc might break something else somewhere.

                                                              1 Reply Last reply Reply Quote 0
                                                              • J
                                                                JohnH last edited by

                                                                // Set memory limit to 512M on amd64.
                                                                if ($ARCH == "amd64") {
                                                                ini_set("memory_limit", "512M");
                                                                } else {
                                                                ini_set("memory_limit", "128M");
                                                                }

                                                                changed to

                                                                // Set memory limit to 512M on amd64.
                                                                if ($ARCH == "amd64") {
                                                                ini_set("memory_limit", "2048M");
                                                                } else {
                                                                ini_set("memory_limit", "128M");
                                                                }

                                                                reverting to default shouldn't hurt as pfBlocker is uninstalled until a real fix is in place.

                                                                1 Reply Last reply Reply Quote 0
                                                                • RonpfS
                                                                  RonpfS last edited by

                                                                  I would put it back to 512MB.
                                                                  Modify the pfblockerng.inc instead, this will only affect the pfblocker pkg.
                                                                  However, the php.ini limit is 512M, so if you need more than 512M, you will need both fixes, the one for php and the on for pfblockerng.inc.

                                                                  1 Reply Last reply Reply Quote 0
                                                                  • J
                                                                    JohnH last edited by

                                                                    Will do, thanks.

                                                                    1 Reply Last reply Reply Quote 0
                                                                    • P
                                                                      ProxyMoron last edited by

                                                                      Hi All,
                                                                        I think i have a better solution to this although feel free to shoot it down if i've messed up.

                                                                      Basically, say you only want the UK to be permitted, but not the ROTW…

                                                                      If you deny everything then it takes a a shed load of memory and causes the problems above. However my solution is to only PERMIT what you want and then DENY everything that isnt permitted. You dont need the entire Maxmind database, only the IP's of what you want to permit.

                                                                      To do this, go into PfblockerNG, GeoIP, Top20 and select only UK (or your country /countries) and then in List action select Permit Inbound.

                                                                      This will create a floating rule in Firewall tab that is called pfB_Top_v4 that contains all the IP's of the countries you select to be permitted from that tab.

                                                                      Now create another rule that is EXACTLY the same as the above one in floating rules, except tick the "Invert Match" box and change the rule to a block or reject rule then add it before the auto generated pfB_Top_v4 rule and name it !pfB_Top_v4 or something similar.

                                                                      Now all IP's that aren't UK based (in my example) will be denied and whenever you update the pfB_Top_v4 alias, it will also update your deny rule automatically too.

                                                                      You may need to duplicate multiple rules if your permits appear in other Geo Locations like Africa or Antartica for example as this will create additional Floating rules.

                                                                      This also has the massive benefit of having an order of magnitude less rules in your Firewall table as you only checking a specific permitted countries IP range as opposed to the ROTW's IP ranges.

                                                                      1 Reply Last reply Reply Quote 0
                                                                      • RonpfS
                                                                        RonpfS last edited by

                                                                        To install or re-install pfBlockerNG 2.1.1_2

                                                                        https://forum.pfsense.org/index.php?topic=102470.msg647400#msg647400

                                                                        1 Reply Last reply Reply Quote 0
                                                                        • RonpfS
                                                                          RonpfS last edited by

                                                                          @JorgeOliveira:

                                                                          I've submitted a PR to pfSense's GitHub repo:
                                                                          https://github.com/pfsense/pfsense/pull/3101

                                                                          After that, the following changes suggested by @Perforado on the package could be implemented and should work.
                                                                          @Perforado:

                                                                          Temporary Fix for

                                                                          php /usr/local/www/pfblockerng/pfblockerng.php update. 
                                                                          Failing with memory exhaustion:

                                                                          edit /usr/local/pkg/pfblockerng/pfblockerng.inc as discussed above:
                                                                          …
                                                                          pfb_global();
                                                                          ini_set('memory_limit', '640M');
                                                                          ...

                                                                          After some testing, I finally consider that modifying the /etc/inc/config.inc is the better solution.

                                                                          However regarding the PR, 192M isn't enough for the MaxMind database creation, 256M was fine on my system, I didn't test 224M.

                                                                          After fixing /etc/inc/config.inc, the installation of pfBlockerNG was successful. :D

                                                                          1 Reply Last reply Reply Quote 0
                                                                          • J
                                                                            JohnH last edited by

                                                                            @ProxyMoron:

                                                                            Hi All,
                                                                              I think i have a better solution to this although feel free to shoot it down if i've messed up.

                                                                            Basically, say you only want the UK to be permitted, but not the ROTW…

                                                                            If you deny everything then it takes a a shed load of memory and causes the problems above. However my solution is to only PERMIT what you want and then DENY everything that isnt permitted. You dont need the entire Maxmind database, only the IP's of what you want to permit.

                                                                            To do this, go into PfblockerNG, GeoIP, Top20 and select only UK (or your country /countries) and then in List action select Permit Inbound.

                                                                            This will create a floating rule in Firewall tab that is called pfB_Top_v4 that contains all the IP's of the countries you select to be permitted from that tab.

                                                                            Now create another rule that is EXACTLY the same as the above one in floating rules, except tick the "Invert Match" box and change the rule to a block or reject rule then add it before the auto generated pfB_Top_v4 rule and name it !pfB_Top_v4 or something similar.

                                                                            Now all IP's that aren't UK based (in my example) will be denied and whenever you update the pfB_Top_v4 alias, it will also update your deny rule automatically too.

                                                                            You may need to duplicate multiple rules if your permits appear in other Geo Locations like Africa or Antartica for example as this will create additional Floating rules.

                                                                            This also has the massive benefit of having an order of magnitude less rules in your Firewall table as you only checking a specific permitted countries IP range as opposed to the ROTW's IP ranges.

                                                                            Absolutely correct, per the instructions in the GeoIP pag:

                                                                            It's also not recommended to block the 'world', instead consider rules to 'Permit' traffic from selected Countries only.
                                                                            Also consider protecting just the specific open WAN ports and it's just as important to protect the outbound LAN traffic.

                                                                            1 Reply Last reply Reply Quote 0
                                                                            • P
                                                                              ProxyMoron last edited by

                                                                              @JohnH:

                                                                              @ProxyMoron:

                                                                              Hi All,
                                                                                I think i have a better solution to this although feel free to shoot it down if i've messed up.

                                                                              Basically, say you only want the UK to be permitted, but not the ROTW…

                                                                              If you deny everything then it takes a a shed load of memory and causes the problems above. However my solution is to only PERMIT what you want and then DENY everything that isnt permitted. You dont need the entire Maxmind database, only the IP's of what you want to permit.

                                                                              To do this, go into PfblockerNG, GeoIP, Top20 and select only UK (or your country /countries) and then in List action select Permit Inbound.

                                                                              This will create a floating rule in Firewall tab that is called pfB_Top_v4 that contains all the IP's of the countries you select to be permitted from that tab.

                                                                              Now create another rule that is EXACTLY the same as the above one in floating rules, except tick the "Invert Match" box and change the rule to a block or reject rule then add it before the auto generated pfB_Top_v4 rule and name it !pfB_Top_v4 or something similar.

                                                                              Now all IP's that aren't UK based (in my example) will be denied and whenever you update the pfB_Top_v4 alias, it will also update your deny rule automatically too.

                                                                              You may need to duplicate multiple rules if your permits appear in other Geo Locations like Africa or Antartica for example as this will create additional Floating rules.

                                                                              This also has the massive benefit of having an order of magnitude less rules in your Firewall table as you only checking a specific permitted countries IP range as opposed to the ROTW's IP ranges.

                                                                              Absolutely correct, per the instructions in the GeoIP pag:

                                                                              It's also not recommended to block the 'world', instead consider rules to 'Permit' traffic from selected Countries only.
                                                                              Also consider protecting just the specific open WAN ports and it's just as important to protect the outbound LAN traffic.

                                                                              The problem is that those instructions mention nothing about needing to create a deny rule. Your essentially permitting a set of counties but blocking nothing unless you create that additional rule, I'd like to see that done automatically to be honest.

                                                                              1 Reply Last reply Reply Quote 0
                                                                              • RonpfS
                                                                                RonpfS last edited by

                                                                                To resolve the issue :

                                                                                https://forum.pfsense.org/index.php?topic=102470.msg647719#msg647719

                                                                                1 Reply Last reply Reply Quote 0
                                                                                • First post
                                                                                  Last post

                                                                                Products

                                                                                • Platform Overview
                                                                                • TNSR
                                                                                • pfSense
                                                                                • Appliances

                                                                                Services

                                                                                • Training
                                                                                • Professional Services

                                                                                Support

                                                                                • Subscription Plans
                                                                                • Contact Support
                                                                                • Product Lifecycle
                                                                                • Documentation

                                                                                News

                                                                                • Media Coverage
                                                                                • Press
                                                                                • Events

                                                                                Resources

                                                                                • Blog
                                                                                • FAQ
                                                                                • Find a Partner
                                                                                • Resource Library
                                                                                • Security Information

                                                                                Company

                                                                                • About Us
                                                                                • Careers
                                                                                • Partners
                                                                                • Contact Us
                                                                                • Legal
                                                                                Our Mission

                                                                                We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

                                                                                Subscribe to our Newsletter

                                                                                Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.

                                                                                © 2021 Rubicon Communications, LLC | Privacy Policy