Php out of memory errors on 2.1 development amd64 - bogonsv6 and pfblocker

iamzam

I installed pfblocker and was setting it up without enabling it via the "enable pfblocker" checkbox yet. Once i hit apply i saw the php error:

php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was '/tmp/rules.debug:109: cannot define table bogonsv6: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded'

I disabled bogons for both of my WAN/WANv6 interfaces, and was then able to setup pfblocker, but once i got to a list that was pretty large, i get this:

php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was '/tmp/rules.debug:36: cannot define table pfBlockerpfBlockerlevel1: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded'

This is for the http://www.iblocklist.com/list.php?list=bt_level1 list.
it contains over 230k ip ranges… The file itself is 7.9M, but php's memory limit is 128M i believe. I have 3G memory, most of which is free.

here is the file listing for pfblocker-aliases and the bogons

[2.1-DEVELOPMENT][admin@pfs.dv.loc]/usr/local/www(58): ls -l /var/db/aliastables/
total 8816
-rw-r–r-- 1 root wheel 31448 Mar 1 22:40 pfBlockerAfrica.txt
-rw-r--r-- 1 root wheel 242259 Mar 1 22:40 pfBlockerAsia.txt
-rw-r--r-- 1 root wheel 202056 Mar 1 22:40 pfBlockerEurope.txt
-rw-r--r-- 1 root wheel 4016 Mar 1 22:40 pfBlockerNorthAmerica.txt
-rw-r--r-- 1 root wheel 1101 Mar 1 22:40 pfBlockerOceania.txt
-rw-r--r-- 1 root wheel 36481 Mar 1 22:40 pfBlockerSouthAmerica.txt
-rw-r--r-- 1 root wheel 239475 Mar 1 22:40 pfBlockerTopSpammers.txt
-rw-r--r-- 1 root wheel 99512 Mar 1 22:40 pfBlockerpfBlockerbt_ads.txt
-rw-r--r-- 1 root wheel 102225 Mar 1 22:40 pfBlockerpfBlockerbt_spyware.txt
-rw-r--r-- 1 root wheel 7962741 Mar 1 22:40 pfBlockerpfBlockerlevel1.txt

[2.1-DEVELOPMENT][admin@pfs.dv.loc]/usr/local/www(59): ls -l /etc/bogons*
-rw-r–r-- 1 root wheel 132 Mar 1 22:05 /etc/bogons
-rw-r--r-- 1 root wheel 814946 Mar 1 22:05 /etc/bogonsv6

marcelloc

This is not related to 2.1. You need to increase firewall maximum table size on system -> advanced -> firewall/nat

Take a look on pfBlocker wiki to see the faq and how it works

http://doc.pfsense.org/index.php/Pfblocker

iamzam

I have increased that a few times already, right now i am at 100000000 (100 million) and i still get the same error.

Do you know if there is something i need to do after increasing this? reboot?

chpalmer

Mar 1 20:39:26 php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was '/tmp/rules.debug:147: cannot define table bogonsv6: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded'
Mar 1 20:39:28 php: : New alert found: There were error(s) loading the rules: /tmp/rules.debug:147: cannot define table bogonsv6: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [147]: table <bogonsv6>persist file "/etc/bogonsv6"
Mar 1 20:39:28 php: : There were error(s) loading the rules: /tmp/rules.debug:147: cannot define table bogonsv6: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [147]: table <bogonsv6>persist file "/etc/bogonsv6"
Mar 1 20:39:35 php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was '/tmp/rules.debug:147: cannot define table bogonsv6: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded'
Mar 1 20:39:38 php: : New alert found: There were error(s) loading the rules: /tmp/rules.debug:147: cannot define table bogonsv6: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [147]: table <bogonsv6>persist file "/etc/bogonsv6"
Mar 1 20:39:38 php: : There were error(s) loading the rules: /tmp/rules.debug:147: cannot define table bogonsv6: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [147]: table <bogonsv6>persist file "/etc/bogonsv6"

Related?</bogonsv6></bogonsv6></bogonsv6></bogonsv6>

marcelloc

Yes, the same error pfBlocker gets but now in bogonsv6 list. Maybe a huge list too.

I'm not testing 2.1 yet but you can try to monitor memory usage while applying rules to see how much RAM it loads before error.

iamzam

I looked at both of my php.ini files, ( i have a /usr/local/lib/php.ini and a /usr/local/etc/php.ini ) and there was no memory_limit key but there was a suhosin.memory_limit = 512435456

According to http://redmine.pfsense.org/issues/2214 suhosin was turned off temporarily in 2.1 so i'm not sure if the suhosin.memory_limit would apply.

In any event, a reboot fixed my problem, so apparently changing 'Firewall Maximum Table Entries' needs a reboot to take effect…

I created a phpinfo() script and loaded it into /usr/local/www and it shows 128M, but the 'suhosin.memory_limit = 512435456' equates to around 488M according to my calculator and http://www.php.net/manual/en/faq.using.php#faq.using.shorthandbytes so perhaps the default of 128M is in effect since /etc/rc.php_ini_setup doesn't set any 'memory_limit'.

Efonnes

None of the log messages you have posted have anything to do with PHP running out of memory. Those are only messages about the external commands that the PHP script executed.

iamzam

ok, thanks, that make sense.

So can you verify that changing firewall maximum table entries needs a reboot? Everywhere i find that references this change doesn't mention rebooting, and changing that variable definitely wasn't working until i rebooted…

jimp

It's a chicken and egg problem.

You have to load the ruleset to increase the table size, and you have to increase the table size to load the ruleset.

Delete or deactivate some tables, fix the table size, save/apply, then add them back.

zandr

I'm seeing the same issue as chpalmer. It's just 2.1, nothing to do with pfblocker, as I haven't installed that.

Turning off ipv6 everywhere doesn't help, I actually had to turn off bogon filtering everywhere to make the error go away. (not my first choice, but probably not a big deal)

databeestje

this is not as much a problem with 2.1 perse. It's pf that can't load the ruleset.

I think there is some contemplation how to go about this.

I've had no issues with pfctl running out of memory related to anything remotely IPv6 related.

zandr

I'm inclined to believe it's not actually memory issue, since I'm using 3% of the 4GB in this box. (amd64)

I think it's a bug of some sort, there was a similar issue back in October in this thread: http://forum.pfsense.org/index.php/topic,40953.msg211475.html#msg211475

EDIT: And /etc/bogonsv6 is actually empty.

databeestje

that thread is entirely unrelated to this.