PfBlocker

hubsd

I discovered a new problem today, pfsense takes more than 1 minute to finish boot (9 minutes to be accurate !)
I noticed that it's blocked most on the time on "Configuring firewall" step.

Without plugin, no problem so…

There is also that usual "Warning message" in webgui after reboot in webgui (then need to "acknowledge all")

Setup is Alix 2c3 + pfsense nanobsd + compact flash 2 GB

Widget and lists :

RAM/CPU/TABLE usage

Packages used

log side, I have this messages (x10)

php: : New alert found: There were error(s) loading the rules: no IP address found for grep: /tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data no IP address found for grep: /tmp/rules.debug:20: file "/var/db/aliastables/pfBlockerInternetTrash.txt" contains bad data no IP address found for grep: /tmp/rules.debug:22: file "/var/db/aliastables/pfBlockerBlizzard.txt" contains bad data no IP address found for grep: /tmp/rules.debug:24: file "/var/db/aliastables/pfBlockeripfilterX.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded The line in question reads [ /tmp/rules.debug]:

php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was 'no IP address found for grep: /tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data no IP address found for grep: /tmp/rules.debug:20: file "/var/db/aliastables/pfBlockerInternetTrash.txt" contains bad data no IP address found for grep: /tmp/rules.debug:22: file "/var/db/aliastables/pfBlockerBlizzard.txt" contains bad data no IP address found for grep: /tmp/rules.debug:24: file "/var/db/aliastables/pfBlockeripfilterX.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded'

ghm

@hubsd:

I discovered a new problem today, pfsense takes more than 1 minute to finish boot (9 minutes to be accurate !)

Setup is Alix 2c3 + pfsense nanobsd + compact flash 2 GB

Same here - massive issues on 2.0 embedded (Countryblock works apart from logging though). I see loads of:

Nov 4 14:14:29 	php: : There were error(s) loading the rules: no IP address found for grep: /tmp/rules.debug:24: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data no IP address found for grep: /tmp/rules.debug:26: file "/var/db/aliastables/pfBlockerNorthAmerica.txt" contains bad data no IP address found for grep: /tmp/rules.debug:28: file "/var/db/aliastables/pfBlockerOceania.txt" contains bad data no IP address found for grep: /tmp/rules.debug:30: file "/var/db/aliastables/pfBlockerSouthAmerica.txt" contains bad data no IP address found for grep: /tmp/rules.debug:32: file "/var/db/aliastables/pfBlockerTopSpammers.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [ /tmp/rules.debug]:
Nov 4 14:14:29 	php: : New alert found: There were error(s) loading the rules: no IP address found for grep: /tmp/rules.debug:24: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data no IP address found for grep: /tmp/rules.debug:26: file "/var/db/aliastables/pfBlockerNorthAmerica.txt" contains bad data no IP address found for grep: /tmp/rules.debug:28: file "/var/db/aliastables/pfBlockerOceania.txt" contains bad data no IP address found for grep: /tmp/rules.debug:30: file "/var/db/aliastables/pfBlockerSouthAmerica.txt" contains bad data no IP address found for grep: /tmp/rules.debug:32: file "/var/db/aliastables/pfBlockerTopSpammers.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded The line in question reads [ /tmp/rules.debug]:
Nov 4 14:14:28 	php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was 'no IP address found for grep: /tmp/rules.debug:24: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data no IP address found for grep: /tmp/rules.debug:26: file "/var/db/aliastables/pfBlockerNorthAmerica.txt" contains bad data no IP address found for grep: /tmp/rules.debug:28: file "/var/db/aliastables/pfBlockerOceania.txt" contains bad data no IP address found for grep: /tmp/rules.debug:30: file "/var/db/aliastables/pfBlockerSouthAmerica.txt" contains bad data no IP address found for grep: /tmp/rules.debug:32: file "/var/db/aliastables/pfBlockerTopSpammers.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded'

The firewall seems to be blocked then altogether. I guess there's an issue with rw mounting nanobsd…
Went back to Countryblock for the time being.
Featurewise pfBlocker seems great though!

marcelloc

I found a fetch issue While booting and on url table pfblocker get file function. ~~I'm fixing it today.~~

EDIT

version 0.1.4.1 released with fixes in:

boot process
embedded platform
url table get file package function

Many thanks for all of you that are helping us with this package.

Hugovsky

Very nice package, indeed. No errors to report so far. pfsense 2.0 release amd64.

hubsd

ok, now it takes 3/4 minutes to finish boot (better than 9 minutes)

I still have those warning messages

There were error(s) loading the rules: no IP address found for grep:
/tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data
pfctl: Syntax error in config file: pf rules not loaded The line in question reads [
/tmp/rules.debug]:

There were error(s) loading the rules: no IP address found for grep:
/tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data
pfctl: Syntax error in config file: pf rules not loaded The line in question reads [
/tmp/rules.debug]:

There were error(s) loading the rules: no IP address found for grep:
/tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data
pfctl: Syntax error in config file: pf rules not loaded The line in question reads [
/tmp/rules.debug]:

There were error(s) loading the rules: no IP address found for grep:
/tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data
pfctl: Syntax error in config file: pf rules not loaded The line in question reads [
/tmp/rules.debug]:

There were error(s) loading the rules: no IP address found for grep:
/tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data
pfctl: Syntax error in config file: pf rules not loaded The line in question reads [
/tmp/rules.debug]:

There were error(s) loading the rules: no IP address found for grep:
/tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data
pfctl: Syntax error in config file: pf rules not loaded The line in question reads [
/tmp/rules.debug]:

There were error(s) loading the rules: no IP address found for grep:
/tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data
pfctl: Syntax error in config file: pf rules not loaded The line in question reads [
/tmp/rules.debug]:

There were error(s) loading the rules: no IP address found for grep:
/tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data
pfctl: Syntax error in config file: pf rules not loaded The line in question reads [
/tmp/rules.debug]:

There were error(s) loading the rules: no IP address found for grep:
/tmp/rules.debug:18: file "/var/db/aliastables/pfBlockerEurope.txt" contains bad data
pfctl: Syntax error in config file: pf rules not loaded The line in question reads [
/tmp/rules.debug]:

marcelloc

what version are you using?

I have no embedded hardware to test it, I'm looking a way to do this using virtualbox.

hubsd

pfSense 2.0-RELEASE (i386) / nanobsd on Compact Flash 2 Gb
pfBlocker-dev 0.1.4.1

I have a 4 Gb microdrive somewhere, should I give it a try ?

marcelloc

I' ve tested virtualbox + pfsense 2.0 amd64 embedded kernel with no issues.

I'm not sure if embedded kernel will work the same way nanobsd on Compact Flash 2 Gb will do.

I found some posts on forum saying that is not easy to run nanobsd img on virtualbox, but I'm still trying..

marcelloc

hubsd,

Second test, first problem:

I've changed /etc/platform to nanobsd.
after a reboot mount -a returns:

/dev/ad0s1a on / (ufs, local, noatime, read-only, synchronous)
devfs on /dev (devfs, local)
/dev/md0 on /tmp (ufs, local)
/dev/md1 on /var (ufs, local)

So, read-only file system.

installed and configured pfblocker with no issues.

after a reboot,
I got the same error:


php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was 'no IP address found for grep: /tmp/rules.debug:15: file "/var/db/aliastables/pfBlockerTopSpammers.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded'
php: : New alert found: There were error(s) loading the rules: no IP address found for grep: /tmp/rules.debug:15: file "/var/db/aliastables/pfBlockerTopSpammers.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded The line in question reads [ /tmp/rules.debug]:
php: : There were error(s) loading the rules: no IP address found for grep: /tmp/rules.debug:15: file "/var/db/aliastables/pfBlockerTopSpammers.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [ /tmp/rules.debug]:

But, without saving any new config, I've looked at diagnostics-> table and pfBlockerTopSpammers was filled up normally, alias was there and firewall rules too.

So, Isn't it just a 'until boot finishes issue'?

Can you check if it happens the same way on your installation?

I know system alerts are annoying and have to be fixed but if package is still working, isn't it a minor issue?

Second problem:
boot takes 3 to 4 minutes.
it's a problem in fetch call that has no time out(or 01:15 timeout), I'll include a timeout patch to pfsense-tools.inc file on next release.

hubsd

Quick update, i exported my nanobsd config and imported it in a microdrive + classic embedded installation (so no nanobsd)

I can confirm that I have zero issue now and boot time takes 2 minutes.

So the problems I noticed are only while using nanobsd.

hubsd

@marcelloc:

hubsd,

Second test, first problem:

I've changed /etc/platform to nanobsd.
after a reboot mount -a returns:
/dev/ad0s1a on / (ufs, local, noatime, read-only, synchronous)
devfs on /dev (devfs, local)
/dev/md0 on /tmp (ufs, local)
/dev/md1 on /var (ufs, local)
So, read-only file system.

installed and configured pfblocker with no issues.

after a reboot,
I got the same error:
php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was 'no IP address found for grep: /tmp/rules.debug:15: file "/var/db/aliastables/pfBlockerTopSpammers.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded'
php: : New alert found: There were error(s) loading the rules: no IP address found for grep: /tmp/rules.debug:15: file "/var/db/aliastables/pfBlockerTopSpammers.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded The line in question reads [ /tmp/rules.debug]:
php: : There were error(s) loading the rules: no IP address found for grep: /tmp/rules.debug:15: file "/var/db/aliastables/pfBlockerTopSpammers.txt" contains bad data pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [ /tmp/rules.debug]:
But, without saving any new config, I've looked at diagnostics-> table and pfBlockerTopSpammers was filled up normally, alias was there and firewall rules too.

So, Isn't it just a 'until boot finishes issue'?

Can you check if it happens the same way on your installation?

I know system alerts are annoying and have to be fixed but if package is still working, isn't it a minor issue?

Second problem:
boot takes 3 to 4 minutes.
it's a problem in fetch call that has no time out(or 01:15 timeout), I'll include a timeout patch to pfsense-tools.inc file on next release.

Just saw your answer, well the problem is only until boot finishes issue yes ! It slow down boot time + spam warning messages. pfBlocker works fine so yes it's not a major issue and is because of nanobsd …

marcelloc

Version 0.1.4.2 includes fixes to 3-9 minutes boot on embedded systems.

hubsd

thanks !

Guest

just installed pfsense and this tonight…. getting emails of this:

There were error(s) loading the rules: /tmp/rules.debug:17: cannot define table pfBlockerAfrica: Cannot allocate memory
/tmp/rules.debug:19: cannot define table pfBlockerAsia: Cannot allocate memory
/tmp/rules.debug:21: cannot define table pfBlockerEurope: Cannot allocate memory
/tmp/rules.debug:23: cannot define table pfBlockerNorthAmerica: Cannot allocate memory
/tmp/rules.debug:25: cannot define table pfBlockerOceania: Cannot allocate memory
/tmp/rules.debug:27: cannot define table pfBlockerSouthAmerica: Cannot allocate memory
/tmp/rules.debug:29: cannot define table pfBlockerTopSpammers: Cannot allocate memory
/tmp/rules.debug:31: cannot define table pfBlockerBlockedLists: Cannot allocate memory
pfctl: Syntax error in config file: pf rules not loaded The line in question reads [17]: table <pfblockerafrica>persist file "/var/db/aliastables/pfBlockerAfrica.txt"

:( how do I fix?

edit just figured out how to up my tables size. Upped it to 100mil.

I'm testing the ad blocking lists and they aren't blocking anything. All I've done was add the lists and tell it to deny_inbound. Is there anything else I should be doing to make this work? Thanks.

Also if anyone knows of any great ad lists to help me block all the advertisements I possibly can please do let me know! :)</pfblockerafrica>

marcelloc

While blocking adwares, proxies, spywares the default way is outbound

Take a look in ipblocklist, the are many lists there.

Also pay attention on direction.
If you want to do not access somewhere, the action is deny outbound.
If you want that someone in somewhere do not access your network, the action is deny inbound.

If 100mil means you Speaker portuguese, there is also a pfBlocker topic in portuguese forum.

Guest

@marcelloc:

While blocking adwares, proxies, spywares the default way is outbound

Take a look in ipblocklist, the are many lists there.

Also pay attention on direction.
If you want to do not access somewhere, the action is deny outbound.
If you want that someone in somewhere do not access your network, the action is deny inbound.

If 100mil means you Speaker portuguese, there is also a pfBlocker topic in portuguese forum.

I'm American. Thanks for the clarification. I was up till 3am last night playing with this, tweaking and fixing several small issues. Still got a few left, but so far I am amazed at pfsense. This is way better than Mikrotik for my needs and its FREE. :)

sekular

Yea the quality of this package is outstanding, if i had money, i would donate to it. :D honest.

One feature request would be send all blocks to a new tab in the system logs status page called pfblocker and then have the ability to quickly add blocked ips to whitelist. Also a blocked log widget similar to the firewall logs one also.

marcelloc

@sekular:

Yea the quality of this package is outstanding.

Thanks for that, our main goal is write package that improves pfSense quality and help daily admins tasks.

@sekular:

One feature request would be send all blocks to a new tab in the system logs status page called pfblocker and then have the ability to quickly add blocked ips to whitelist. Also a blocked log widget similar to the firewall logs one also.

I can't be done until pfsense has a way to choose log direction or prefix.

pfblocker just help alias and rule creation.When you check enable log, it include rules with log enabled.

sekular

OK that makes sense. I guess adding a easyrule rule: pass from firewall will do the same thing.

I do not have xmlrpc sync enabled but i am getting this in syslog after update list jobs.

php: /pkg_edit.php: [pfblocker] pfblocker_xmlrpc_sync.php is starting.

Nov 8 12:30:00 php: : /etc/rc.update_urltables: Starting up.
Nov 8 12:30:00 php: : /etc/rc.update_urltables: Sleeping for 45 seconds.
Nov 8 12:30:45 php: : /etc/rc.update_urltables: Starting URL table alias updates
Nov 8 12:30:45 php: : /etc/rc.update_urltables: pfBlockerAsia does not need updated.
Nov 8 12:30:45 php: : /etc/rc.update_urltables: pfBlockerSouthAmerica does not need updated.
Nov 8 12:30:45 php: : /etc/rc.update_urltables: pfBlockerTopSpammers does not need updated.
Nov 8 12:30:45 php: : /etc/rc.update_urltables: pfBlockerMicrosoft does not need updated.
Nov 8 13:45:17 check_reload_status: Syncing firewall
Nov 8 17:51:00 check_reload_status: Syncing firewall
Nov 8 17:51:00 check_reload_status: Syncing firewall
Nov 8 17:51:00 php: /pkg_edit.php: [pfblocker] pfblocker_xmlrpc_sync.php is starting.

marcelloc

it's just a print before if

not a big deal.

never mind If you do not have checked sync option.