PfBlocker

marcelloc

PfBlocker is a new package that block countries and IP ranges. It joins Countryblock and IPblocklist.

The main goal of this package evolution was writing a package that only uses native functions of pfsense 2.0 instead of file hacks and table manipulation.

Current version is 1.0 and includes:

Countryblock features
Ipblocklist features
Dashboard widget
XMLRPC Sync
Dashboard widget with aliases applied and package hit
Lists update frequency
Many new options to choose what to block and how to block. If you want, you can assign network lists to your own rules.

This package is developed and maintained by marcelloc and tommyboy180.

kilthro

Thanks for all of the hard work to create this. You and Tommyboy180 have done a great job! I am waiting for most of the quirks to be worked out before I install it. I cant wait. :-)

marcelloc

All issues we found has already been fixed.

The last thing to code is lists update frequency.

Supermule

Is this compatible with 1.2.3??

marcelloc

No. Just for 2.0.

Pfsense 1.2.3 does not have many features that pfBlocker uses.

Supermule

ok :)

tommyboy180

IP-Blocklist and Countryblock will remain available for 1.2.3.

Supermule

Thks Tom. Havent changed yet since the 2.0 platform is not totally there yet. :)

It will be soon I hope….

tommyboy180

Version 1.3 problems.

Error: Ipblocklist .txt feature does not work. I have to use .gz file type on txt lists. My recommendation is to remove the URL format option.
Reason: All clear text lists will correctly work regardless of extension. If the list is compressed then it must use .gz as the extension.
Fix: Therefore we can remove the user selection and treat every URL as .gz regardless of extension.

Error: If no interfaces are selected then the package will not process
Reason: pfctl is looking for an interface in the rule syntax (“pass in quick on $LAN )”
Fix: IP-blocklist uses “any” as the default interface. I highly recommend using “any” as the default interface. We can at least have it as an option in the selection box. I get the best results using “any” in ipblocklist and countryblock.

Error: switching inbound and outbound interfaces stops pfblocker from running correctly after saving settings
Reason: unknown
Fix: unknown
Note: After changing interfaces on the fly and saving the settings the rules entry for pfblock are removed from rules.debug even though the table is still defined. Even after a filter reload the package still isn’t working

Error: Selecting loopback as an interface causes rules to be removed from rules.debug
Reason: unknown
Fix: unknown

Error: After blocking a specific IP with action “Deny Outbound” with ipblocklist feature I’m still able to ping
Reason: I’m not exactly sure, but I didn’t have this behavior in ipblocklist
Fix: unknown

Also I figured out fetch.
Problem: When adding dynamic links like http://list.iblocklist.com/?list=bt_ads&fileformat=p2p&archiveformat=gz the package tries to download the file.
Reason: Fetch breaks because of the & symbols. So since the begging of IPblocklist we’ve always had to use direct URLs.
Fix: Parse the URL in “”. Sofetch http://list.iblocklist.com/?list=bt_ads&fileformat=p2p&archiveformat=gzwill break but```
fetch –o FILENAME http://list.iblocklist.com/?list=bt_ads&fileformat=p2p&archiveformat=gz

You will have to assign a file name to each file otherwise the default filename will be something like “?list=bt_ads&fileformat=p2p&archiveformat=gz”

I haven’t had the time to find sample code changes for the above problems. I’m still looking deeper into the reason why some of those errors occur.

marcelloc

Erros fixed :

table reload is now forced after each config change
fix wrong php function to get clear text lists from url

Not erros:
When you choose no interfaces, pfBlocker creates only Aliases for any custom rules you want.
some examples:

block only smtp from top spammers
Block everything except inbound 80

Many thanks to tommyboy180.

NOTE:
These fixes solves almost all erros tommyboy180 found in 1.3
pfctrl was not reloading tables so almost all tests failed.
while doing new tests, check diagnostics -> tables updates.
I've did same tests after fix and seems to be ok for me.

how to apply? reinstall package.

johnnybe

Congratulations marcelloc and tommyboy180!
Very well done.

Just a note:
A better brief description in the Package Manager, and probably the most important: a Package Info link with more details.
Does it blocks MSN, Facebook, Orkut and this or that even behind a transparent Proxy or not?

marcelloc

If you have social networks ip ranges on a url, it will work on transparent proxies.

If you have squid on same pfsense you have to create a custom floating rule for this.

AhnHEL

Trying out your new pfblocker, awesome idea to combine packages.

I've added about 13 iblocklist lists in gz format and I'm getting this error on save.

php: /pkg_edit.php: New alert found: pfBlockerInbound alias table is too large. Reduce Inbound list or increase "Firewall Maximum Table Entries" value to at least 818134 in "system - advanced - Firewall/NAT".

I have 4 gigs RAM and my current Table Entries Size is 100000 so i raised it to 1000000. Enabled pfblocker and hit save and then got this

php: : There were error(s) loading the rules: /tmp/rules.debug:18: cannot define table pfBlockerInbound: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [18]: table <pfblockerinbound> persist file "/var/db/aliastables/pfBlockerInbound.txt"
Oct 30 02:11:29 	php: : New alert found: There were error(s) loading the rules: /tmp/rules.debug:18: cannot define table pfBlockerInbound: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded The line in question reads [18]: table <pfblockerinbound> persist file "/var/db/aliastables/pfBlockerInbound.txt"
Oct 30 02:11:26 	php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was '/tmp/rules.debug:18: cannot define table pfBlockerInbound: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded'</pfblockerinbound></pfblockerinbound>

Neither IPblocklist nor countryblock were installed when i installed pfblocker. Memory usage is at 19%.

Line 18 in /tmp/rules.debug

table <pfblockerinbound> persist file "/var/db/aliastables/pfBlockerInbound.txt"</pfblockerinbound>

marcelloc

Did you tried to reaply pfBlocker config, after increasing Firewall Maximum Table Entries to 1000000?

It seems that your change in "Firewall Maximum Table Entries" value did not take effect.

Cannot allocate memory pfctl: Syntax error in config file error is exactly what happens when "Firewall Maximum Table Entries" is lower then applied table size.

EDIT
Last package update was about 08 hours ago without version changing, so try to reinstall package to be sure you are on latest version.

dhatz

I'm looking forward to trying the new package, when you'll add configurable update frequency.

I'm currently using a couple of shell scripts, to add pf tables of unwanted IPs to pfsense, e.g.

http://www.spamhaus.org/drop/drop.lasso
DROP (Don't Route Or Peer) is an advisory "drop all traffic" list, consisting of stolen 'hijacked' netblocks and netblocks controlled entirely by professional spammers. DROP is a tiny subset of the SBL designed for use by firewalls and routing equipment.

http://feeds.dshield.org/top10-2.txt
DShield's current Most Active Attacking IPs
(Same data as is used on DShield.org Top 10 Most Wanted.)
0 = IP Address, 1 = Resolved domain of IP Address

which are both updated on a daily basis.

marcelloc

I'm looking forward to trying the new package, when you'll add configurable update frequency.

Maybe today.

I'll also check ips without netmask to be able to include dshield.org lists compatibility.

AhnHEL

@marcelloc:

Did you tried to reaply pfBlocker config, after increasing Firewall Maximum Table Entries to 1000000?

It seems that your change in "Firewall Maximum Table Entries" value did not take effect.

Cannot allocate memory pfctl: Syntax error in config file error is exactly what happens when "Firewall Maximum Table Entries" is lower then applied table size.

EDIT
Last package update was about 08 hours ago without version changing, so try to reinstall package to be sure you are on latest version.

Double checked Maximum Table Entries in /System/Advanced/Firewall-NAT, uninstalled and reinstalled package, even rebooted box, still same error. Only other packages running are Snort, Unbound and Cron.

tommyboy180

@dhatz:

I'm looking forward to trying the new package, when you'll add configurable update frequency.

I'm currently using a couple of shell scripts, to add pf tables of unwanted IPs to pfsense, e.g.

http://www.spamhaus.org/drop/drop.lasso
DROP (Don't Route Or Peer) is an advisory "drop all traffic" list, consisting of stolen 'hijacked' netblocks and netblocks controlled entirely by professional spammers. DROP is a tiny subset of the SBL designed for use by firewalls and routing equipment.

http://feeds.dshield.org/top10-2.txt
DShield's current Most Active Attacking IPs
(Same data as is used on DShield.org Top 10 Most Wanted.)
0 = IP Address, 1 = Resolved domain of IP Address

which are both updated on a daily basis.

iblocklist.com has a the DROP list from spamhaus.org URL: http://list.iblocklist.com/?list=sh_drop&fileformat=p2p&archiveformat=gz

iblocklist.com also has the dshield list from Bluetack URL: http://list.iblocklist.com/?list=bt_dshield&fileformat=p2p&archiveformat=gz

iblocklist.com has many lists in p2p format.

marcelloc

@onhel:

Double checked Maximum Table Entries in /System/Advanced/Firewall-NAT, uninstalled and reinstalled package, even rebooted box, still same error. Only other packages running are Snort, Unbound and Cron.

send me your lists in PM. I'll try on my vm.

dhatz

Great, I look forward to it.

Actually, I think pfsense would benefit from an enhanced aliastable feature, that would maintain (collect data, process, store) IP ranges of interest as text files in CIDR-notation fomat in /var/db/aliastables and load them into pf tables.

Those would include the usual IP black-list sources (e.g. Spamhaus' drop.lasso, DShield etc), country IP ranges (though I personally don't favor the practice of blocking entire countries) but also IP ranges of popular sites (Amazon EC2, Facebook, Google, Hotmail, SAP) to be black-listed or white-listed.

I think it's a feature that will be increasingly asked for, as more companies move to SaaS http://en.wikipedia.org/wiki/Software_as_a_service provider

In fact I recently suggested it Feature #1901: Maintain IP range tables for popular Internet sites having that in mind.

marcelloc

@dhatz:

Great, I look forward to it.

Actually, I think pfsense would benefit from an enhanced aliastable feature, that would maintain (collect data, process, store) IP ranges of interest as text files in CIDR-notation fomat in /var/db/aliastables and load them into pf tables.

Those would include the usual IP black-list sources (e.g. Spamhaus' drop.lasso, DShield etc), country IP ranges (though I personally don't favor the practice of blocking entire countries) but also IP ranges of popular sites (Amazon EC2, Facebook, Google, Hotmail, SAP) to be black-listed or white-listed.

I think it's a feature that will be increasingly asked for, as more companies move to SaaS http://en.wikipedia.org/wiki/Software_as_a_service provider

In fact I recently suggested it Feature #1901: Maintain IP range tables for popular Internet sites having that in mind.

Isn't this the main feature of pfBlocker lists??

tommyboy180

@marcelloc:

Isn't this the main feature of pfBlocker lists??

Yup. Exactly.

dhatz,
iblocklist.com has all those lists you asked for and more. p2p is a widely used format and is incredibly popular across many websites.

@dhatz:

…Actually, I think pfsense would benefit from an enhanced aliastable feature, that would maintain (collect data, process, store) IP ranges of interest as text files in CIDR-notation fomat in /var/db/aliastables and load them into pf tables….

That is exactly what is working in pfblocker right now!

@dhatz:

….Those would include the usual IP black-list sources (e.g. Spamhaus' drop.lasso, DShield etc), country IP ranges (though I personally don't favor the practice of blocking entire countries) but also IP ranges of popular sites (Amazon EC2, Facebook, Google, Hotmail, SAP) to be black-listed or white-listed….

That is exactly what the ipblocklist feature does in pfblocker. Your lists just have to follow p2p format which iblocklist.com has for the lists that you want!

pfblocker gives users more power when it comes to blocking IP ranges (country sources and list sources), whitelisting IP ranges (country sources and list sources), whitelisting popular IPs of sites like Google, Hotmail, etc, and are updated frequently by the list maintainer.
pfblocker already does everything that you are asking for!

dhatz

Well, I haven't tried it yet, but given the name pfBlocker I assumed it was mainly about blocking :D

PS: How "safe" is list.iblocklist.com as a source (site security, DNS hijacking, GPG signing etc) ?

tommyboy180

@dhatz:

Well, I haven't tried it yet, but given the name pfBlocker I assumed it was mainly about blocking :D

lol, so you're asking for features without even testing the software? Thanks.

marcelloc

@dhatz:

Well, I haven't tried it yet, but given the name pfBlocker I assumed it was mainly about blocking :D

PS: How "safe" is list.iblocklist.com as a source (site security, DNS hijacking, GPG signing etc) ?

I will follow Johnnybe suggestion and increase pfBlocker description for better understanding.

AhnHEL

@marcelloc:

send me your lists in PM. I'll try on my vm.

PM sent, thank you.

johnnybe

@marcelloc:

@dhatz:

Well, I haven't tried it yet, but given the name pfBlocker I assumed it was mainly about blocking :D

PS: How "safe" is list.iblocklist.com as a source (site security, DNS hijacking, GPG signing etc) ?

I will follow Johnnybe suggestion and increase pfBlocker description for better understanding.

Sounds good. Do it… do it. Thanks. ;D

marcelloc

php: : There were error(s) loading the rules: /tmp/rules.debug:18: cannot define table pfBlockerInbound: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded - The line in question reads [18]: table <pfblockerinbound> persist file "/var/db/aliastables/pfBlockerInbound.txt"
Oct 30 02:11:29 	php: : New alert found: There were error(s) loading the rules: /tmp/rules.debug:18: cannot define table pfBlockerInbound: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded The line in question reads [18]: table <pfblockerinbound> persist file "/var/db/aliastables/pfBlockerInbound.txt"
Oct 30 02:11:26 	php: : The command '/sbin/pfctl -o basic -f /tmp/rules.debug' returned exit code '1', the output was '/tmp/rules.debug:18: cannot define table pfBlockerInbound: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded'</pfblockerinbound></pfblockerinbound>

~~I seems to be a table hard limit. Even when setting Firewall Maximum Table Entries higher then 900000, pfctl fails to apply tables larger then this value.~~

EDIT:

There is no hard limit.
I found two issues:

pfsense Firewall Maximum Table Entries option may require to be 2x table size applied. I'm checking this.
To increase a table in pfctl, you must kill it to free old table used and create a new one. patch applied to pfBlocker

pfBlocker 0.1.4 will include these fixes.

marcelloc

@onhel:

PM sent, thank you.

Take a look on description of lists you selected. Level1 list from iblocklist is not nasty.

List Information

List name: level1
Author: Bluetack
Author's website: bluetack.co.uk
Author's description:
Companies or organizations who are clearly involved with trying to stop filesharing.
Companies which anti-p2p activity has been seen from.
Companies that produce or have a strong financial interest in copyrighted material.
Government ranges or companies that have a strong financial interest in doing work for governments.
Legal industry ranges.
IPs or ranges of ISPs from which anti-p2p activity has been observed.

sekular

Just want to confirm that the current version has resolved the problem that I reported in the country block thread.

I am still unable to deselect a country from the country lists. Once I have clicked in to a list I am unable to select none. If you could add a blank line that does not apply any list that could resolve it. Maybe that problem is browser specific?

marcelloc

I am still unable to deselect a country from the country lists. Once I have clicked in to a list I am unable to select none. If you could add a blank line that does not apply any list that could resolve it. Maybe that problem is browser specific?

Just hold CTRL and then unselect country.

johnpoz

Just curious – what is actually the point of this? Would not the default block rule just drop packets from these IPs anyway, unless they were in answer to something you requested in the first place.

Why would you be talking to these IPs in the first place, so other than say not logging this traffic Im not quite grasping the point of this package other than say something like running p2p software and you don't want to connect to any of these IPs, etc.

tommyboy180

@johnpoz:

Just curious – what is actually the point of this? Would not the default block rule just drop packets from these IPs anyway, unless they were in answer to something you requested in the first place.

Why would you be talking to these IPs in the first place, so other than say not logging this traffic Im not quite grasping the point of this package other than say something like running p2p software and you don't want to connect to any of these IPs, etc.

Lets say you run an email relay or have a website but you constantly get SPAMd or your website constantly gets hit by bad IPs. What can you do to block them all? Nothing really unless you take all your time and dedicate it to blocking each individual IP AFTER something was done.
What if you run a hotel or an open AP but you don't want people to visit mass sites or use p2p that is encrypted that gets past any layer7 filter.

Basically the uses and abilities of this package are endless. This package has it all and will help network admins secure their network even easier.

I have a user that works at a nuclear research facility. He uses countryblock to avoid other countries from corporate espionage. After using countryblock to block countries they don't have any business with their network attacks immediately dropped significantly. The benefits of this package is outstanding for admins all around the world.

marcelloc

version 0.1.4 is almost done.

some fixes and new gui options will be included for even more control on lists.

johnpoz

Yeah that dawned on me after I posted ;) If your running any sort of service that is open to the public, email/web/ntpd/ftp/etc then yeah it makes sense.. Might have to give it a test run, what would be nice is if it logged hits to this rule in a different log vs just firewall log so could see what kind of traffic getting from the bad IPs – for curiosity sake.

I don't really run anything open to the public other than p2p and ntpd (pool.ntp member) My ssh and openvpn access is locked down to require cert, etc. So no issue with bruteforce and and have linux box that runs sshd blocking IPs on 4 bad attempts anyway to keep the logs from filling up.

kevross33

If possible could you add in block lists for the following hostile lists:

Russian Business Network (bad), dshield and botnet CnCs - http://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
Compromised hosts - rules.emergingthreats.net/blockrules/compromised-ips.txt
Russian business network malvertisers - http://rules.emergingthreats.net/blockrules/rbn-malvertisers-ips.txt
Ciarmy list - http://ciarmy.com/

I blocked these inbound and outbound on my firewall already by linking a URL alias to these lists so it downloads them and then put that Alias in a block all traffic to and from on LAN & WAN interfaces and that stops my network talking to these IPs.

It would be nice though to have a tool where I could say click to block them and it will put the correct block in the firewall. You can get seperate lists for Dshield, RBN etc. I think this would help extend lots of blocking capabilities of bad hosts, malware command and control servers, bad servers distributing malware etc to normal users.

marcelloc

@kevross33:

If possible could you add in block lists for the following hostile lists:

Russian Business Network (bad), dshield and botnet CnCs - http://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt

Compromised hosts - rules.emergingthreats.net/blockrules/compromised-ips.txt

Russian business network malvertisers - http://rules.emergingthreats.net/blockrules/rbn-malvertisers-ips.txt

Ciarmy list - http://ciarmy.com/

I blocked these inbound and outbound on my firewall already by linking a URL alias to these lists so it downloads them and then put that Alias in a block all traffic to and from on LAN & WAN interfaces and that stops my network talking to these IPs.

It would be nice though to have a tool where I could say click to block them and it will put the correct block in the firewall. You can get seperate lists for Dshield, RBN etc. I think this would help extend lots of blocking capabilities of bad hosts, malware command and control servers, bad servers distributing malware etc to normal users.

It will be in version 0.1.4. these lists has only single ips.

dhatz

The first one (http://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt) is a mix of single IPs and CIDR, apparently discrete lists concatenated one after another.

tommyboy180

@dhatz:

The first one (http://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt) is a mix of single IPs and CIDR, apparently discrete lists concatenated one after another.

That list seems to be a combination of some lists on iblocklist.com. It looks like it's already available in a format that pfblocker could use in version 1.3.
Check iblocklist.com for any lists that you want. I'm guessing chances are it's there.

hubsd

interesting package !

can I use all list with an Alix box ? (AMD Geode / 256 MB RAM), I use 37% of RAM on my system.