PfBlocker
-
Thanks for the update: I am gonna give it a shot like right after I post this.
I noticed that my script would fail when some blocklist would include multiple colons on one line. Here is my updated script, which now accept a number of lines as the second argument.
#!/usr/bin/perl use Net::CIDR::Lite; my $filenum = "0" x 4; # if you ++ a string, it keeps the padding my $limit = 100000; # default max number of lines sub new_file { $filenum ++; my $name = "splat_$filenum.lis"; open OUT, ">$name" or die "canne open $name cap'n:$!\n"; warn "writing to:$name\n"; } my $cidr = Net::CIDR::Lite->new; open (MYFILE, $ARGV[0]); if ( defined($ARGV[1])) { if ( $ARGV[1] =~ m/^\d{2,6}$/ ) { $limit = int($ARGV[1]); } } while (<myfile>) { chomp; my @line = split(/:+/); my $range = $line[-1]; #get IP Range, verify it is IPv4 if ( $range =~ m/^\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\-\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}$/ ) { $cidr->add_range($range); } } close (MYFILE); my $index = 0; my @cidr_list = $cidr->list; foreach my $block ( @cidr_list ) { if ( $index % $limit == 0 ) { close (OUT); new_file; } print OUT $block,"\n"; $index++; } close (OUT);</myfile>Since my platform is not x64, and only have 256MB of RAM, I am not sure the new patch will fix the memory allocation issue for me… I am running with 60% memory used on average. Right now I am using a 60,000 lines as my maximum. 100,000 would seem to fail on some occasions.
-
Vlurk,
The memory patch is only for am64.
As we try to avoid file hacks, I've applied a value that is defined on config.inc but not reflected on gui.
As you have few memory available, the best option is the way you are doing or of course a hardware upgrade.Thank's for the script update.
-
I know Rules ReOrdering after a pfBlocker change has been covered in this thread.
I'd like to bring it back up because it's making me crazy.Here's my situation.
I use the pfBlocker widget. I also have my rules customized and ordered a certain way.In the last pfBlocker ver., I'd set every Action to Deny Inbound.
Next I'd customize and reorder the auto-created rules. I'd be finished in 10 min or so.
I'm pretty sure pfBlocker automatically changed Action to Alias when I had adjusted the rules.
The end result was the rules wouldn't change after an update.In this latest pfBlocker ver., my last method doesn't work. I have to set action to Alias myself.
If I don't, my rule changes are wiped out after every update.So, I make any changes at all to pfBlocker, I'm re-writing my blocking rules totally from scratch.
It's the only way I can have Widget+CustomizedRules+CustomRulesOrder.It's doubled my time to restore settings after each pfBlocker config change.
Selecting a single country becomes a 20+min process, per machine.I'm to weary to come up with any helpful suggestions/workarounds right now.
I'll revisit the thread when my brain is working again.Thanks.
edit: I had another look at the Backup feature and discovered the option for FirewallRules.
I've make my copy and will try to restore from it after my next pfBlocker change. -
Linuxtracker,
After a update, as well as I know, you need just to enable pfBlocker to get all your settings working again.
Maybe I misundertood you but I did not coded an automatic action switch from deny to alias only.
The steps I do for rule reordering are:
Apply pfBlocker conf with action I want on rules.
Change alias description on created firewall rules and then customize it's order.
Back on pfBlocker and change action to alias only. -
- Apply pfBlocker conf with action I want on rules.
- Change alias description on created firewall rules and then customize it's order.
- Back on pfBlocker and change action to alias only.
I did #1 and #2 and had just started on #3.
The moment I set the first country-group to alias (S.America) it tosses that country group off the list.
The remaining rules - order and customizations - were all reset.As near as I can tell, any change at all in pfBlocker now mandates that I rewrite my rules from scratch.
It may be that every list update does the same.
I offer that because the rules table completely reset about 11:30pm today - I have to rewrite them again. -
Linuxtracker,
How are you renaming rule description before changing action to alias only?
I did a clean install and then:-
Installed pfblocker
-
denied inbound access to argentina and some countries on Oceania
-
Renamed the rule description from South America to block Argentina
-
saved firewall rules and applied changes
-
back to pfblocker, set action to alias only on South America tab
-
saved config
After this, both rules(South america and Oceania) are still there.
I'll do some tests with lists applied too.
-
-
Linuxtracker,
How are you renaming rule description before changing action to alias only?
I don't change the rule descriptions that are generated by pfBlocker.
I figured they were necessary for the widget to work.When I write the rules from scratch, the descriptions are identical to the pfBlocker generated ones.
ie:```
pfBlockerSouthAmerica auto ruleThanks -
I did a clean install and then:
-
Installed pfblocker
-
denied inbound access to Argentina and some countries on Oceania
-
Renamed the rule description from South America to block Argentina
-
saved firewall rules and applied changes
-
back to pfblocker, set action to alias only on South America tab
-
saved config
After this, both rules(South America and Oceania) are still there.
I'll do some tests with lists applied too.
I need to clarify something.
- Renamed the rule description from South America to block Argentina
You mean you changed the rule description from "South America", so that it read "block Argentina" - correct?
The last time I changed my rule descriptions, my pfBlocker widget quit working.
So, I've kept my rules descriptions identical to whatever pfBlocker created.But:
It seems we can rename the pfBlocker-generated alias name
as long as the new alias name is at the beginning of the rules description.That won't break the widget. Do I understand correctly?
-
-
Linuxtracker,
I've changed rule description to "pfBlockerSouthAmerica deny inbound" to do not break widget and also included a list with every hour update and rules are still there.
-
Linuxtracker,
I've changed rule description to "pfBlockerSouthAmerica deny inbound" to do not break widget and also included a list with every hour update and rules are still there.
OK Thanks for your time on this.
I'll uninstall the package tonight and see what a fresh start yields.
Question: How do I force a manual list update?
-
Question: How do I force a manual list update?
As I forgot to include this option, you can change update frequency to every hour and then run
/usr/local/bin/php -q /usr/local/www/pfblocker.php cron
on console.
-
Linuxtracker,
I've changed rule description to "pfBlockerSouthAmerica deny inbound" to do not break widget and also included a list with every hour update and rules are still there.
My custom lists weren't pulling new updates. I don't think the countries were updating either.
So I uninstalled the package and deleted the pgblocker*.xml files in /usr/local/pkg.
After reinstalling pfBlocker, both lists and countries updated correctly.After that, I followed your guide as before.
Once my rules were setup, I went back into pfBlocker and changed Oceana from Deny All to Alias
and all my rule changes and ordering were thrown out.That made me sad.
Update:
So with a heavy heart I set out to rewrite my rules from scratch.
I set the rest of the pfBlocker options to Alias and applied the settings.I next went to rules - and discovered that my rule settings and ordering - were restored back to where I wanted them.
I am no longer sad. Now I am confused.
-
My custom lists weren't pulling new updates. I don't think the countries were updating either.
Did you tried to run it on console the way I described to you?
I don't think the countries were updating either.
Country lists are updated on pfblocker releases, not via cron job.
So I uninstalled the package and deleted the pgblocker*.xml files in /usr/local/pkg.
After reinstalling pfBlocker, both lists and countries updated correctly.After that, I followed your guide as before.
Once my rules were setup, I went back into pfBlocker and changed Oceana from Deny All to Alias
and all my rule changes and ordering were thrown out.That made me sad.
Update:
So with a heavy heart I set out to rewrite my rules from scratch.
I set the rest of the pfBlocker options to Alias and applied the settings.I next went to rules - and discovered that my rule settings and ordering - were restored back to where I wanted them.
I am no longer sad. Now I am confused.
I'll keep trying to simulate this issue.
All tests I did, preserving the aliasname on firewall rule description were fine. -
Did you tried to run it on console the way I described to you?
Yes. That did work and helped me to find a misspelled list name.
It'll also come in handy in the future.
Country lists are updated on pfblocker releases, not via cron job.
After I uninstalled the package -> deleted the pfblocker*.xml files -> reinstalled it - the country lists updated normally.
I think the package handler was wonky and didn't update pfblocker properly the last time.
I'll keep trying to simulate this issue.
All tests I did, preserving the aliasname on firewall rule description were fine.That my rules would suddenly show correctly - after they were reset - seems really strange.
I have other pfSense boxes out there. I'll update one or two of them and see if any issues pop up.
I certainly appreciate your efforts.
For now I'll keep looking into things on my end. -
My exp for using pfBlocker ,
if I set max table size = 100000 ,
there still have error logged ,php: : New alert found: There were error(s) loading the rules: /tmp/rules.debug:23: cannot define table pfBlockerNorthAmerica: Cannot allocate memory pfctl: Syntax error in config file: pf rules not loaded The line in question reads [23]: table <pfblockernorthamerica>persist file "/var/db/aliastables/pfBlockerNorthAmerica.txt"
if I set max table size = 1000000
problem solve , just like to share .</pfblockernorthamerica>
-
I wonder if the pfBlocker developers have considered using pf anchors ( http://openbsd.org/faq/pf/anchors.html ) ?
IMHO it'd be a nice design practice for pfsense packages to use anchors.
Check article http://forum.pfsense.org/index.php/topic,45277.0.html which among others notes the recent pf extensions by Apple to make sure Mac OS X applications that interact with the packet filter configuration do not clobber each others' rules.
-
dhatz,
Pfblocker use pfsense firewall rules and url table.
No pf rule is created by this package, only xml info to pfsense alias and rules.
anyway, thanks for this suggestion :)
-
I managed to get Pfblocker to work here (2.0.1 i386 full), and have a few suggestions for improvement. No obvious tracker so posting them here!
-
If php memory still needs editing, can there be an option "increase/override PHP memory" (hacking code is ugly compared to interface or config options). If not possible and yet needed should a feature be requested for system tunables -> PHP memory?
-
Can "reload lists" and "clear tables" buttons be added? The first because if it fails (and perhaps isn't aware it failed) it can wait a long time to try again, the second because if used on hardware that can't increase PHP allocation to display the tables and their "delete all" option, the user doesn't have any easy way to delete the tables.
-
Can we have "allow on all interfaces" to match "block on all interfaces"? There are allow/exclusion lists in use too.
-
Last request is a bit more involved. Can there be an option that if set, pfblock produces a merged allow/block lists (by removing allow IPs from the block ranges) or can automatically set the tag rules below, so that manual aliases + tagging aren't needed when using an allow/exclusion list.
As well as blocklists, I-BL publish known exceptions (the "exclusions" list), and these can need sneaky config. I'm new-ish to this but AFAIK this works, if so then can someone add it to the Pfblock FAQ/wiki/howto?
Example:
Imagine Spamhaus "evil-spam" list contains 44.0.0.0/12, I-Blocklist exclusion list contains a number of important exclusions including one for 44.0.8.0/24 (nice company/genuine news site), and I currently have the firewall set to deny by default all unsolicited incoming traffic other than on specific open ports: 192.168.1.15:[3000-3001] and 192.168.1.22:[20000-20001]. Nothing unusual - we want to block evil ranges, not autoblock known exclusions, and have a few tightly controlled open ports. There are just 4 rules plus "default deny": (A) block Spamhaus "evil spam" list, (B) Allow I-Blocklist "exclusions" list, (C) Open port for any traffic with destination 192.168.1.15:[3000-3001], (D) Open port for any traffic with destination 192.168.1.22:[20000-20001].
Normal filters won't work. If the rules are ordered B->A->C->D then traffic from 44.0.8.0/24 or any other "exclusion list" site for other servers or ports will "pass" under rule B before they can be rejected by rule C/D. If ordered C->D->B->A then traffic from "evil spam" list sites won't be blocked from the open ports. The problem is that downloadable exclusions/allow lists aren't an absolute allow, they only mean the downloadable block lists shouldn't be applied to those ranges. Also these are regularly updated lists so ditching them or manually editing are both unlikely to be optimal. What's really needed is a way to say "don't block via rule: <blocklist alias="">if the packet matches rule:<allowlist alias="">".
I use tagging (AKA packet marking, policy filters):
- Test packet against block rule -> tag as "BLOCKLIST" if a match (see "advanced" options at the end of the rule definition form)
- Test packet against allow rule -> delete the tag if a match (set to "")
- Test if packet tag == "BLOCKLIST" -> block it.
- Continue with other rules
Result - packets matching the allow list aren't blocked, but nor are they immediately passed. They get tested against the rest of the rules. I'm not sure if 2+3 can be combined to make 2 rules: (Test packet against allow list INVERTED and tag if matched with "NOTEXCLUDED") -> (Test packet against double condition: matches blocklist AND tag = "NOTEXCLUDED", block if a match).
Haven't seen a reference to this here so hopefully it will help someone!</allowlist></blocklist>
-
-
Hi stilez,
Thanks for your feedback.
The pfBlocker wiki is:
http://doc.pfsense.org/index.php/PfblockerIf php memory still needs editing, can there be an option "increase/override PHP memory" (hacking code is ugly compared to interface or config options). If not possible and yet needed should a feature be requested for system tunables -> PHP memory?
I'll try to push an update to pfsense system advanced features to enable this but I have first to ask core team if It's a feature they want on pfsense.
Can "reload lists" and "clear tables" buttons be added? The first because if it fails (and perhaps isn't aware it failed) it can wait a long time to try again, the second because if used on hardware that can't increase PHP allocation to display the tables and their "delete all" option, the user doesn't have any easy way to delete the tables.
The update now/reload list is a feature I forgot to include, but it is on my todo list. :)
To remove a table, just delete the list that generated it and the table will be flushed.
Can we have "allow on all interfaces" to match "block on all interfaces"? There are allow/exclusion lists in use too.
If you select all interfaces on Inbound and create a list/country list with allow inbound isn't it a way to get it working?
Last request is a bit more involved. Can there be an option that if set, pfblock produces a merged allow/block lists (by removing allow IPs from the block ranges) or can automatically set the tag rules below, so that manual aliases + tagging aren't needed when using an allow/exclusion list.
As well as blocklists, I-BL publish known exceptions (the "exclusions" list), and these can need sneaky config. I'm new-ish to this but AFAIK this works, if so then can someone add it to the Pfblock FAQ/wiki/howto?
allow action works as a whitelist and is applied before deny rules.
If you block 44.0.0.0/12 on a list and allow 44.0.8.0/24 on other list, it will workExample:
Imagine Spamhaus "evil-spam" list contains 44.0.0.0/12, I-Blocklist exclusion list contains a number of important exclusions including one for 44.0.8.0/24 (nice company/genuine news site), and I currently have the firewall set to deny by default all unsolicited incoming traffic other than on specific open ports: 192.168.1.15:[3000-3001] and 192.168.1.22:[20000-20001]. Nothing unusual - we want to block evil ranges, not autoblock known exclusions, and have a few tightly controlled open ports. There are just 4 rules plus "default deny": (A) block Spamhaus "evil spam" list, (B) Allow I-Blocklist "exclusions" list, (C) Open port for any traffic with destination 192.168.1.15:[3000-3001], (D) Open port for any traffic with destination 192.168.1.22:[20000-20001].
Using action as alias only, you can create rules the way you want on firewall -> rules
-
Can we have "allow on all interfaces" to match "block on all interfaces"? There are allow/exclusion lists in use too.
If you select all interfaces on Inbound and create a list/country list with allow inbound isn't it a way to get it working?
Not really.
Well, technically yes but at the cost that it's a global setting for all PFblock rules, meaning every other list would also get applied to all interfaces. You might not want that. (If it was that easy then logically nobody would need "Deny Both" as an option, they'd just use "Deny Inbound" and trap all interfaces instead! If "Deny Both" isn't redundant then logically "Allow Both" probably isn't either.) Simple option worth adding, for completeness if nothing else :)
Last request is a bit more involved. Can there be an option that if set, pfblock produces a merged allow/block lists (by removing allow IPs from the block ranges) or can automatically set the tag rules below, so that manual aliases + tagging aren't needed when using an allow/exclusion list.
As well as blocklists, I-BL publish known exceptions (the "exclusions" list), and these can need sneaky config. I'm new-ish to this but AFAIK this works, if so then can someone add it to the Pfblock FAQ/wiki/howto?
allow action works as a whitelist and is applied before deny rules.
If you block 44.0.0.0/12 on a list and allow 44.0.8.0/24 on other list, it will workExample:
Imagine Spamhaus "evil-spam" list contains 44.0.0.0/12, I-Blocklist exclusion list contains a number of important exclusions including one for 44.0.8.0/24 (nice company/genuine news site), and I currently have the firewall set to deny by default all unsolicited incoming traffic other than on specific open ports: 192.168.1.15:[3000-3001] and 192.168.1.22:[20000-20001]. Nothing unusual - we want to block evil ranges, not autoblock known exclusions, and have a few tightly controlled open ports. There are just 4 rules plus "default deny": (A) block Spamhaus "evil spam" list, (B) Allow I-Blocklist "exclusions" list, (C) Open port for any traffic with destination 192.168.1.15:[3000-3001], (D) Open port for any traffic with destination 192.168.1.22:[20000-20001].
Using action as alias only, you can create rules the way you want on firewall -> rules
No, I wonder if it's well explained. In fact you can do it with aliases (that's how I'm doing it) but not because it acts as a whitelist and not in an easy obvious way. The problem is that normally the flow of rules is like this:
rule 1 -> if matched then pass/block/refuse/route otherwise continue with rule 2
rule 2 -> if matched then pass/block/refuse/route otherwise continue with rule 3
rule 3 -> if matched then pass/block/refuse/route otherwise continue with rule 4
etc (eventually some rule matches or the default rule applies)But with block and allow lists the flow is like this:
rule 1 -> if matched then pass/block/refuse/route otherwise continue with rule 2
rule 2 -> if matched then pass/block/refuse/route otherwise continue with rule 3
rule 3 (ALLOW LIST) if matched then IGNORE/SKIP rule 4 and continue with rule 5 otherwise continue with rule 4
rule 4 (BLOCK LIST) if matched then block/refuse otherwise continue with rule 5
rule 5 -> if matched then pass/block/refuse/route otherwise continue with rule 6
etcAn allow list doesn't cause a pass or block event, it causes a skip event - the block list won't be tested. Unfortunately usual pfsense rules don't include a "skip" option, you can't add (without advanced options) a rule that says "if matched, ignore some other rules and continue with checking the rules after them". You also can't avoid this structure if an allow list is used, unless your other rules are really simple, but we can say that people choosing pfsense usually won't have such very simple rules. Logically it can't be turned into a list that avoids skip-on-match. You can do it only 2 ways, by adding tag conditions to the existing automatic rules or by pre-merging the lists to create an "effective" block list (without allow list), either works. Logically, allow lists really set an automatic extra tag condition on the pfblock automatic rules which may be needed whenever an allow list is used, but I think it's not obvious.
