PfBlocker

[stilez]

I managed to get Pfblocker to work here (2.0.1 i386 full), and have a few suggestions for improvement. No obvious tracker so posting them here!

• If php memory still needs editing, can there be an option "increase/override PHP memory" (hacking code is ugly compared to interface or config options). If not possible and yet needed should a feature be requested for system tunables -> PHP memory?

• Can "reload lists" and "clear tables" buttons be added? The first because if it fails (and perhaps isn't aware it failed) it can wait a long time to try again, the second because if used on hardware that can't increase PHP allocation to display the tables and their "delete all" option, the user doesn't have any easy way to delete the tables.

• Can we have "allow on all interfaces" to match "block on all interfaces"? There are allow/exclusion lists in use too.

• Last request is a bit more involved. Can there be an option that if set, pfblock produces a merged allow/block lists (by removing allow IPs from the block ranges) or can automatically set the tag rules below, so that manual aliases + tagging aren't needed when using an allow/exclusion list.

  As well as blocklists, I-BL publish known exceptions (the "exclusions" list), and these can need sneaky config. I'm new-ish to this but AFAIK this works, if so then can someone add it to the Pfblock FAQ/wiki/howto?

  Example:

  Imagine Spamhaus "evil-spam" list contains 44.0.0.0/12, I-Blocklist exclusion list contains a number of important exclusions including one for 44.0.8.0/24 (nice company/genuine news site), and I currently have the firewall set to deny by default all unsolicited incoming traffic other than on specific open ports: 192.168.1.15:[3000-3001] and 192.168.1.22:[20000-20001]. Nothing unusual - we want to block evil ranges, not autoblock known exclusions, and have a few tightly controlled open ports. There are just 4 rules plus "default deny": (A) block Spamhaus "evil spam" list, (B) Allow I-Blocklist "exclusions" list, (C) Open port for any traffic with destination 192.168.1.15:[3000-3001], (D) Open port for any traffic with destination 192.168.1.22:[20000-20001].

  Normal filters won't work. If the rules are ordered B->A->C->D then traffic from 44.0.8.0/24 or any other "exclusion list" site for other servers or ports will "pass" under rule B before they can be rejected by rule C/D. If ordered C->D->B->A then traffic from "evil spam" list sites won't be blocked from the open ports.  The problem is that downloadable exclusions/allow lists aren't an absolute allow, they only mean the downloadable block lists shouldn't be applied to those ranges. Also these are regularly updated lists so ditching them or manually editing are both unlikely to be optimal. What's really needed is a way to say "don't block via rule: <blocklist alias="">if the packet matches rule:<allowlist alias="">".

  I use tagging (AKA packet marking, policy filters):

  1. Test packet against block rule -> tag as "BLOCKLIST" if a match (see "advanced" options at the end of the rule definition form)
  2. Test packet against allow rule -> delete the tag if a match (set to "")
  3. Test if packet tag == "BLOCKLIST" -> block it.
  4. Continue with other rules

  Result - packets matching the allow list aren't blocked, but nor are they immediately passed. They get tested against the rest of the rules.  I'm not sure if 2+3 can be combined to make 2 rules: (Test packet against allow list INVERTED and tag if matched with "NOTEXCLUDED") -> (Test packet against double condition: matches blocklist AND tag = "NOTEXCLUDED", block if a match).

  Haven't seen a reference to this here so hopefully it will help someone!</allowlist></blocklist>

[marcelloc]

Hi stilez,

Thanks for your feedback.

The pfBlocker wiki is:
http://doc.pfsense.org/index.php/Pfblocker

@stilez:

If php memory still needs editing, can there be an option "increase/override PHP memory" (hacking code is ugly compared to interface or config options). If not possible and yet needed should a feature be requested for system tunables -> PHP memory?

I'll try to push an update to pfsense system advanced features to enable this but I have first to ask core team if It's a feature they want on pfsense.

@stilez:

Can "reload lists" and "clear tables" buttons be added? The first because if it fails (and perhaps isn't aware it failed) it can wait a long time to try again, the second because if used on hardware that can't increase PHP allocation to display the tables and their "delete all" option, the user doesn't have any easy way to delete the tables.

The update now/reload list is a feature I forgot to include, but it is on my todo list.  :)

To remove a table, just delete the list that generated it and the table will be flushed.

@stilez:

Can we have "allow on all interfaces" to match "block on all interfaces"? There are allow/exclusion lists in use too.

If you select all interfaces on Inbound and create a list/country list with allow inbound isn't it a way to get it working?

@stilez:

Last request is a bit more involved. Can there be an option that if set, pfblock produces a merged allow/block lists (by removing allow IPs from the block ranges) or can automatically set the tag rules below, so that manual aliases + tagging aren't needed when using an allow/exclusion list.

As well as blocklists, I-BL publish known exceptions (the "exclusions" list), and these can need sneaky config. I'm new-ish to this but AFAIK this works, if so then can someone add it to the Pfblock FAQ/wiki/howto?

allow action works as a whitelist and is applied before deny rules.
If you block 44.0.0.0/12 on a list and allow 44.0.8.0/24 on other list, it will work

@stilez:

Example:

Imagine Spamhaus "evil-spam" list contains 44.0.0.0/12, I-Blocklist exclusion list contains a number of important exclusions including one for 44.0.8.0/24 (nice company/genuine news site), and I currently have the firewall set to deny by default all unsolicited incoming traffic other than on specific open ports: 192.168.1.15:[3000-3001] and 192.168.1.22:[20000-20001]. Nothing unusual - we want to block evil ranges, not autoblock known exclusions, and have a few tightly controlled open ports. There are just 4 rules plus "default deny": (A) block Spamhaus "evil spam" list, (B) Allow I-Blocklist "exclusions" list, (C) Open port for any traffic with destination 192.168.1.15:[3000-3001], (D) Open port for any traffic with destination 192.168.1.22:[20000-20001].

Using action as alias only, you can create rules the way you want on firewall -> rules

[stilez]

@marcelloc:

@stilez:

Can we have "allow on all interfaces" to match "block on all interfaces"? There are allow/exclusion lists in use too.

If you select all interfaces on Inbound and create a list/country list with allow inbound isn't it a way to get it working?

Not really.

Well, technically yes but at the cost that it's a global setting for all PFblock rules, meaning every other list would also get applied to all interfaces. You might not want that. (If it was that easy then logically nobody would need "Deny Both" as an option, they'd just use "Deny Inbound" and trap all interfaces instead! If "Deny Both" isn't redundant then logically "Allow Both" probably isn't either.) Simple option worth adding, for completeness if nothing else :)

@marcelloc:

@stilez:

Last request is a bit more involved. Can there be an option that if set, pfblock produces a merged allow/block lists (by removing allow IPs from the block ranges) or can automatically set the tag rules below, so that manual aliases + tagging aren't needed when using an allow/exclusion list.

As well as blocklists, I-BL publish known exceptions (the "exclusions" list), and these can need sneaky config. I'm new-ish to this but AFAIK this works, if so then can someone add it to the Pfblock FAQ/wiki/howto?

allow action works as a whitelist and is applied before deny rules.
If you block 44.0.0.0/12 on a list and allow 44.0.8.0/24 on other list, it will work

@stilez:

Example:

Imagine Spamhaus "evil-spam" list contains 44.0.0.0/12, I-Blocklist exclusion list contains a number of important exclusions including one for 44.0.8.0/24 (nice company/genuine news site), and I currently have the firewall set to deny by default all unsolicited incoming traffic other than on specific open ports: 192.168.1.15:[3000-3001] and 192.168.1.22:[20000-20001]. Nothing unusual - we want to block evil ranges, not autoblock known exclusions, and have a few tightly controlled open ports. There are just 4 rules plus "default deny": (A) block Spamhaus "evil spam" list, (B) Allow I-Blocklist "exclusions" list, (C) Open port for any traffic with destination 192.168.1.15:[3000-3001], (D) Open port for any traffic with destination 192.168.1.22:[20000-20001].

Using action as alias only, you can create rules the way you want on firewall -> rules

No, I wonder if it's well explained. In fact you can do it with aliases (that's how I'm doing it) but not because it acts as a whitelist and not in an easy obvious way. The problem is that normally the flow of rules is like this:

rule 1 -> if matched then pass/block/refuse/route otherwise continue with rule 2
rule 2 -> if matched then pass/block/refuse/route otherwise continue with rule 3
rule 3 -> if matched then pass/block/refuse/route otherwise continue with rule 4
etc (eventually some rule matches or the default rule applies)

But with block and allow lists the flow is like this:

rule 1 -> if matched then pass/block/refuse/route otherwise continue with rule 2
rule 2 -> if matched then pass/block/refuse/route otherwise continue with rule 3
rule 3 (ALLOW LIST)  if matched then IGNORE/SKIP rule 4 and continue with rule 5 otherwise continue with rule 4
rule 4 (BLOCK LIST)  if matched then block/refuse otherwise continue with rule 5
rule 5 -> if matched then pass/block/refuse/route otherwise continue with rule 6
etc

An allow list doesn't cause a pass or block event, it causes a skip event - the block list won't be tested. Unfortunately usual pfsense rules don't include a "skip" option, you can't add (without advanced options) a rule that says "if matched, ignore some other rules and continue with checking the rules after them". You also can't avoid this structure if an allow list is used, unless your other rules are really simple, but we can say that people choosing pfsense usually won't have such very simple rules. Logically it can't be turned into a list that avoids skip-on-match.  You can do it only 2 ways, by adding tag conditions to the existing automatic rules or by pre-merging the lists to create an "effective" block list (without allow list), either works.  Logically, allow lists really set an automatic extra tag condition on the pfblock automatic rules which may be needed whenever an allow list is used, but I think it's not obvious.

[marcelloc]

I understand your point but it will not be easy to exclude a cidr inside another cidr.

The skip rule option is native on ipfw, and as well as I know about pfsense, floating rules use ipfw but it think there is no skip option on gui to apply it.

The idea of selecting interfaces for each tab is not bad, but I think most users prefer a simple way to apply conf. Every advanced rule apply can be done using alias just like you do.

[stilez]

@marcelloc:

I understand your point but it will not be easy to exclude a cidr inside another cidr.

The skip rule option is native on ipfw, and as well as I know about pfsense, floating rules use ipfw but it think there is no skip option on gui to apply it.

Yeah, that's what I found. But man pf suggests a way to get round it perhaps so simple that pfblock's rules could be tweaked to do it automatically.

Suppose we have 2 lists, normally Pfblock creates automated aliases, for example "PFBLOCK_BLOCKS" and "PFBLOCK_ALLOWS". It uses these to create automated rules:

[some rules…]
If (MATCHES: PFBLOCK_ALLOWS) then (pass)
If (MATCHES: PFBLOCK_BLOCKS) then (block/reject)
[other rules…]

Pfblock has to do that because that's pretty much all that it can do with usual rule conditions.
But suppose when an allow list is in use, the automatic rules had these different conditions:

[some rules…]
If (MATCHES: PFBLOCK_BLOCKS) then (ADVANCED: TAG WITH: "PFBLOCK_BLOCK")
If (! MATCHES: PFBLOCK_ALLOWS) AND (ADVANCED: IS_TAGGED WITH: "PFBLOCK_BLOCK") then (block/reject)
[other rules…]

This uses ordinary Pfsense rules to do exactly what's desired, exclude a CIDR within a CIDR.
It's not just good for simple pfblock lists, but also multiple block/allow lists and any mix of reject/block, exactly correctly:

[some rules…]
If (MATCHES: PFBLOCK_REJECTLIST_1) then (ADVANCED: TAG WITH: "PFBLOCK_REJECT")
If (MATCHES: PFBLOCK_REJECTLIST_2) then (ADVANCED: TAG WITH: "PFBLOCK_REJECT")
If (MATCHES: PFBLOCK_BLOCKLIST_1) then (ADVANCED: TAG WITH: "PFBLOCK_BLOCK")
If (MATCHES: PFBLOCK_BLOCKLIST_2) then (ADVANCED: TAG WITH: "PFBLOCK_BLOCK")
If (MATCHES: PFBLOCK_ALLOWS_1) then (ADVANCED: TAG WITH: <null string="">or "PFBLOCK_ALLOW")
If (MATCHES: PFBLOCK_ALLOWS_2) then (ADVANCED: TAG WITH: <null string="">or "PFBLOCK_ALLOW")
IF (ADVANCED: IS_TAGGED WITH: "PFBLOCK_BLOCK") then (block)
IF (ADVANCED: IS_TAGGED WITH: "PFBLOCK_REJECT") then (reject)
[other rules…]

This sequence of testing and use of advanced -> tag conditions (built into pfsense rules) correctly handles any combination of block, reject or allow lists on any interface, that it's possible to configure within pfblock. It prioritises block over reject if the IP is on both kinds of list. It's also simple, efficient and easy to understand. If pfblock created these automatic rules rather than its existing ones it would then correctly handle any lists the user can enter.</null></null>

[marcelloc]

That's a good suggestion, include a checkbox with tag rules for example.

I'm quite busy on work + sarg package + mailscanner quarantine tab but it's on my todo list now.

[stilez]

@marcelloc:

I'm quite busy on work + sarg package + mailscanner quarantine tab but it's on my todo list now.

Sounds good. I'm trying to figure how to use syslog-ng, logzilla/octopussy or some other advanced syslog web viewer working with pfsense so the logs can be kept on the router and analysed/viewed in full via the web. Don't suppose you know whether there's a simple how-to pfsense this that I missed anywhere? :)

[marcelloc]

Sorry, I've never configured a syslog server on pfsense.

You can install almost any freebsd8.1 package on pfsense, so I think it will not That hard to configure.

[kaneda]

I just installed the package but it seems that i must add manually urls to get ip lists to deny.
I think it could be great to have the option of a drop down list with most known black list of spam, malware, etc… without having to put it manually, of course, at the end for example an option like "custom" where you can put custom urls.

The great option is to indentify the most stable and sopported lists and make the package able to interprete them automatically.

I think could be great to add individually lists from google, antivirus companies, antispam blacklists,
Some examples to start  looking for most interesting sources of ip lists:

http://www.malware.com.br/lists.shtml
http://sites.google.com/site/blocklist/
http://www.phishtank.com/
http://www.taringa.net/posts/info/10070366/Listas-bloqueo-de-direcciones-IP-y-URL-malicioso-Ing.html
http://www.getmantra.com/galley/malwares/index.html
and alienware, the authors of OSSIM hace another great service of ip reputation to do something similar to spamassasin with ip addresses:
http://labs.alienvault.com/labs/index.php/projects/open-source-ip-reputation-portal/list-ips/

[marcelloc]

You can add any ip based/cidr list to pfblocker but there is no plan to create dropdown list pointing to external links.

Anyway, thanks for you suggestion.

[marcelloc]

Hi all,

The country lists that we use on pfblocker will move to a paid service, take a look:

Country IP Blocks is moving to a paid services model

att,
Marcello Coutinho

[LinuxTracker]

I combined the Bluetack Level 1 and the TBG Primary Threats lists.  I converted them into CIDR and (I hope) trimmed out the overlapping ranges.
Neither Pastebin nor Pastie would take 6MB pastes so I sent the gz files to an uploader service.

IPs .0-.68 http://efjf7g.dl4free.com/en/
IPs .69-.193 http://6pvjo6.dl4free.com/en/
IPs .194-255 http://zsh382.dl4free.com/en/

Here's the CIDR count

The original Peerguardian formatted files from IBlocklist seem to require a lot of resources to convert.

Also, BlueTackL1 and TGP-PT have a lot of overlap so I felt converting+filtering them would help minimize resources.

Sorry - I'm not clever enough to automate this process or offer a live update service.

Since these are blind d/l's for you, prudence dictates you examine the contents before use.

[kilthro]

@marcelloc:

Hi all,

The country lists that we use on pfblocker will move to a paid service, take a look:

Country IP Blocks is moving to a paid services model

att,
Marcello Coutinho

:-(

So does that mean we will have to subscribe and then add the lists into pfblocker ourselves?

[LinuxTracker]

US Spammers:
IPs 0-68 http://pastebin.com/iKTpuy9n
IPs 69-179 http://pastebin.com/8Z6iSCSp
IPs 180-255 http://pastebin.com/PJLjUWeA

Other Spammers:
Worst: http://pastebin.com/aKnBhMLX
Corp Spamvertisers: http://pastebin.com/3Yd97rZ4
Non-US: http://pastebin.com/t9ej10BZ

Worst Spammers are the nitwits who were filling up my logs so I moved them into their own group.
They may not be targeting you and could be wrapped into other lists.

Corp Spam is stuff like Constant Contact, Linked-In, FTD, PsychzNetworks, ExactTarget….
They may not be blacklisted by DNSBLs but they're a pain in my butt.

Attn: marcelloc and Tommyboy180
I can't tell if my PMs get through to you because my browser doesn't properly redirect after posting.

The URLs I sent you before you have new filenames.  They are:
US_SpamIPs_0-69.txt.gz
US_SpamIPs_70-179.txt.gz
US_SpamIPs_180-255.txt.gz
Non-US_Spam_IPs.txt.gz
CorporateSpam.txt.gz
Worst_Spammers.txt.gz

The rest of the URL is the same.

marcelloc, if you'd delete the old files you're hosting and post these instead I'd appreciate it.

[marcelloc]

Linuxtracker,
I did received Pm, I'll publish files this week.

Kilthro,
Updated lists will need a subscribe but the files we already have will still be there.

Until april I'll update lists again and of course try to implement countryip subscription on pfBlocker package.

[kilthro]

@marcelloc:

Linuxtracker,
I did received Pm, I'll publish files this week.

Kilthro,
Updated lists will need a subscribe but the files we already has will still be there.

Until april I'll update lists again and of course try to implement countryip subscription on pfBlocker package.

That's what I thought. Thanks for the clarification.

[LinuxTracker]

@marcelloc:

Linuxtracker,
I did received Pm, I'll publish files this week.

Kilthro,
Updated lists will need a subscribe but the files we already have will still be there.

Until april I'll update lists again and of course try to implement countryip subscription on pfBlocker package.

Not good enough marcelloc.  I want more things done and I want them done now.
. . . or not.

I guess busting your tail for little more than a thanks will have to be good enough for me.

[freakalad]

Hi guys,

This seems to be the most appropriate place to ask this question, since it closest matches my need.

There have been a few recent reports re issues associated with ads on mobiles, as well putting load on batteries (hardly seems surprising), and I don't want to root my Android to block ads (if I can block ads for all nodes, all the better):

• http://www.engadget.com/2012/03/19/android-study-privacy-security-risks-in-app-ads/
• http://www.csc.ncsu.edu/faculty/jiang/pubs/WISEC12_ADRISK.pdf
• http://www.pcworld.com/article/252204/free_android_apps_packed_with_ads_are_major_battery_drains.html

We have several such devices in-house, and I'd like to simply block the ads at the boundary.
I've seen posts (19756 & 12737) providing a hack by virtue of Squid (I'd like to avoid adding Squid for this limited function if possible, since it seems overkill), but I'd like to leverage something similar to the Android app AdFree, that simply does a hosts file amendment to void IP's/DNS's: http://www.mvps.org/winhelp2002/hosts.txt

I've tried manually adding this data to my firewall's hosts file, but it does not seem to take.

Any suggestions would be greatly appreciated.

[LinuxTracker]

Withdrawn for now.

[LinuxTracker]

@freakalad:

There have been a few recent reports re issues associated with ads on mobiles, as well putting load on batteries (hardly seems surprising), and I don't want to root my Android to block ads (if I can block ads for all nodes, all the better)

I've done this by creating an alias of the mobile ad servers and adding a rule that blocked outbound traffic to them.

However, first I had to find the addresses for the mobile ad servers.  I did that by packet capturing the IP of my mobile, while I opened the apps.