PfBlockerNG

jawz101 · Jan 5, 2016, 2:25 PM

Converting MaxMind Country databases for pfBlockerNG.
This may take a few minutes…

I just did fresh a install of pfSense after using the box for other stuff for a while and the converting the database process just hangs. It didn't a few months ago. I was running 2.2.5 the last time I had installed pfB so that and whatever has changed with the addon since a few months is really the only things that have changed in the process.

This is a fast 50+MB connection and an Intel c2758 board so I can't think it's the equipment. Logs aren't showing anything funky. I've removed and reinstalled pfB a few times to see if something would change but the CPU is idle at this point and logs aren't indicating anything's going on at this point of the install. I let it go about 20+ minutes on one attempt and just now let it sit there while I slept 7+ hours.

I can clearly see the lists have downloaded to the box via ssh and I even looked at the github source and downloaded the country lists from the url's via my browser.

edit- just redownloaded the pfSense iso, verified hash, reinstalled pfSense and still get the same hang

jawz101 · Jan 6, 2016, 1:02 AM

@BBCan17
I've since reinstalled Firefox and started walking through all of my addons setting them back up how I had them. So far I haven't seen it not install now. Maybe either successfully installing it once from another browser is what fixed it or I haven't checked a setting in my add-ons that was causing the error or maybe something in my browser cache… no idea. Kinda frustrating I haven't pinned it down but at least it wasn't a big deal I guess. If something comes up I'll let you know because that's really odd.

Thanks for the help!

Also, I'm thinking of writing a tut at some point but I doubt it'll be concise as some would wish. It's one of those apps I want to gush on about. Besides, after using blockers for 10+ years they're handy but using >10 blocklists seems excessive prefer being conservative.

DLFerguRD · Jan 6, 2016, 6:07 PM

@BBcan177

Hi I have just upgrqaded to pfsense 2.3B and am having a problem with the pfBlockerNG webserver service.
The service will not start for some reason. I do get a crash report from pfsense.

					Crash report begins.  Anonymous machine information:

amd64
10.2-STABLE
FreeBSD 10.2-STABLE #296 d8ff348(devel): Wed Jan  6 08:09:19 CST 2016     root@pfs23-amd64-builder:/usr/home/pfsense/pfsense/tmp/obj/usr/home/pfsense/pfsense/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[06-Jan-2016 10:10:32 America/Edmonton] PHP Stack trace:
[06-Jan-2016 10:10:32 America/Edmonton] PHP   1\. {main}() /usr/local/www/pfblockerng/pfblockerng.php:0
[06-Jan-2016 10:10:32 America/Edmonton] PHP   2\. sync_package_pfblockerng() /usr/local/www/pfblockerng/pfblockerng.php:101
[06-Jan-2016 10:10:32 America/Edmonton] PHP   3\. pfb_create_dnsbl() /usr/local/pkg/pfblockerng/pfblockerng.inc:3031
[06-Jan-2016 10:10:32 America/Edmonton] PHP   4\. link() /usr/local/pkg/pfblockerng/pfblockerng.inc:693
[06-Jan-2016 10:15:00 America/Edmonton] PHP Stack trace:
[06-Jan-2016 10:15:00 America/Edmonton] PHP   1\. {main}() /usr/local/www/pfblockerng/pfblockerng.php:0
[06-Jan-2016 10:15:00 America/Edmonton] PHP   2\. pfblockerng_sync_cron() /usr/local/www/pfblockerng/pfblockerng.php:94
[06-Jan-2016 10:15:00 America/Edmonton] PHP   3\. sync_package_pfblockerng() /usr/local/www/pfblockerng/pfblockerng.php:387
[06-Jan-2016 10:15:00 America/Edmonton] PHP   4\. pfb_create_dnsbl() /usr/local/pkg/pfblockerng/pfblockerng.inc:3031
[06-Jan-2016 10:15:00 America/Edmonton] PHP   5\. link() /usr/local/pkg/pfblockerng/pfblockerng.inc:693

I have tried reinstalling, removing the package completely and then installing.

It would be great if you could have a look at this.
BTW Thanks for a great package.

Dan

ddimick · Jan 7, 2016, 4:45 PM

pfSense moved from lighttpd to nginx in the beta which broke pfBNG (along with all other packages relying on lighttpd). I believe the author is working on an update.

https://forum.pfsense.org/index.php?topic=104854.0

DLFerguRD · Jan 7, 2016, 8:22 PM

Ok Thank you.
Since the package was available for 2.3 I thought it was already done.

varazir · Jan 8, 2016, 9:00 PM

I'm getting this error in the update log

===[  DNSBL Process  ]================================================
Missing DNSBL stats and/or Unbound DNSBL conf file - Rebuilding

BBcan177 · Jan 8, 2016, 10:11 PM

Hi trumee,

Some of those download fails are due to cURL errors. See the messages below. Set those to "Flex".

@trumee:

SSL: no alternative certificate subject name matches target host name 'feeds.dshield.org'
SSL certificate problem: unable to get local issuer certificate Retry in 5 seconds…

Also Dangerulez and VMX are being blocked by other Firewall Lists.

[ pfB_PRI3 - DangerRulez ] Download FAIL [ 01/03/16 20:11:47 ]
[ 188.40.39.38 ] Firewall IP block found in: [ pfB_Top_v4 | 188.40.0.0/16 ]

[ pfB_PRI3 - VMX ] Download FAIL [ 01/03/16 20:12:04 ]
[ 195.209.40.11 ] Firewall IP block found in: [ pfB_Top_v4 | 195.208.0.0/15 ]

BBcan177 · Jan 8, 2016, 10:13 PM

@DLFerguRD:

Hi I have just upgrqaded to pfsense 2.3B and am having a problem with the pfBlockerNG webserver service. The service will not start for some reason. I do get a crash report from pfsense.

@DougD:

pfSense moved from lighttpd to nginx in the beta which broke pfBNG (along with all other packages relying on lighttpd). I believe the author is working on an update.

https://forum.pfsense.org/index.php?topic=104854.0

Yes the recent changes in pfense 2.3 from Lighttpd to NGINX broke the DNSBL feature… I will be submitting a PR to fix this up soon....

BBcan177 · Jan 8, 2016, 10:15 PM

@varazir:

I'm getting this error in the update log

===[  DNSBL Process  ]================================================
Missing DNSBL stats and/or Unbound DNSBL conf file - Rebuilding

Its not an error… Its just a notice to say that its rebuilding the database. Possibly due to a "Force Reload" or re-enabling pfBlockerNG... or adding/removing a DNSBL Alias/Feeds...

abujammy · Jan 9, 2016, 1:26 AM

@BBcan177, this is going to take some explaining. :) This feature affects other services, but let's just use AWS as an example.

AWS controls (among many, many others) the entire 54.240.128.0/18 range, which means that all /24 ranges from 54.240.128.0/24-54.240.191.0/24 are included. Since AWS allows others to use their servers, it's highly likely that many of those /24 ranges are going to be flagged.

Let's say that 128, 129, 140, 141, 142, 143, 144 get flagged in a particular /24. As far as AWS is concerned, there's no correlation which implies that 130, 139 or 145 are the same individual or group. This is especially true for CloudFront where the IP assignments mean nothing.

So the entire /24 net gets blacklisted when, in actuality, for these particular subnets, the relationships between IPs on the same /24 subnet is almost zero.

What I'd like would be for a way to say: "Hey, go ahead and parse out something like this: https://ip-ranges.amazonaws.com/ip-ranges.json and, if the IPs are within those ranges, still blacklist them, but don't add them to the list for reputation processing."

Thoughts?

BBcan177 · Jan 9, 2016, 10:01 PM

@abujammy:

What I'd like would be for a way to say: "Hey, go ahead and parse out something like this: https://ip-ranges.amazonaws.com/ip-ranges.json and, if the IPs are within those ranges, still blacklist them, but don't add them to the list for reputation processing."

Hi abujammy, Its best to use the Country code exclusion for Reputation and add "USA" to the list of exclusions.

tommyboy180 · Jan 13, 2016, 12:39 AM

Wow! BBcan177. Wow.

I've been gone for such a long time, I come back and pfBlockerNG is a hit! This is so cool. I'm so glad that you stepped up and helped move this idea. Thanks to Marcello Coutinho for helping me get pfBlocker (back when it was IP blocklist and country Block) off the ground from it's first inception back when it was release 0.1.

This is absolutely awesome. And you gave throwback credit to me and Marcello Coutinho.

Well done! This is why I love the pfsense community. We all work together to make our worlds better.

BBcan177 · Jan 14, 2016, 1:11 AM

@tommyboy180:

Wow! BBcan177. Wow.

I've been gone for such a long time, I come back and pfBlockerNG is a hit! This is so cool. I'm so glad that you stepped up and helped move this idea. Thanks to Marcello Coutinho for helping me get pfBlocker (back when it was IP blocklist and country Block) off the ground from it's first inception back when it was release 0.1.

This is absolutely awesome. And you gave throwback credit to me and Marcello Coutinho.

Well done! This is why I love the pfsense community. We all work together to make our worlds better.

Thanks… Most don't realize all the effort that goes into developing and maintaining a package with an ever changing landscape... ;) welcome back...

peppegate · Jan 14, 2016, 3:23 PM

Hello,
what did you prefere as blocked iplist?
did you have some web service that you know where i can buy access to some blocklist that i can insert by URL and forget it?
did you know also some of this service for Whitelist (like googlebot and similar)
tnx for your help.

Peppe

AmosK · Jan 14, 2016, 4:52 PM

Is it possible to block a list of dynamic IP's but only specific ports?

We get a lot of spam H.323 calls originating from AWS (185+ EC2 IP ranges) and DigitalOcean (10+ IP ranges) and others. We'd like to automate the blacklisting but only on port's 1503 and 1720.

Expert Mode Question: Most of the originator calls are named "unknown" or "cisco" and it would be great to parse those out and block those, though I'm betting that's beyond the scope of pfBlockerNG.

BBcan177 · Jan 17, 2016, 4:03 PM

@peppegate:

Hello,
what did you prefere as blocked iplist?
did you have some web service that you know where i can buy access to some blocklist that i can insert by URL and forget it?
did you know also some of this service for Whitelist (like googlebot and similar)
tnx for your help.

Peppe

Here is link to a script to import ~50 blocklists…
https://forum.pfsense.org/index.php?topic=86212.msg549973#msg549973

I think there are some Whitelists available... But I don't have those URLs myself... Google search and/or hopefully other users can share some URLs for whitelisting...

BBcan177 · Jan 17, 2016, 4:06 PM

@AmosK:

Is it possible to block a list of dynamic IP's but only specific ports?

We get a lot of spam H.323 calls originating from AWS (185+ EC2 IP ranges) and DigitalOcean (10+ IP ranges) and others. We'd like to automate the blacklisting but only on port's 1503 and 1720.

Expert Mode Question: Most of the originator calls are named "unknown" or "cisco" and it would be great to parse those out and block those, though I'm betting that's beyond the scope of pfBlockerNG.

You can use the "Adv. Inbound Firewall" rule settings to protect only certain ports/Dest IPs etc… I will be adding "Adv. Outbound Firewall" rule settings in the next release. You can also use "Alias" type rules, and manually create the firewall rules to suit more complex setups...

ToddCsense · Jan 19, 2016, 5:53 AM

pfBlockcerNG IPv4-lists: Changes from script: as of 2016-01-18

Re: https://forum.pfsense.org/index.php?topic=86212.msg510369;topicseen#msg510369

After perusing the log from the CRON run for pfBlockerNG_import.php script as listed here, it appears that there are a couple changes that might reduce errors that I have documented below.

Might want to delete the "no access," "discontinued" and "not found" or change status to "off."

Cheers,

Todd

*Startlist

Abuse_Spyeye
from: https://spyeyetracker.abuse.ch/blocklist.php?download=ipblocklist
to: discontinued

as per: https://spyeyetracker.abuse.ch/ :

SpyEye Tracker has been disconntinued. More information will follow soon on https://www.abuse.ch

Thanks for all your support!

Snort_BL
from: https://labs.snort.org/feeds/ip-filter.blf
to: http://talosintel.com/feeds/ip-filter.blf

as per: http://blog.snort.org/2015/09/ip-blacklist-feed-has-moved-locations.html :
Wednesday, September 2, 2015
IP Blacklist feed has moved locations!
For those of you using the IP Blacklist feed on labs.snort.org, we've had to move the URL to the new link.
You can find it at the following URL: http://talosintel.com/feeds/ip-filter.blf

Infiltrated
from: http://www.infiltrated.net/blacklisted
to: no access

as per: http://www.infiltrated.net/blacklisted
Forbidden
You don't have permission to access /blacklisted on this server.

Geospy
from: http://www.geopsy.org/blacklist.html
to: not found

as per: http://www.geopsy.org/blacklist.html
Not Found
The requested URL /blacklist.html was not found on this server.

*Endlist

DJProject · Feb 24, 2016, 11:23 AM

Is it possible to import "Easylist Germany" to "DNSBL Easylist" from AdBlock Plus into pfBlockerNG?

Easylist Germany: https://easylist.adblockplus.org/de/

oddworld19 · Mar 1, 2016, 3:51 PM

@BBCan17
I am having trouble viewing the pfblockerNG alerts.

On the main index page, I can see (for example) X packets for "pfB_NAmericav4" (I created an inbound rule/alias to block everything inbound on WAN that's not from United States).

I can see that it blocked 475 packets.

Clicking on it bring me to the alerts page, which says "Found 29 Alert Entries - Insufficient Firewall Alerts found."

Why does it only find 29 of the 475 blocked packets?

I've disabled remote logging for the moment to hopefully eliminate a variable. I've also
-disenable/re-enable pfbng
-force update
-force cron
-force reload
-clear logs (from firewall settings tab)

pics below