PfBlockerNG v2.0 w/DNSBL

BBcan177 · Mar 8, 2016, 5:25 PM

Grep for the domain that you can't suppress and see if you can find some other derivatives of that domain that might be blocked… See the other recommendations that I posted above...

azzaron · Mar 8, 2016, 5:27 PM

We did search for that domain and the others and it does not exist (sorry, should have put that in previous troubleshooting steps, but was trying to be succinct).

However, to be clear on the matter, the behavior occurs even when the pfb_dnsbl.conf is empty. Only DNSBL has to be turned on to cause the symptoms described with HTTPS.

BBcan177 · Mar 8, 2016, 5:28 PM

Clear your browser and OS cache… If you use chrome, you can use this link:
chrome://net-internals/#dns

azzaron · Mar 8, 2016, 6:42 PM

So here's how we resolved the issue with HTTPS redirects:

Matched up the domain with its alias (i.e. nslookup lh4.googleusercontent.com = googlehosted.l.googleusercontent.com)
Added the actual domain to the suppression list (i.e. googlehosted.l.googleusercontent.com)

It is odd that even with a totally blank conf, this behavior occurs. This is somehow related to browser security and DNSBL not parsing HTTPS aliases…maybe it's a limitation due to that security? Or maybe it's a feature by design, as it was clearly written by someone smarter than me! 8)

Is it possible to add an advanced option that bypasses HTTPS aliases when they can't be matched to a rule? Or are we overthinking it?

Many thanks once again for a stupendous package. Donating to the cause today.

BBcan177 · Mar 8, 2016, 11:33 PM

Glad you figured it out… the "googlehosted.l.googleusercontent.com" is actually a cname... So not sure if there is a workaround for this issue...

drill lh4.googleusercontent.com

;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 60959
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; lh4.googleusercontent.com.   IN      A

;; ANSWER SECTION:
lh4.googleusercontent.com.      86399   IN      CNAME   googlehosted.l.googleusercontent.com.
googlehosted.l.googleusercontent.com.   300     IN      A       216.58.216.193

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 710 msec
;; SERVER: 127.0.0.1
;; WHEN: Tue Mar  8 18:04:25 2016
;; MSG SIZE  rcvd: 88

I also don't see which blocklist has this domain listed?
grep "googlehosted.l.googleusercontent.com" /var/unbound/pfb_dnsbl.conf
grep "lh4.googleusercontent.com" /var/unbound/pfb_dnsbl.conf

edit:

After further thought, I guess I could add some code to the Alerts Tab - DNSBL Suppression code… So if there is no match for a domain in DNSBL, i guess I could run the drill command and see if it returns any cnames, and then attempt to suppress both domains at the same time... Will add that to the todo list...

azzaron · Mar 8, 2016, 11:41 PM

To be completely fair, the interface of DNSBL Configuration does state:

Note: DNSBL will block and partially log Alerts for HTTPS requests. To debug issues…

We did find some of the CNAMES being choked in the squidblacklist.org's Malicious list. This doesn't explain why a blank conf file produced similar results. But, the suppression list works and we move on. :)

Nachtfalke · Mar 9, 2016, 6:12 PM

Hi,

I did not get it work properly with squid and pfblockerng. Squid is running in non-transperent mode and it seems to make problems with the redirect to the :8081 and :8443 ports.
This is probably because the pfblockerNG NAT rules are on the "LAN" interface but the squid initiated traffic is not on LAN but on localhost (127.0.0.1). I don't know exactly how it works and where the problem is.

But what I found out is the reason why squid blocked access to https with port 8443. I had to add port 844 to the squid "ACL SSLPorts" on the ACL tab in squid so that squid accepts https traffic on this port.

So at the moment I would say that pfblockerng is is blocking on http but redirects to pfsense webui and it does not log ist.
For https I did not found a website or ist does not work.

So if anyone has any additional ideas please let me know hot to get squid and pfblockerng working.

BBcan177 · Mar 10, 2016, 12:05 AM

@azzaron:

To be completely fair, the interface of DNSBL Configuration does state:

Note: DNSBL will block and partially log Alerts for HTTPS requests. To debug issues…

We did find some of the CNAMES being choked in the squidblacklist.org's Malicious list. This doesn't explain why a blank conf file produced similar results. But, the suppression list works and we move on. :)

Yes I know the text your mentioning but its not really related… DNSBL is not breaking the HTTPS by MITMing the connection. So for HTTPS, the only details that DNSBL can capture is the Event timestamp and the Domain name... All of the other details (Interface and URL etc...) can't be captured currently..

BTW: I have a test box with a CNAME lookup to fix this issue for the next release... so Thanks for bring it to my attention... ;)

BBcan177 · Mar 10, 2016, 12:06 AM

@Nachtfalke:

I did not get it work properly with squid and pfblockerng.

So if anyone has any additional ideas please let me know hot to get squid and pfblockerng working.

Hi Nachtfalke,

Sorry but I don't use Squid and can't help much there… There are several people who use both packages so I am surprised that no one is responding to help you out...

There's a great community here and hopefully we can nudge an answer out for you :)

Panja · Mar 13, 2016, 7:03 PM

I have setup DNSBL EasyList but when browsing to YouTube I'm getting an invalid certificate error.
ad.doubleclick.net: root certificate is not trusted.

How can I prevent this?

BBcan177 · Mar 13, 2016, 7:17 PM

@Panja:

I have setup DNSBL EasyList but when browsing to YouTube I'm getting an invalid certificate error.
ad.doubleclick.net: root certificate is not trusted.

What browser are you using? Is it up-to-date?

What URL are you using for Youtube that reports that message? Or is this in a Youtube App?

Panja · Mar 13, 2016, 7:40 PM

I tried Safari on OSX and Internet Explorer (11) on Windows 10.
They are up to date.

https://www.youtube.com

See attached image.

![Screen Shot 2016-03-13 at 20.41.26.png_thumb](/public/imported_attachments/1/Screen Shot 2016-03-13 at 20.41.26.png_thumb)
![Screen Shot 2016-03-13 at 20.41.26.png](/public/imported_attachments/1/Screen Shot 2016-03-13 at 20.41.26.png)

BBcan177 · Mar 13, 2016, 8:21 PM

@Panja:

I tried Safari on OSX and Internet Explorer (11) on Windows 10.

Chrome and FF do not have this issue, as they silently drop those connections to a non-secure site. I suspect over time that Safari and IE (didn't test Edge) will get their act in gear … Not much I can do to fix that issue...

Panja · Mar 13, 2016, 8:48 PM

Edge is having the same issue. But thanks for clearing that out.

That's probably the reason why I didn't notice this problem before.
I've been using Chrome for ages but recently I re-installed my GF's laptop with Win10 and she uses IE.

BBcan177 · Mar 18, 2016, 4:17 AM

Recent Malvertising Campaign hits several Top Websites… bbc.com, msn.com, nfl.com, aol.com, answers.com:

https://blog.malwarebytes.org/malvertising-2/2016/03/large-angler-malvertising-campaign-hits-top-publishers/
http://blog.trendmicro.com/trendlabs-security-intelligence/malvertising-campaign-in-us-leads-to-angler-exploit-kitbedep/
https://www.trustwave.com/Resources/SpiderLabs-Blog/Angler-Takes-Malvertising-to-New-Heights/

These malicious domains are not currently listed by the DNSBL blocklists in use.

I have added them to my DNSBL Gist:
https://gist.githubusercontent.com/BBcan177/4a8bf37c131be4803cb2/raw

Ramosel · Mar 18, 2016, 8:27 PM

@BBcan177:

These malicious domains are not currently listed by the DNSBL blocklists in use.

I have added them to my DNSBL Gist:
https://gist.githubusercontent.com/BBcan177/4a8bf37c131be4803cb2/raw

Too cool, thank you!

Rick

varazir · Mar 20, 2016, 6:31 AM

Hello,

Only me who has crash problem on my pfSense?

PHP Errors:
[20-Mar-2016 07:15:00 Europe/Stockholm] PHP Fatal error:  Maximum execution time of 900 seconds exceeded in /usr/local/pkg/pfblockerng/pfblockerng.inc on line 902

BBcan177 · Mar 20, 2016, 2:36 PM

@varazir, are you on the latest version of pfBlockerNG? If not, please update and see if that fixes your issue.

varazir · Mar 20, 2016, 3:01 PM

@BBcan177:

@varazir, are you on the latest version of pfBlockerNG? If not, please update and see if that fixes your issue.

I don't have any newer in the System: Package Manager, 2.0.4

BBcan177 · Mar 28, 2016, 6:56 PM

@varazir:

@BBcan177:

@varazir, are you on the latest version of pfBlockerNG? If not, please update and see if that fixes your issue.

I don't have any newer in the System: Package Manager, 2.0.4

Line 902 has code for the Alexa database conversion… If this only happened once, then discard it, but if its happening more often please provide some additional details on your hardware.

Do you see these two files:

ls /var/db/pfblockerng/top*

/var/db/pfblockerng/top-1m.csv

Can you open the top-1m.csv file?

When you run this command, it will show how many Alexa TLDs are being used… The count should match the Alexa count that you defined in the DNSBL tab (Number of Alexa Top Domains to Whitelisting):

wc -l /var/db/pfblockerng/pfbalexawhitelist.txt

You can also review the error.log file, to see if the Alexa Database is failing…