PfBlockerNG v2.0 w/DNSBL

BBcan177

I have a failover gateway group defined, is there a way to use that in pfBlockerNG? I can't seem to find the correct spot. Now it works with the main gateway but not the failover.

Do those interfaces show in the pfBNG: General Tab: Interface Options?

azzaron

Thank you for this amazing package! :D

We are experiencing unusual behavior in HTTPS redirects with DNSBL turned on…hoping someone has some ideas on this...

DNSBL works flawlessly with our blacklists to block malicious and inappropriate content for all normal browsing. However, when a user tries to download a file from an alternative domain, the filter chokes.

Here's the symptom:
A user clicks on a link in a valid (non-blacklisted, let's say abcd.tld) site to an HTTPS file located cross-domain (let's say subdomain.subdomain.dcba.tld, also not blacklisted). The page comes up with an HTTP Time Out.

We have observed this behavior in links in common apps like GMAIL and not-so-common ones hosted on Amazon AWS. It appears (although we can't confirm 100%) to only occur on redirects using an alias DNS entry (i.e. abcd.tld/file.ext = subdomain.subdomain.dcba.tld/file.ext) and it only happens in HTTPS.

Here are the steps in troubleshooting:

Turn off DNSBL (but leave pfB on) fixes the issue instantly after a Reload.
Turn on DNSBL but turn off the specific blacklists (basically no filtering active), issue still occurs (also tried with Alexa on/off).
Add redirected FQDN's to suppression list, issue still occurs.
No waiting or flushdns command needed (so it does not appear to be DNS Resolver related).

The VIP and DNSBL ports are all defaults and there are no conflicts we can detect. We have multiple LAN segments, so the Firewall Rule is selected across all segments correctly. No Advanced Inbound Firewall Rules are configured.

We are running pfBlockerNG 2.0.4 on pfSense 2.2.6. There are no conflicting installed packages (only Cron, File Manager, LCDproc-dev, and suricata are installed; suricata is not enabled).

Any thoughts are much appreciated. I cannot find a previous posting reporting this behavior.

BBcan177

Hi azzaron,

Are you getting DNSBL alerts when this occurs? Maybe run a wireshark capture and see if you can figure out the issue. Have you tried hitting "F12" in the browser and loading DEV mode? Goto "console" and see if you can get some additional clues…

shopro

@BBcan177:

@shopro:

I have a failover gateway group defined, is there a way to use that in pfBlockerNG? I can't seem to find the correct spot. Now it works with the main gateway but not the failover.

Do those interfaces show in the pfBNG: General Tab: Interface Options?

If you meant the "Inbound Firewall Rules"/"Outbound Firewall Rules" then no, the gateway group does not appear there.

azzaron

Here is an example of the log (from various times to show what's occurring):

DNSBL Reject HTTPS,Mar 04 11:00:38,s2.googleusercontent.com
DNSBL Reject HTTPS,Mar 04 11:00:38,s2.googleusercontent.com
DNSBL Reject HTTPS,Mar 04 11:01:00,lh4.googleusercontent.com
DNSBL Reject HTTPS,Mar 04 11:01:00,lh4.googleusercontent.com
DNSBL Reject,Mar 04 11:02:04,daisybill-production.s3.amazonaws.com,10.50.5.9,https://go.daisybill.com/reports/business_accounts/331/billing_providers/683/bill_exports | /exports/bills/documents/000/013/937/original/bill_exports_requested_by_(name removed)_2016-03-04-09-59-21.csv?AWSAccessKeyId=(key removed)&Expires=1457114534&Signature=(signature removed) | Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/48.0.2564.116 Safari/537.36
DNSBL Reject HTTPS,Mar 04 11:02:04,daisybill-production.s3.amazonaws.com
DNSBL Reject HTTPS,Mar 04 11:02:04,daisybill-production.s3.amazonaws.com
DNSBL Reject HTTPS,Mar 04 11:02:04,daisybill-production.s3.amazonaws.com
DNSBL Reject HTTPS,Mar 08 9:34:13,r20swj13mr.microsoft.com
DNSBL Reject HTTPS,Mar 08 9:34:13,iecvlist.microsoft.com
DNSBL Reject HTTPS,Mar 08 9:34:14,iecvlist.microsoft.com

Not only do none of those domains appear on the list we use, this behavior occurs when no lists are active (only DNSBL is turned on). We are collecting additional browser information, but the blockage seems to be happening at the pfB level according to these logs.

The specific browser error these users get is: NET::ERR_CERT_AUTHORITY_INVALID. And then browsing to that link is dead until DNSBL is turned off.

Under the Alerts tab in pfB, for example, lh3.googleusercontent.com is listed but with "no match" in the List column. Clicking the + icon next to the name results in "Domain [lh3.googleusercontent.com] does not exist in the Unbound Resolver DNSBL."

Could it be that pfB cannot detect obfuscated domains when aliases/links occur in HTTPS?

BBcan177

Did you check the pfB IP block alerts, or Snort/Suricata (if you use an IDS)?

You can try to grep from the commandline to search for domains IE:
grep "s3.amazonaws.com" /var/unbound/pfb_dnsbl.conf

There has to be a domain that is blocked that is causing your issue… You just need to isolate it.

beatstick

I have the following problems:
The list: http://hosts-file.net/download/hosts.zip blocks some website I frequent and I can't suppress it via alerts and clicking +.
Suppression works for blocks by other lists, though.
Also after setting upd pfblockerng windows 7 sometimes seems to think I need to logon to the network via web browser and shows the network not connected symbol in the task bar. The connections works fine, though.
What seems to be the problem here?

BBcan177

Grep for the domain that you can't suppress and see if you can find some other derivatives of that domain that might be blocked… See the other recommendations that I posted above...

azzaron

We did search for that domain and the others and it does not exist (sorry, should have put that in previous troubleshooting steps, but was trying to be succinct).

However, to be clear on the matter, the behavior occurs even when the pfb_dnsbl.conf is empty. Only DNSBL has to be turned on to cause the symptoms described with HTTPS.

BBcan177

Clear your browser and OS cache… If you use chrome, you can use this link:
chrome://net-internals/#dns

azzaron

So here's how we resolved the issue with HTTPS redirects:

Matched up the domain with its alias (i.e. nslookup lh4.googleusercontent.com = googlehosted.l.googleusercontent.com)
Added the actual domain to the suppression list (i.e. googlehosted.l.googleusercontent.com)

It is odd that even with a totally blank conf, this behavior occurs. This is somehow related to browser security and DNSBL not parsing HTTPS aliases…maybe it's a limitation due to that security? Or maybe it's a feature by design, as it was clearly written by someone smarter than me! 8)

Is it possible to add an advanced option that bypasses HTTPS aliases when they can't be matched to a rule? Or are we overthinking it?

Many thanks once again for a stupendous package. Donating to the cause today.

BBcan177

Glad you figured it out… the "googlehosted.l.googleusercontent.com" is actually a cname... So not sure if there is a workaround for this issue...

drill lh4.googleusercontent.com

;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 60959
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; lh4.googleusercontent.com.   IN      A

;; ANSWER SECTION:
lh4.googleusercontent.com.      86399   IN      CNAME   googlehosted.l.googleusercontent.com.
googlehosted.l.googleusercontent.com.   300     IN      A       216.58.216.193

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 710 msec
;; SERVER: 127.0.0.1
;; WHEN: Tue Mar  8 18:04:25 2016
;; MSG SIZE  rcvd: 88

I also don't see which blocklist has this domain listed?
grep "googlehosted.l.googleusercontent.com" /var/unbound/pfb_dnsbl.conf
grep "lh4.googleusercontent.com" /var/unbound/pfb_dnsbl.conf

edit:

After further thought, I guess I could add some code to the Alerts Tab - DNSBL Suppression code… So if there is no match for a domain in DNSBL, i guess I could run the drill command and see if it returns any cnames, and then attempt to suppress both domains at the same time... Will add that to the todo list...

azzaron

To be completely fair, the interface of DNSBL Configuration does state:

Note: DNSBL will block and partially log Alerts for HTTPS requests. To debug issues…

We did find some of the CNAMES being choked in the squidblacklist.org's Malicious list. This doesn't explain why a blank conf file produced similar results. But, the suppression list works and we move on. :)

Nachtfalke

Hi,

I did not get it work properly with squid and pfblockerng. Squid is running in non-transperent mode and it seems to make problems with the redirect to the :8081 and :8443 ports.
This is probably because the pfblockerNG NAT rules are on the "LAN" interface but the squid initiated traffic is not on LAN but on localhost (127.0.0.1). I don't know exactly how it works and where the problem is.

But what I found out is the reason why squid blocked access to https with port 8443. I had to add port 844 to the squid "ACL SSLPorts" on the ACL tab in squid so that squid accepts https traffic on this port.

So at the moment I would say that pfblockerng is is blocking on http but redirects to pfsense webui and it does not log ist.
For https I did not found a website or ist does not work.

So if anyone has any additional ideas please let me know hot to get squid and pfblockerng working.

BBcan177

@azzaron:

To be completely fair, the interface of DNSBL Configuration does state:

Note: DNSBL will block and partially log Alerts for HTTPS requests. To debug issues…

We did find some of the CNAMES being choked in the squidblacklist.org's Malicious list. This doesn't explain why a blank conf file produced similar results. But, the suppression list works and we move on. :)

Yes I know the text your mentioning but its not really related… DNSBL is not breaking the HTTPS by MITMing the connection. So for HTTPS, the only details that DNSBL can capture is the Event timestamp and the Domain name... All of the other details (Interface and URL etc...) can't be captured currently..

BTW: I have a test box with a CNAME lookup to fix this issue for the next release... so Thanks for bring it to my attention... ;)

BBcan177

@Nachtfalke:

I did not get it work properly with squid and pfblockerng.

So if anyone has any additional ideas please let me know hot to get squid and pfblockerng working.

Hi Nachtfalke,

Sorry but I don't use Squid and can't help much there… There are several people who use both packages so I am surprised that no one is responding to help you out...

There's a great community here and hopefully we can nudge an answer out for you :)

Panja

I have setup DNSBL EasyList but when browsing to YouTube I'm getting an invalid certificate error.
ad.doubleclick.net: root certificate is not trusted.

How can I prevent this?

BBcan177

@Panja:

I have setup DNSBL EasyList but when browsing to YouTube I'm getting an invalid certificate error.
ad.doubleclick.net: root certificate is not trusted.

What browser are you using? Is it up-to-date?

What URL are you using for Youtube that reports that message? Or is this in a Youtube App?

Panja

I tried Safari on OSX and Internet Explorer (11) on Windows 10.
They are up to date.

https://www.youtube.com

See attached image.

![Screen Shot 2016-03-13 at 20.41.26.png_thumb](/public/imported_attachments/1/Screen Shot 2016-03-13 at 20.41.26.png_thumb)
![Screen Shot 2016-03-13 at 20.41.26.png](/public/imported_attachments/1/Screen Shot 2016-03-13 at 20.41.26.png)

BBcan177

@Panja:

I tried Safari on OSX and Internet Explorer (11) on Windows 10.

Chrome and FF do not have this issue, as they silently drop those connections to a non-secure site. I suspect over time that Safari and IE (didn't test Edge) will get their act in gear … Not much I can do to fix that issue...