PfBlockerNG v2.0 w/DNSBL

RonpfS

I'm running the latest pfBlockerNG package on pfSense 2.3, and I have the same issue with Safari on OS X and iOS complaining about invalid certificates from googleads.g.doubleclick.net. The latest events were from a banking site and when reading an article on macgroup.org.

I've confirmed that I can ping to the VIP, and a browser sent to the VIP:8081 gets a 1x1 favicon. I've white listed googleads.g.doubleclick.net.

Is there anything else I can do in the pfBlockerNG settings to sort this out?

I'd send a bug report to Apple on this, but I don't understand the problem nearly well enough to know how to properly describe the issue.

When you whitelist using the Alerts Tab, pfBlockerNG will identify if there are CNAMEs for the domain name and whitelist the CNAME as well.

However when you put a domainname in the Custom Domain Whitelist, you have to find the CNAMEs on your own:

drill @8.8.8.8 googleads.g.doubleclick.net

;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 19703
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; googleads.g.doubleclick.net.	IN	A

;; ANSWER SECTION:
googleads.g.doubleclick.net.	299	IN	CNAME	pagead46.l.doubleclick.net.
pagead46.l.doubleclick.net.	299	IN	A	172.217.1.98

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 74 msec
;; SERVER: 8.8.8.8
;; WHEN: Wed Aug 31 15:25:50 2016
;; MSG SIZE  rcvd: 86

Then you add pagead46.l.doubleclick.net to the Custom Domain Whitelist

khorton

@RonpfS:

@khorton:

I'm running the latest pfBlockerNG package on pfSense 2.3, and I have the same issue with Safari on OS X and iOS complaining about invalid certificates from googleads.g.doubleclick.net. The latest events were from a banking site and when reading an article on macgroup.org.

I've confirmed that I can ping to the VIP, and a browser sent to the VIP:8081 gets a 1x1 favicon. I've white listed googleads.g.doubleclick.net.

Is there anything else I can do in the pfBlockerNG settings to sort this out?

I'd send a bug report to Apple on this, but I don't understand the problem nearly well enough to know how to properly describe the issue.

When you whitelist using the Alerts Tab, pfBlockerNG will identify if there are CNAMEs for the domain name and whitelist the CNAME as well.

However when you put a domainname in the Custom Domain Whitelist, you have to find the CNAMEs on your own:
drill @8.8.8.8 googleads.g.doubleclick.net

;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 19703
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; googleads.g.doubleclick.net.	IN	A

;; ANSWER SECTION:
googleads.g.doubleclick.net.	299	IN	CNAME	pagead46.l.doubleclick.net.
pagead46.l.doubleclick.net.	299	IN	A	172.217.1.98

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 74 msec
;; SERVER: 8.8.8.8
;; WHEN: Wed Aug 31 15:25:50 2016
;; MSG SIZE  rcvd: 86
Then you add pagead46.l.doubleclick.net to the Custom Domain Whitelist

Thanks for the info. I've added pagead46.l.doubleclick.net to the Custom Domain Whitelist. We'll see if that helps.

But, I already had .doubleclick.net in the Custom Domain Whitelist. I thought the "." at the start was supposed to cover all the sub-domains. Does that not work? Or, is this a hint that we're chasing the wrong fix to the problem?

BBcan177

@khorton:

But, I already had .doubleclick.net in the Custom Domain Whitelist. I thought the "." at the start was supposed to cover all the sub-domains. Does that not work? Or, is this a hint that we're chasing the wrong fix to the problem?

If you had the prefixed dot, then it should have cleared all of those Domains…

You can check with the following cmd and see if there are any Domains ending with doubleclick.net in the DNSBL blocklist.

grep "doubleclick.net" /var/unbound/pfb_dnsbl.conf

You can also go into Dev mode in the Browser (then goto console), then see if it could be a different Domain that is causing this issue…

khorton

@BBcan177:

If you had the prefixed dot, then it should have cleared all of those Domains…

You can check with the following cmd and see if there are any Domains ending with doubleclick.net in the DNSBL blocklist.
grep "doubleclick.net" /var/unbound/pfb_dnsbl.conf
You can also go into Dev mode in the Browser (then goto console), then see if it could be a different Domain that is causing this issue…

It looks like there was quite a long list of doubleclick.net domains already caught:

grep "doubleclick.net" /var/unbound/pfb_dnsbl.conf
local-data: "ad-apac.doubleclick.net 60 IN A 10.10.10.1"
local-data: "ad-emea.doubleclick.net 60 IN A 10.10.10.1"
local-data: "ad.doubleclick.net 60 IN A 10.10.10.1"
local-data: "ad.mo.doubleclick.net 60 IN A 10.10.10.1"
local-data: "doubleclick.net 60 IN A 10.10.10.1"
local-data: "gan.doubleclick.net 60 IN A 10.10.10.1"
local-data: "googleads.g.doubleclick.net 60 IN A 10.10.10.1"
local-data: "iv.doubleclick.net 60 IN A 10.10.10.1"
local-data: "n4403ad.doubleclick.net 60 IN A 10.10.10.1"
local-data: "pubads.g.doubleclick.net 60 IN A 10.10.10.1"

I am unable to trigger the SSL certificate error now, but next time it happens I'll check the Safari error console in Dev mode.

BBcan177

@khorton:

It looks like there was quite a long list of doubleclick.net domains already caught:

If you manually add any domains to the DNSBL Whitelist, you must execute a "Force Reload - DNSBL" for the whitelist to take effect.

If you added .doubleclick.net to the DNSBL Whitelist, then the grep command above should have returned no results, since that Domain is in the DNSBL whitelist…

khorton

@BBcan177:

@khorton:

It looks like there was quite a long list of doubleclick.net domains already caught:

If you manually add any domains to the DNSBL Whitelist, you must execute a "Force Reload - DNSBL" for the whitelist to take effect.

Is this the "Reload" selection on the Update page? I was running "Update" after making any changes.

If you added .doubleclick.net to the DNSBL Whitelist, then the grep command above should have returned no results, since that Domain is in the DNSBL whitelist…

By "DNSBL whitelist", do you mean the "Custom Domain Whitelist" on the DNSBL page? ".doubleclick.net" (without the quotes) is the first line on that list. Or should I be looking a different whitelist?

BBcan177

@khorton:

Is this the "Reload" selection on the Update page? I was running "Update" after making any changes.

Yes the Update tab has "Update|Cron|Reload" buttons… For Reload you can further select a reload of IP/DNSBL or All...

Force Update will not "Reload" any changes that you made... Without running a "Force Reload", a previously downloaded Feed will not use the new settings, until Cron runs, and its re-downloaded... So thats the reason for the "Force Reload" button.

Click on the blue "i" infoblock Icons on the pages… It will show additional help text...

By "DNSBL whitelist", do you mean the "Custom Domain Whitelist" on the DNSBL page? ".doubleclick.net" (without the quotes) is the first line on that list. Or should I be looking a different whitelist?

Yes, DNSBL Custom Whitelist.

khorton

@BBcan177:

@khorton:

Is this the "Reload" selection on the Update page? I was running "Update" after making any changes.

Yes the Update tab has "Update|Cron|Reload" buttons… For Reload you can further select a reload of IP/DNSBL or All...

Force Update will not "Reload" any changes that you made... Without running a "Force Reload", a previously downloaded Feed will not use the new settings, until Cron runs, and its re-downloaded... So thats the reason for the "Force Reload" button.

Click on the blue "i" infoblock Icons on the pages… It will show additional help text...

By "DNSBL whitelist", do you mean the "Custom Domain Whitelist" on the DNSBL page? ".doubleclick.net" (without the quotes) is the first line on that list. Or should I be looking a different whitelist?

Yes, DNSBL Custom Whitelist.

Thanks for the clarification. I had previously read the info triggered by the blue "i"s. It makes perfect sense now that I know how the option works. I had assumed, incorrectly, that the lists would be reloaded after an update.

Now, when I run 'grep "doubleclick.net" /var/unbound/pfb_dnsbl.conf', I get a null result, apparently as expected. And, happily, my wife hasn't complained about Safari barfing up SSL certificate errors for over 24 hours.

RonpfS

@reason:

When running the force reload dnsbl, I noticed some of the TLD's, that I enter in the custom whitelist, were listed but not the exact subdomains that should have been whitelisted by using (for example) .facebook.com. I wanted mail.facebook.com to be whitelisted but it whitelisted ads.facebook.

~~If you put .FQDN it will whitelist the whole .FQDN domain,~~ If you put .facebook.com it will whitelist the whole .facebook.com domain, mail.facebook.com, ads.facebook.com, what.ever.they.use.facebook.com, etc.

@reason:

Newegg was not even listed.

So it looks like it searching the other DNSBL lists for those hostnames but not finding them so it's not exclusively whitelisting them.

Do I need to enter the fqdn as listed in the alerts?

Look at the Alerts Tab DNSBL section and hover over the "+" icon, it will tell you why the blocks happens.
You can also click on it to Whitelist it from the alerts tabs. When the whitelisting happens, it will comes back with what actions it tooks. Sometimes it will also whitelist CNAME of the FQDN. Then have a look at the Custom Domain Whitelist to see what was done.

There is also a legend infoblock at the bottom of the page.

maverik1

@BBcan177:

Here are some basic instructions to get started with DNSBL.

Open the pfBNG "DNSBL" Tab:

(Use the defaults unless you have a need to use otherwise)
Enter the DNSBL VIP as 10.10.10.1
Enter the DNSBL Listening Port as 8081
Enter the DNSBL SSL Listening port as 8443
Select the DNSBL Listening Interface as Lan

I have vlans configured on pfsense. Therefore there is no LAN option. The my only options are the different vlans, so what can I choose for the listening interface?

BBcan177

@maverik1:

@BBcan177:

Here are some basic instructions to get started with DNSBL.

Open the pfBNG "DNSBL" Tab:

(Use the defaults unless you have a need to use otherwise)
Enter the DNSBL VIP as 10.10.10.1
Enter the DNSBL Listening Port as 8081
Enter the DNSBL SSL Listening port as 8443
Select the DNSBL Listening Interface as Lan

I have vlans configured on pfsense. Therefore there is no LAN option. The my only options are the different vlans, so what can I choose for the listening interface?

Hi maverik1,

I haven't tested that before, but I assume that it should be ok to choose any one of the VLANS…

After enabling DNSBL, run this command from the shell to see if the DNSBL VIP is assigned to the VLAN:

ifconfig

You should see a line like this in the VLAN:

inet 10.10.10.1 netmask 0xffffffff broadcast 10.10.10.1

Then try to ping the DNSBL VIP address and see if it responds…

Since you have multiple VLANs, its a good idea to enable the DNSBL Permit rule option....

q54e3w

I can confirm pfbNG works fine with VLANs.

robatwork

OK here's a strange occurrence…
pfSense just upgraded to 2.3.2 but this behaviour happened with 2.2.1 on a physically different router.

pfb version 2.1.1_4.

On our LAN when we use google chrome (latest) and go to google it can take several seconds for google to display. Doing a search can take 20 seconds to respond, but it does come back eventually.

On firefox or any other browser, internet access is instant, including identical searches on the same PC simultaneously.

pfb is working OK when enabled ie blocking incoming and outgoing ipv4 correctly.

When pfBlockerNG is disabled, searches and access on chrome become instant too, so something in pfBlocker is slowing down just chrome.
This behaviour is consistent across different PCs on different versions of windows.

I don't really have much idea what this can be, and logs aren't telling me much.

tl;dr - chrome+pfblockerng = very slow. Firefox = fast. Turn off pfblocker =all browsers incl chrome fast

tonymorella

@robatwork:

OK here's a strange occurrence…
pfSense just upgraded to 2.3.2 but this behaviour happened with 2.2.1 on a physically different router.

pfb version 2.1.1_4.

On our LAN when we use google chrome (latest) and go to google it can take several seconds for google to display. Doing a search can take 20 seconds to respond, but it does come back eventually.

On firefox or any other browser, internet access is instant, including identical searches on the same PC simultaneously.

pfb is working OK when enabled ie blocking incoming and outgoing ipv4 correctly.

When pfBlockerNG is disabled, searches and access on chrome become instant too, so something in pfBlocker is slowing down just chrome.
This behaviour is consistent across different PCs on different versions of windows.

I don't really have much idea what this can be, and logs aren't telling me much.

tl;dr - chrome+pfblockerng = very slow. Firefox = fast. Turn off pfblocker =all browsers incl chrome fast

What block lists are you running? Also, ctrl-shift-I to get into dev mode for both look at performance in FF and timeline in chrome, run them both and compare.

robatwork

@tonymorella:

What block lists are you running? Also, ctrl-shift-I to get into dev mode for both look at performance in FF and timeline in chrome, run them both and compare.

Currently running top_v4 and top_v6 by selecting various countries in the Top 20.
Also 5 lists in IPv4 to block some social networking sites by AS number.
And DNSBL is enabled although I was testing the speed prior to this and I don't think it made any difference. TLD isn't ticked. All other options are default.

BBcan177

@robatwork:

On our LAN when we use google chrome (latest) and go to google it can take several seconds for google to display. Doing a search can take 20 seconds to respond, but it does come back eventually.

You can open the Chrome cache viewer with this URL (Clear that and see if it helps):

chrome://net-internals/#dns

Also ensure that the LAN devices only have pfSense as its DNS server.

robatwork

@BBcan177:

@robatwork:

On our LAN when we use google chrome (latest) and go to google it can take several seconds for google to display. Doing a search can take 20 seconds to respond, but it does come back eventually.

You can open the Chrome cache viewer with this URL (Clear that and see if it helps):
chrome://net-internals/#dns
Also ensure that the LAN devices only have pfSense as its DNS server.

Will try the cache clear, thanks.

However the LAN devices are all set with our domain servers as DNS servers. The DNS on the AD servers are all set to just have the pfsense as the only forwarder. pfSense has DNS Resolver only. I did just watch the last hangout about DNS and believe that I am setup OK with just resolver even though we are multi-WAN. If you think I'd be better off with DNS forwarder instead please let me know.

Thanks for your help :)

BBcan177

@robatwork,

So to be clear, ensure that the LAN devices have only your AD DNS servers defined. Then set the AD DNS server DNS Forwarder settings to pfSense only… This way DNSBL will filter the traffic.

The resolver can be in Resolver mode or in Forwarder mode.... I recommend Resolver mode....

If you are referring to the DNS Forwarder (dnsmasq), then DNSBL will not function, as its not configured for that.

robatwork

@BBcan177:

@robatwork,

So to be clear, ensure that the LAN devices have only your AD DNS servers defined. Then set the AD DNS server DNS Forwarder settings to pfSense only… This way DNSBL will filter the traffic.

The resolver can be in Resolver mode or in Forwarder mode.... I recommend Resolver mode....

That is exactly how we are configured (in Resolver mode). Testing continues….

jim82

Hi all,

Yesterday I added 4 new lists to DNSBL, see attachment 1.

Right after I added these new lists, my DNSBL Alerts/Log stopped working, and all blocks are now shown in the "DENY" log section. See attachment 2.

Can anyone shed light as to what I have done wrong?

Thanks

BR Jim

1.JPG_thumb

2.JPG_thumb