DNSBL Certificate errors

DarkKnyte

I attempted to setup DNSBL in the PFBlockerNG package yesterday and following adding lists (provided from another post here) and activation I immediately started getting SSL CA errors in chrome on google.com. I had to disable DNSBL to stop the errors.

I find it likely that I must have missed some option or toggled a box I shouldn't have, but I can't seem to narrow down the issue.

Can someone point me in a direction if this is an obvious problem, or tell me what I need to provide to help troubleshoot?

Thanks.

doktornotor

There is no such option. Whitelist obvious false positives in the DNSBL (or stop using such broken lists).

BBcan177

You could edit the code and use 0.0.0.0 instead of the DNSBL VIP… but that will also negate any logging capabilities...

https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L3609

/usr/local/pkg/pfblockerng/pfblockerng.inc

Change this line #3609:

$domain_data .= "local-data: \"" . $line . " 60 IN A {$pfb['dnsbl_vip']}\"\n";

To:

$domain_data .= "local-data: \"" . $line . " 60 IN A 0.0.0.0\"\n";

Upcoming version will allow mixing of the DNSBL VIP and 0.0.0.0 so that the domains that are causing these erorrs, can utilize 0.0.0.0 whilst all others use the DNSBL VIP and the logging remains intact… or if you didn't want to log the blocked alerts....

bole5

I solved this problem by adding a firewall rule on my LAN (and OpenVPN) interfaces to reject all requests destined for 127.0.0.1 port 8443. This way I don't get to see the annoying message about certificate for https based blocked sites.

The benefit of my solution is that it requires no code change to pfBlockerNG.

To do this create new rule with following:

Action: Reject
Interface: LAN
Address Family: IPV4
Protocol: TCP
Source: any
Destination: Single host or alias 127.0.0.1
Destination port range: custom 8443 (in both custom fields)

Give it a name and press Save.
Don't forget to move this rule before any rules that would allow this traffic.

BBcan177

@bole5

Can you change the code as indicated (utilizing 0.0.0.0) and see if that fixes your cert errors. I don't have many Apple devices to test with, so any help testing would be appreciated. Will need to run a "Force Reload - DNSBL" for it to take effect.

Any other users feedback welcome also…

Thanks!

bole5

I just tested with 0.0.0.0. After modifying the /usr/local/pkg/pfblockerng/pfblockerng.inc I verified that the force update worked:


#head -10 /var/unbound/pfb_dnsbl.conf
local-data: "007-gateway.com 60 IN A 0.0.0.0"
local-data: "00zasdf.pw 60 IN A 0.0.0.0"
local-data: "04dn8g4f.space 60 IN A 0.0.0.0"
local-data: "0755.pics 60 IN A 0.0.0.0"
local-data: "07zq44y2tmru.xyz 60 IN A 0.0.0.0"
local-data: "0emn.com 60 IN A 0.0.0.0"
local-data: "0fmm.com 60 IN A 0.0.0.0"
local-data: "0icep80f.com 60 IN A 0.0.0.0"
local-data: "0llii0g6.com 60 IN A 0.0.0.0"
local-data: "0pixl.com 60 IN A 0.0.0.0"

Then I went to amazon.de in both Safari and Chrome on my MacBook and no more annoying certificate error.

Just to verify time it needed to run the request I copied the Ad link as cUrl and run in the terminal. Here is what I get instantly:


$ curl 'https://aax-eu.amazon-adsystem.com/e/xsp/getAd?slot=desktop-ad-center-2&rid=01010de2fa3142bbf24492371c23e1a48121a47315d92fa3142bbf24492372fa3142b' \
→ -XGET \
→ -H 'Referer: https://www.amazon.de/' \
→ -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/602.4.8 (KHTML, like Gecko) Version/10.0.3 Safari/602.4.8' \
→ -H 'Origin: https://www.amazon.de'
*   Trying 0.0.0.0...
* connect to 0.0.0.0 port 443 failed: Connection refused
* Failed to connect to aax-eu.amazon-adsystem.com port 443: Connection refused
* Closing connection 0
curl: (7) Failed to connect to aax-eu.amazon-adsystem.com port 443: Connection refused

So to conclude changing address to 0.0.0.0 worked on OSX (10.11.6) in Safari and Chrome.

Hopsdaballa04

I replaced the line in the .inc file to match $domain_data .= "local-data: "" . $line . " 60 IN A 0.0.0.0"\n";

But I still see the cert issue popping up in iOS and MacOS. I'm not sure what I didn't do right, I restarted the service and did a force reload in pfblockerng but I'm still seeing the errors. When I apply the suggested firewall rule it works, so not sure if I just botched something up. Can you give me a step by step of what to change again?

BBcan177

Did you run a "Force Reload - DNSBL" for the change to take effect? You also might have cached DNS responses in your browser and OS causing issues. Someone also mentioned that you might need to clear out any DNSBL certificate approvals in your browser.

HeMaN

@BBcan177:

You could edit the code and use 0.0.0.0 instead of the DNSBL VIP… but that will also negate any logging capabilities...

https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L3609

/usr/local/pkg/pfblockerng/pfblockerng.inc

Change this line #3609:
$domain_data .= "local-data: \"" . $line . " 60 IN A {$pfb['dnsbl_vip']}\"\n";
To:
$domain_data .= "local-data: \"" . $line . " 60 IN A 0.0.0.0\"\n";
Upcoming version will allow mixing of the DNSBL VIP and 0.0.0.0 so that the domains that are causing these erorrs, can utilize 0.0.0.0 whilst all others use the DNSBL VIP and the logging remains intact… or if you didn't want to log the blocked alerts....

Hi @BBcan177, thnx for the great work on pfBlockerNG!

I also ran into the issue of the warning about the certificate not belonging to the site and not being trusted.

Did some research on the web and found several statements about using a generic self signed certificate for all redirected traffic, and that the solution would be to create certificates for the domains on the fly with a certificate known/trusted by the users.

I only switched to using pfSense a week ago, and was using an Asus router with wrt-Merlin firmware before.
On this router I also had a "tool" installed for dns based ad-blocking called AB-Solution (https://www.snbforums.com/threads/release-ab-solution-3-6-5-the-ad-blocking-solution.37511/).
The beauty of this tool is amongst others the option to use pixelsrv-tls. This is a 1*1 pixel server also capable of https.
It does also create on-the-fly certificates specific for the https-urls being blocked. (https://github.com/kvic-z/pixelserv-tls)
No more annoying certificate warnings in my browser for any blocked site (after I installed the used CA-certificate of course)

I am absolutely no programmer, but maybe this is something worth to look into if it is possible to incorporate this functionality with pfBlockerNG?
If on the fly certificate creation is possible on a small home router, it should also be possible on a better utilised pfSense system ;)

BBcan177

Thanks for the links… It's not something that I would want for the package... MITM anything is bad in my books :)
I am working on improving this issue… So stay tuned...

HeMaN

@BBcan177:

Thanks for the links… It's not something that I would want for the package... MITM anything is bad in my books :)
I am working on improving this issue… So stay tuned...

OK thnx! I will stay tuned ;)

Just to be sure, it is not mitm that pixelserv-tls is doing.
It is a webserver only serving a transparant 1*1 pixel for requests made to that server (redirected to the pixelserv-tls ip by dns blocklist), and also capable of generating a certificate for that pixel when the request was made with https.

All traffic that is not dns-blocked/redirected never go through the pixelserv-tls server.

motific

Creating custom certs for domains you don't own is a MITM method.

Not that it would work for google.com as they use HSTS preloading and public-key pinning. Browser makers bake information about the certificate chain for some sites into the package/installation. The browser knows about the certs it should be expecting for those sites before a request is even made and will warn the user if the certificate has been tampered with.

Sekrit

@BBcan177:

@bole5

Can you change the code as indicated (utilizing 0.0.0.0) and see if that fixes your cert errors. I don't have many Apple devices to test with, so any help testing would be appreciated. Will need to run a "Force Reload - DNSBL" for it to take effect.

Any other users feedback welcome also…

Thanks!

Hi BBcan,
I was able to fix the certificate invalid errors in Safari by editing the pfblockerng.inc. Is this a one time fix or needs to be done on every update? Could you create a GUI option for this very annoying problem for Safari users?
Thanks

BBcan177

@Sekrit:

Could you create a GUI option for this very annoying problem for Safari users?

It's already in the beta of the next release… stay tuned!

Sekrit

After 2 weeks, I started getting certificate errors again. Strangely, pfblockerng.inc reverted to the original (dnsbl_vip). I replaced it and it works again.

BBcan177

@Sekrit:

After 2 weeks, I started getting certificate errors again. Strangely, pfblockerng.inc reverted to the original (dnsbl_vip). I replaced it and it works again.

If you make manual changes to the pfblockerng.inc file, those will be lost on a pkg installation. So you most likely installed v2.1.1_8 which reset the file back to default… The next release should have this fix built-in...

SLIMaxPower

I followed this guide and dropbox is refused to connect.

DNSBL

May 28 14:34:43 Unknown Unknown www.google-analytics.com
Not available for HTTPS alerts

May 28 14:19:03 Unknown Unknown www.dropbox.com
Not available for HTTPS alerts

both show the domains are in the whitelist.

If and Source and Unknown and list shows no match.

BBcan177

Can you post a screenshot of the Whitelist and the Alerts Tab showing these blocked domains.

SLIMaxPower

Currently the domains are whitelisted in the custom domain whitelist. Is this correct or should they go in the TLD whitelist

BBcan177

@SLIMaxPower:

Currently the domains are whitelisted in the custom domain whitelist. Is this correct or should they go in the TLD whitelist

The Custom Domain Whitelist is used to "whitelist" domains…

The TLD Whitelist is only used in combination with TLD Blacklist… An example of that would be where you want to block all "ru" domains with TLD Blacklist, but you want to allow certain ru domains to get thru.

GoldServe

What is the option to not serve up a https image to avoid certificate errors in 2.1.1_8?

Matze_

[ DNSBL FAIL ] [ Skipping : SuspiciousDomains ]

What feed URL are you using? There are three options available:

https://isc.sans.edu/feeds/suspiciousdomains_High.txt
https://isc.sans.edu/feeds/suspiciousdomains_Medium.txt
https://isc.sans.edu/feeds/suspiciousdomains_Low.txt

Otherwise check that you didn't copy/paste the new patched line incorrectly…

Sekrit

@BBcan177:

@Sekrit:

After 2 weeks, I started getting certificate errors again. Strangely, pfblockerng.inc reverted to the original (dnsbl_vip). I replaced it and it works again.

If you make manual changes to the pfblockerng.inc file, those will be lost on a pkg installation. So you most likely installed v2.1.1_8 which reset the file back to default… The next release should have this fix built-in...

BBcan,
Just installed the next release and certificate error has returned. :(

BBcan177

@Sekrit:

@BBcan177:

@Sekrit:

After 2 weeks, I started getting certificate errors again. Strangely, pfblockerng.inc reverted to the original (dnsbl_vip). I replaced it and it works again.

If you make manual changes to the pfblockerng.inc file, those will be lost on a pkg installation. So you most likely installed v2.1.1_8 which reset the file back to default… The next release should have this fix built-in...

BBcan,
Just installed the next release and certificate error has returned. :(

Sorry to get your hopes up… but the last release was just a small patch... Not quite finished with it yet...

kjstech

The firewall rule worked! I changed my DNSBL SSL port to 8082 though since I have a Unifi controller running on my pfsense box on 8443.

kvic

@BBcan177:

Thanks for the links… It's not something that I would want for the package... MITM anything is bad in my books :)
I am working on improving this issue… So stay tuned...

You can decide what to include in your package. But pixelserv-tls is not MITM blah.

kvic

@motific:

Creating custom certs for domains you don't own is a MITM method.

Not that it would work for google.com as they use HSTS preloading and public-key pinning. Browser makers bake information about the certificate chain for some sites into the package/installation. The browser knows about the certs it should be expecting for those sites before a request is even made and will warn the user if the certificate has been tampered with.

First time I hear such a definition of MITM. Maybe you have a point. Perhaps blocking ad by poisoning DNS record shall be in this category too.

Your understanding of HSTS and what's built in chrome/firefox doesn't seem right to me.

Velcro

Just my 2 cents not sure of implications but since upgrading to IOS 11.0.3(including the 3 IOS updates in the last 2-3 weeks), I used to get a pop-up's on my iPhone safari…now I get a "safari cannot open...could not establish a secure connection...".

In firefox on Linux I got redirected to a certificate error...went thru and made an exception...now I get the 1x1 pixel page.

I'll take a cert error or 1x1 pixel page...just no spying!!!

I love you BBCAN!

NasKar

I've tried
Action: Reject
Interface: LAN
Address Family: IPV4
Protocol: TCP
Source: any
Destination: Single host or alias 127.0.0.1 or 10.10.10.1
Destination port range: custom 8443 (in both custom fields)

DNSBL configuration
DNSBL Virtual IP 127.0.0.1 or 10.10.10.1

It does prevent the certificate errors but doesn't block the ads on the yahoo.com home page on ipad or macbook pro

If I edit the code in the on line 3636 in /usr/local/pkg/pfblockerng/pfblockerng.inc to

  $domain_data .= "local-data: \"" . $line . " 60 IN A 0.0.0.0\"\n";

No certificate error but doesn't block the ads on yahoo.com home page on ipad or mackbook pro
I did reload the DNSBL.
My version of pfBlockerNG is 2.1.2

Not sure what I'm doing incorrectly.

repomanz

Hi folks - I was wondering if others that have upgraded to pfsense 2.4.1 are having the certificate errors again? Previously in 2.3.4 the solution described above was working fine with the lan rule blocking any traffic to the dnsbl vip. Post upgrade to 2.4.1 I'm getting certificate errors again from my AV solution. This began happening immediately after upgrade of pfsense to 2.4.1.

Anyone else having this trouble?

gp-se

@repomanz:

Hi folks - I was wondering if others that have upgraded to pfsense 2.4.1 are having the certificate errors again? Previously in 2.3.4 the solution described above was working fine with the lan rule blocking any traffic to the dnsbl vip. Post upgrade to 2.4.1 I'm getting certificate errors again from my AV solution. This began happening immediately after upgrade of pfsense to 2.4.1.

Anyone else having this trouble?

I'm on 2.4.1 with the latest PFBlocker installed, I haven't gotten the certificate errors at all. When I was on 2.3.4 I had to edit the file or I would get certificate errors.

repomanz

Can you share your dnsbl configuration and any lan / float rules you may have on this? I'm hoping it's some minor configuration change i need to make instead of a full re-install.

Sekrit

No more certificate errors on the 2.4.1 and latest version of pfblocker. This was a fresh installation.

HeMaN

anyone figured out (with pfblockerng 2.1.2-1) why blocking of 127.0.0.1 or 10.10.10.1 port 8443 is not working anymore to stop the certificate error messages with the current version of pfsesne (2.4.1)?

Is there a solution that does not include the editing of the inc-file with 0.0.0.0?

repomanz

I still have the issue. Currently i have dnsbl turned off and using pihole until it get's figured out.

BBcan177

@HeMaN:

anyone figured out (with pfblockerng 2.1.2-1) why blocking of 127.0.0.1 or 10.10.10.1 port 8443 is not working anymore to stop the certificate error messages with the current version of pfsesne (2.4.1)?

Is there a solution that does not include the editing of the inc-file with 0.0.0.0?

The only thing that changed was a recent commit to add the "Filter rule association" in the NAT rules. It was not previously defined, and since was changed to "pass"…

https://github.com/pfsense/FreeBSD-ports/commit/b4afa470b6599acfd48f902322452ee426995d76#diff-2fbe23c4e674f981e4a4003fb2794b27

Next release will have functionality to force "0.0.0.0" for a DNSBL group to avoid this issue....

HeMaN

thnx!!
this was the hint I was looking for ;)

I now managed to loose the error warnings by editing the line created by pfblockerNG in Firewall->Nat->Port Forward

 PFSENSE_LAN TCP * * 10.10.10.1 443 (HTTPS) 127.0.0.1 8443 pfB DNSBL - DO NOT EDIT

I changed the filterrule link from "pass" to "none", since it seems to precede over the flowing rule I made to block traffic to 127.0.0.1:8443

No more certificate errors for me

PS
I think a floating rule to block 10.10.10.1:443 might work as well (now I was trying to block 10.10.10.1:8443 but that port was not being used after all)

hkirschk

Here are my observations and solution proposals regarding certificate handling, pfBlockerNG and macOS.

Configuration

I'm using macOS clients version 10.13.2, Safari version 11.0.2, pfSense version 2.4.2-RELEASE, pfBlockerNG version 2.1.2_2. I did not try those modified pfSense firewall rules as explained in the posts here, but kept the pfSense and pfBLockerNG configuration as unchanged as possible, i.e. no changes to firewall rules as set up by pfBlockerNG, and also no changes to php files directly.

What I observed:

When clicking on URLs in Safari whose DNS names were blacklisted by pfBlockerNG, I received an invalid certificate warning by Safari, asking me to confirm that I want to continue loading that page, which then should result in that 1x1 pixel image delivered by the pfBlockerNG lighttpd_pfb process. Technically, confirming to load such a page results in an entry into the macOS keychain marking certificate CN_DNSBL as trusted so that Safari should continue loading that page. I was asked over and over again for that load confirmation, even after having confirmed the certificate, and never ended up with that 1x1 pixel image of pfBlockerNG.
What also happened from time to time, the php-fpm process running on the pfsense did not answer any more. Calling the URLs https://10.10.10.1:8443 or http://10.10.10.1:8081 timed out in Safari. All I could do was a hard reboot of pfSense, even the ssh login to the pfsense machine stalled at the place where /etc/rc.initial calls /etc/rc.banner, I had to suspend the login to the pfsense by Control-Z, receiving a shell prompt then for reboot. Restarting the php-fpm process itself was not possible, it did not react to any kill request. This behaviour was quite annoying, since i had to reboot pfSense sometimes after a few hours, latest after a few days. This happened first some time after I upgraded to macOS Sierra, if I remember right.

Possible root causes:

More or less by chance I looked into the macOS keychain and found several instances of CN_DNSBL certificates, obviously with the same name and only differentiated by their finger print (and associated data, like valid from', valid to' date). When deleting all those certificates named CN_DNSBL from keychain, things began to work as expected: I have only been asked once by Safari for certificate & load confirmation, and the pfBlockerNG 1x1 pixel page loads as expected. No need for regular reboots of pfSense any more, and pfBlockerNG logging keeps on working.

So I guess the root cause for observation 1. is that Safari picks the certificate to check against from the keychain by name (CN_DNSBL), and if there is more than one instance with the same name it constantly picks the wrong certificate, the one from the keychain which does not match the certificate delivered by the 1x1 pixel server of pfBlockerNG.
I have no real explanation for observation 2., that stalled php-fpm process, I guess this happened because pfBlockerNG was flooded by requests coming from Safari, and at some point in time that process dived into some race condition. php-fpm stalled very often when I called pages in Safari where pretty much of its content was blacklisted by pfBlockerNG.

Solutions/workarounds:

Workaround 1: With each update of pfBlockerNG, a new instance of the CN_DNSBL certificate gets installed, thus use keychain utilitiy on each macOS client to remove all CN_DNSBL certificates after a pfBlockerNG update. Means touching every macOS client manually, or to have some clever shell script doing this (semi-)automatically.
Solution 2.a: pfBlockerNG should always use the same CN_DNSBL certificate, it should not change after an update. That way, the keychain utility will not store multiple instances of CN_DNSBL.
Solution 2.b: Introduce a drop down in the pfBlockerNG configuration where the user can select a certificate stored in pfSense' certificate manager.

Personally, I would prefer solution 2.b. That way I can install a self signed CA as trusted to my private machines & use a certificate for the 1x1 pixel server signed by my own CA.

jdeloach

Has anybody come up with a way to stop the DNSBL certificate errors when going to different web sites using IE 11 in windows? I see lots messages about this but doesn't seem to be any solution yet. I just started using DNSBL but looks like I need to hold off till there is a solution to these annoying errors.

HeMaN

@BBcan177:

@HeMaN:

anyone figured out (with pfblockerng 2.1.2-1) why blocking of 127.0.0.1 or 10.10.10.1 port 8443 is not working anymore to stop the certificate error messages with the current version of pfsesne (2.4.1)?

Is there a solution that does not include the editing of the inc-file with 0.0.0.0?

The only thing that changed was a recent commit to add the "Filter rule association" in the NAT rules. It was not previously defined, and since was changed to "pass"…

https://github.com/pfsense/FreeBSD-ports/commit/b4afa470b6599acfd48f902322452ee426995d76#diff-2fbe23c4e674f981e4a4003fb2794b27

Next release will have functionality to force "0.0.0.0" for a DNSBL group to avoid this issue....

I manually removed the "pass" you added to the code, and now the firewall rules are working again.
I had to remove it from the code, because if I manually edited the FW rule, the next time the definitions were reloaded the rule was changed back.

Happily waiting for the new version :)