Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes

mpfrench

Steve, please try browsing to xnxx.com, xvideos.com and report whether or not those sites were blocked in your setup.
Thanks,
Mike

SteveITS

@mpfrench I can't get the list downloaded.

On your setup on the pfBlocker log page is the dnsbl.log file empty? That should list everything that is blocked.

mpfrench

Steve, I gave up trying to use the DNSBL UT1_Adult. My Netgate 1100 is not capable of running it as we both learned the hard way.

Also, I gave up using all DNSBLs since I can't get them to run using the DNS Resolver set to forward instead of resolve. My system ignores the setting to forward in the resolver. It resolves everything without forwarding.

So on my system, even though I specified external DNS addresses 1.1.1.3 and 1.0.0.3 which block malicious and adult content, my system does not use them if I have the DNS Resolver selected in the Settings menu. So adult sites xnxx.com and xvideos.com com busting through on a web browser.

Consequently, I turned off the DNS Resolver and turned on the DNS Forwarder in the Services menu. Now my system blocks adult content. However, ALL the DNSBLs are inactive.

This is the catch-22 I started this thread about. BBcan177 should redesign pfBlockerNG to let the DNSBLs be active while using the DNS Forwarder if this is possible.

I am still curious to know if you tried browsing to xnxx.com and xvideos.com to test whether or not your system really blocks them.
Thanks,
Mike

SteveITS

@mpfrench If I query those domains they aren't blocked but I don't have a list that would block them so that is expected.

If you do it with a different list does it work? Because it does for me.

Enable DNS Resolver, set to forward
Enable the StevenBlack_ADs feed.
run pfBlocker update
nslookup fls-na.amazon.com

Does it return the 10.10.10.1 IP?

The problem with DNSBL and DNS Forwarder is that Forwarder does not hold any information locally. It literally just forwards the query. Using Resolver it can create fake local zone files that override public DNS. In Forwarder it might be able to do that if it adds a bunch of host overrides but those might need to all go in the pfSense config file? Not sure.

mpfrench

Steve, in a prior message you stated that you used OpenDNS Family Shield to block adult sites. If you had your forwarding DNS settings set to these servers, you should not have been able to get the content from xnxx.com or xvideos.com.

I have the forwarding addresses added as follows:
family.cloudflare-dns.com 1.1.1.3 & 1.0.0.3.
These are added at System - General Setup - DNS Servers.

When I use the pure forwarder (dnsmasq) - [Services - DNS Forwarder], all adult content is blocked.

When I turn off dnsmasq and turn on Unbound resolver [Services - DNS Resolver], adult content is not blocked even though the Enable Forwarding Mode option is activated.

It looks as though there is a bug in the GUI or underlying software. Unbound [Services - DNS Resolver] is not forwarding as the option says it should.

SteveITS

@mpfrench Sorry if I wasn’t clear I have quad9 set as forwarder, and DNSBL is working with the list I have set.

mpfrench

Steve, if you would please, set your forwarded DNS to 1.1.1.3 and 1.0.0.3. Then try to browse to xnxx.com and xvideos.com. If you can see those sites, your system is NOT using those two DNS servers.

Gertjan

@mpfrench said in Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes:

When I turn off dnsmasq and turn on Unbound resolver [Services - DNS Resolver], adult content is not blocked even though the Enable Forwarding Mode option is activated.

I'm using, right now :

https://forum.netgate.com/topic/178413/major-dns-bug-23-01-with-quad9-on-ssl/59?_=1680846815156

So, unbound is forwarding over TLS (port 853) to 1.1.1.1 and related.

I've pfBlockerng, with this DNSBL :

Here is the URL of that feed :

Whne I copied https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts in mybrowser, I saw the list.

I tested with two host names :

[23.01-RELEASE][admin@pfSense.myplace.tld]/root: host ck.getcookiestxt.com
ck.getcookiestxt.com has address 0.0.0.0
Host ck.getcookiestxt.com not found: 3(NXDOMAIN)
[23.01-RELEASE][admin@pfSense.myplace.tld]/root: host www.30-day-change.com
www.30-day-change.com has address 0.0.0.0

Now I disable pfBlocker :

[23.01-RELEASE][admin@pfSense.pfSense.myplace.tld]/root: host ck.getcookiestxt.com
Host ck.getcookiestxt.com not found: 3(NXDOMAIN)
[23.01-RELEASE][admin@pfSense.pfSense.myplace.tld]/root: host www.30-day-change.com
www.30-day-change.com has address 199.59.243.223

So, the first host name doesn't even exist anymore ;)
The second got resolved.

So,

Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes

Ok, I'm not using a 4100, not a 1100 - and I use 23.01 with all the official patches.
But still, never saw some one telling that was only for 'x100' and up, not '1100', so, without proving I still tend to say : pfBlockerNG does DNSBL on a 1100 when forwarding.

Btw : using 'huge' lists on Netgate's smallest device (not counting the now expires '1000' which I see more as a show-case device) will create issues.
Available RAM always counts.

The device you use, is it a PC ?
Can you run ipconfig /all ?
What is the DNS used by the device ?
MAC : a command showing the same info exists.
I mean : if your device is using for example 8.8.8.8 then it will bypass pfSense (and thus unbound, pfBlockerng etc).

edit :

I activated "UT1" and activated the first 'Adult' feed.
It stayed at this point for over two minutes :

and after this rather long delay it continued :

"TLD Analyses" took 'ages'.

This list is even 'to much' for my 4 Gb RAM Netgate '4100', although, it was loaded.
TLD Analyses" not doing its job isn't a big deal.

Btw : 4504216 in a list is pure madness. Running DNS smoothly with such lists to parse for every DNS request needs "Intel I9" stuff.

I re-post my message, and pfBlocker still hasn't finished doing its forced reload.
Memory allocation went up to

This is interessting :

[23.01-RELEASE][admin@pfSense.myplace.tld]/root: host xnxx.com
xnxx.com has address 0.0.0.0
xnxx.com mail is handled by 30 aspmx2.googlemail.com.
xnxx.com mail is handled by 30 aspmx3.googlemail.com.
xnxx.com mail is handled by 30 aspmx4.googlemail.com.
xnxx.com mail is handled by 30 aspmx5.googlemail.com.
xnxx.com mail is handled by 10 aspmx.l.google.com.
xnxx.com mail is handled by 20 alt1.aspmx.l.google.com.
xnxx.com mail is handled by 20 alt2.aspmx.l.google.com.

So, great : no more xnxx.com.

On a PC that I just fired up (so it has not cached locally xnxx.com) :

C:\Users\Gauche>nslookup xnxx.com.
Serveur :   pfSense.myplace.tld
Address:  2a01:dead:907:a6dc::1

Réponse ne faisant pas autorité :
Nom :    xnxx.com
Addresses:  185.88.181.60
          185.88.181.59
          185.88.181.58
          185.88.181.57
          185.88.181.56
          185.88.181.55
          185.88.181.54
          185.88.181.53

Ok, no panic :

ipconfig /all

But again : xnxx.com still resolved on my PC : wtf !

This is probably related to the fact that my pfBlockerng full reload can't finish !!! it's job anymore :

without any errors, it just times out / stops.

This means unbound didn't get restarted => the newly generated list wasn't taken in account etc.
Manually restating unbound does seem to help (a bit).

Conclusion : stay away from big lists with millions of entries.
Blocking adult sites needs adult equipment ;)

mpfrench

@Gertjan, I came to the same conclusion as you did regarding the use of the DNSBL UT1_Adult. It is imprudent to try running it on anything but a very powerful computer. This is the reason I want to forward DNS queries to a "family" service that blocks adult content for me.

I would like to see your results when you forward DNS queries to Cloudflare's family servers, 1.1.1.3 and 1.0.0.3.

After making the proper changes, let me know if you can see porn sites such as xnxx.com and xvideos.com while using Unbound in the forwarding mode.

I can't get this to work on my Netgate 1100 running the latest software versions using Unbound in the forwarding mode. However, it does work correctly using DNSmasq.

SteveITS

@mpfrench said in Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes:

Steve, if you would please, set your forwarded DNS to 1.1.1.3 and 1.0.0.3. Then try to browse to xnxx.com and xvideos.com. If you can see those sites, your system is NOT using those two DNS servers.

It returns 0.0.0.0 which is also what I get if I query them directly. Ensure there are no other servers listed on your System/General tab?

As above, you're testing with nslookup or dig, and not a browser?

mpfrench

@SteveITS , you did not answer the question I asked. Did you see the content of the sites xnxx.com and xvideos.com when you browsed to them after using 1.1.1.3 and 1.0.0.3?

You asked about my DNS settings. Here they are:

Thanks for your help.
Mike

SteveITS

@mpfrench I did not test a web browser; I was using our office router.

If nslookup returns 0.0.0.0 and your web browser is connecting anyway, then either:

your PC has cached the result
your PC has a DNS server configured besides pfSense
your browser is using DNS over HTTPS and bypassing your local DNS

For the former, restart the DNS Cache service in Windows. For the latter, block DoH via the pfSense doc at https://github.com/jpgpi250/piholemanual.

Gertjan

@mpfrench said in Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes:

I would like to see your results when you forward DNS queries to Cloudflare's family servers, 1.1.1.3 and 1.0.0.3.

I can't.
1113 and 1003 filters without my control.
I've a company (hotel) to run with my connection. Not a bunch of curious kids ;)
And I've nothing against p0rn sites. It's that, or reeving strange ladies at the reception

But I don't doubt about the fact that 1113 and 1003 work well.

mpfrench

@Gertjan , @SteveITS , thanks to both for your help.

I've run some more tests which may shed some light on the root cause of my not being able to get Unbound to forward instead of resolve.

Using DNSmasq [Services - DNS Forwarder], NSLOOKUP returns 0.0.0.0 for both xnxx.com and xvideos.com. A web browser refuses to show both sites.

Using Unbound [Services - DNS Resolver], I get two different responses from NSLOOKUP.

For xnxx.com, I get 10.10.10.1 and a browser shows the pfB blocking message screen. Evidently, that site is in a DNSBL other than UT1_Adult which is not enabled on my system.

For xvideos.com, NSLOOKUP delivers the message "can't find xvideos.com: Server failed." However, a web browser actually shows the site.

When performing NSLOOKUP xnxx.com 1.1.1.3 and NSLOOKUP xvideos.com 1.1.1.3, I get the answer returned 0.0.0.0 for both.

This is proof that Unbound did not forward the DNS query to 1.1.1.3 or 1.0.0.3 per my setup settings.

I hate to give up on Unbound since the DNSBLs work with it and do not work with DNSmasq, but Unbound just isn't forwarding as it should and I'm not sure why. It appears to be a bug.

In the mean time, I'll use DNSmasq and forgo DNSBL filtering since DNSmasq forwards correctly.

SteveITS

@mpfrench said in Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes:

This is proof that Unbound did not forward the DNS query

As I've asked above did you block DoH? Because most web browsers use DoH now. Other than the browser it sounds like the DNS block is working.

re: the failed query also check if you have DNS over TLS enabled, and uncheck that. (https://forum.netgate.com/topic/178413/major-dns-bug-23-01-with-quad9-on-ssl)

SteveITS

(drums fingers) Realizing what I'm writing, it seems like a browser using DoH would use it regardless of the DNS setup on pfSense. But, it is expected nowadays that browsers do use it and go around DNS, notably Firefox. If the problem here is that DoH is being used for DNS Resolver and not DNS Forwarder that isn't normal as the browser shouldn't know the configuration on the router.

mpfrench

@SteveITS , DoH was never a cause of my problem. I made sure the browser with which I tested did not have DoH enabled.

After, meticulously reviewing every DNS setting in pfB and pfSense, rebooting everything (Netgate 1100, router, computer), and issuing a flushDNS command between every test, I think I've gotten Unbound to work as a forwarder to my "family" DNS servers, 1.1.1.3, and 1.0.0.3.

Now NSLOOKUP returns 10.10.10.1 for xnxx.com and the browser shows the pfB blocking screen, indicating that xnxx.com is on the UT1_Redirector blocking list.

NSLOOKUP returns 0.0.0.0 for xvideos.com and the browser shows the "Try again Charlie" screen.

I think the Enable DNSSEC option was interfering with Unbound's proper operation. I'm currently running without DNSSEC enabled.

I also think I was having an issue with the DNS cache not being flushed between tests which I corrected.

So far, the teenagers in my house have not figured out what DoH is good for, but if they do, it would seem that I could set pfB to block the exact web sites DoH uses. But I'll cross this bridge when I need to -- another day.

I appreciate your help. I think it is safe to close this topic.
Mike

Gertjan

@mpfrench said in Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes:

and the browser shows the "Try again Charlie" screen.

They won't. They'll understand.
They have Google. They will do what you would do.
.... 5 minutes later ....
They stop using 'your network', and take another one, like a SIM 4G/5G data card from their phone.
Case 'solved'.

I say this because " I've been there - seen it - thought I needed to do something with a tool ".
All you can do is explaining, and showing the right example.
It has been written somewhere : everybody has the right to dig its own hole, and then fall into it.
I bought a rope, so I can help, if asked or needed ;)