Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes

mpfrench

Netgate 1100 running 23.01-RELEASE (arm64)
pfBlockerNG 3.2.0_3

I'm in a catch-22 situation with my Netgate 1100. I would like to block sites that contain adult content but the UT1_Adult DNSBL will not run on my 1100. See the topic that I posted yesterday titled "Is it Possible to Run the DNSBL UT1_Adult on a Netgate 1100?" to which nobody has replied.

Since I can't run the UT1_Adult DNSBL, I'm forced to use the 1100 with Unbound running in the forwarding mode to a family shield type DNS server such as OpenDNS Family Shield (https://www.opendns.com/setupguide/?url=familyshield) or Cloudflare DNS Family Shield (https://blog.cloudflare.com/introducing-1-1-1-1-for-families/).

However, when Unbound runs in the forwarding mode, none of the DNSBLs run in pfBlockerNG which eliminates a great deal of filtering other than adult content.

BBcan177, to fix this situation, pfBlockerNG should be redesigned to run DNSBLs while Unbound runs in either mode - resolver or forwarder. Is this possible?
Mike

SteveITS

@mpfrench Just to be clear, you're using this checkbox in the DNS Resolver settings?

I have that running with DNSBL and know it's working since I've had to allow a handful of hostnames.

mpfrench

Steve, thanks for you help but I still can't get it block adult content using your settings. I have the forwarding DNS servers set to 1.1.1.3 and 1.0.0.3 in the System/General Settings page which servers block malware and adult content which I've verified. When I deactivate the Unbound Resolver and active the DNS Forwarder (Services/DNS Forwarder menu option), adult content is blocked. However, when I deactive the DNS Forwarder and activate the DNS Resolver and set it up per the page shown in the attached file, the 1100 does not block adult content. 20230405_Services DNS Resolver General Settings_Page_1.jpg

SteveITS

@mpfrench If forwarding uncheck DNSSEC as that's known to cause failures...apparently more so on 23.01.

In your pictures, in the two Interfaces fields, is All selected? It looks like nothing is selected...

Are you testing with "nslookup domain 192.168.13.1"? Windows and most OSs will cache DNS, and browsers can go around local DNS by using DoH.

mpfrench

Disabling DNSSEC and rebooting had no effect.

"All" is selected for both network interfaces boxes. I printed the page to a PDF file and converted the PDF to jpeg files to upload them since the forum doesn't allow PDF files as uploads. I don't know why the highlighting got lost.

The LAN is 192.168.13.1/24. DNS queries from devices on the LAN are sent to 192.168.13.1.

My system acts as though it ignores the check box to forward DNS queries instead of resolve them directly.

mpfrench

Steve, if it is not too much trouble, run a few porn tests on your setup to see whether or not your system is really blocking adult content.

Try browsing to some well know porn sites such as pornhub.com, xvideos.com, and xnxx.com.

My system did not block any of them using the DNS Resolver in any configuration.
Thanks,
Mike

SteveITS

@mpfrench I edited the config.inc file to increase the PHP limit, added the UT1 list, and when I updated it overran the 1 GB RAM disk (which was under 100 MB to start) and ran out of space. Had to delete the *.raw files and kill all unbound processes to (re)start unbound and get DNS working again. So, I tried.

Anyway, I then did what I probably should have done initially and just pull a name from the dnsbl.log file:

>nslookup fls-na.amazon.com
Server:  pfSense
Address:  10.0.0.1

Non-authoritative answer:
Name:    fls-na.amazon.com
Address:  10.10.10.1

...so, resolving to the pfB block IP.

I did catch one line during the update:

TLD analysis........xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx completed [ 04/5/23 22:23:37 ]

  ** TLD Domain count exceeded. [ 800000 ] All subsequent Domains listed as-is **

So maybe it's too big and the TLD feature is truncating the list?

mpfrench

Steve, I think your last post belongs in the thread "Is it Possible to Run the DNSBL UT1_Adult on a Netgate 1100?"

The thread were in right now deals with another issue. I'm still interested to know whether or not your setup blocks the porn sites that I mentioned in my last post.
Mike

SteveITS

@mpfrench No it was this thread. I tried but ran out of disk space on the RAM disk, to load the adult list. Sites that are in my DNSBL ad/tracker lists are blocked though.

mpfrench

Steve, please try browsing to xnxx.com, xvideos.com and report whether or not those sites were blocked in your setup.
Thanks,
Mike

SteveITS

@mpfrench I can't get the list downloaded.

On your setup on the pfBlocker log page is the dnsbl.log file empty? That should list everything that is blocked.

mpfrench

Steve, I gave up trying to use the DNSBL UT1_Adult. My Netgate 1100 is not capable of running it as we both learned the hard way.

Also, I gave up using all DNSBLs since I can't get them to run using the DNS Resolver set to forward instead of resolve. My system ignores the setting to forward in the resolver. It resolves everything without forwarding.

So on my system, even though I specified external DNS addresses 1.1.1.3 and 1.0.0.3 which block malicious and adult content, my system does not use them if I have the DNS Resolver selected in the Settings menu. So adult sites xnxx.com and xvideos.com com busting through on a web browser.

Consequently, I turned off the DNS Resolver and turned on the DNS Forwarder in the Services menu. Now my system blocks adult content. However, ALL the DNSBLs are inactive.

This is the catch-22 I started this thread about. BBcan177 should redesign pfBlockerNG to let the DNSBLs be active while using the DNS Forwarder if this is possible.

I am still curious to know if you tried browsing to xnxx.com and xvideos.com to test whether or not your system really blocks them.
Thanks,
Mike

SteveITS

@mpfrench If I query those domains they aren't blocked but I don't have a list that would block them so that is expected.

If you do it with a different list does it work? Because it does for me.

Enable DNS Resolver, set to forward
Enable the StevenBlack_ADs feed.
run pfBlocker update
nslookup fls-na.amazon.com

Does it return the 10.10.10.1 IP?

The problem with DNSBL and DNS Forwarder is that Forwarder does not hold any information locally. It literally just forwards the query. Using Resolver it can create fake local zone files that override public DNS. In Forwarder it might be able to do that if it adds a bunch of host overrides but those might need to all go in the pfSense config file? Not sure.

mpfrench

Steve, in a prior message you stated that you used OpenDNS Family Shield to block adult sites. If you had your forwarding DNS settings set to these servers, you should not have been able to get the content from xnxx.com or xvideos.com.

I have the forwarding addresses added as follows:
family.cloudflare-dns.com 1.1.1.3 & 1.0.0.3.
These are added at System - General Setup - DNS Servers.

When I use the pure forwarder (dnsmasq) - [Services - DNS Forwarder], all adult content is blocked.

When I turn off dnsmasq and turn on Unbound resolver [Services - DNS Resolver], adult content is not blocked even though the Enable Forwarding Mode option is activated.

It looks as though there is a bug in the GUI or underlying software. Unbound [Services - DNS Resolver] is not forwarding as the option says it should.

SteveITS

@mpfrench Sorry if I wasn’t clear I have quad9 set as forwarder, and DNSBL is working with the list I have set.

mpfrench

Steve, if you would please, set your forwarded DNS to 1.1.1.3 and 1.0.0.3. Then try to browse to xnxx.com and xvideos.com. If you can see those sites, your system is NOT using those two DNS servers.

Gertjan

@mpfrench said in Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes:

When I turn off dnsmasq and turn on Unbound resolver [Services - DNS Resolver], adult content is not blocked even though the Enable Forwarding Mode option is activated.

I'm using, right now :

https://forum.netgate.com/topic/178413/major-dns-bug-23-01-with-quad9-on-ssl/59?_=1680846815156

So, unbound is forwarding over TLS (port 853) to 1.1.1.1 and related.

I've pfBlockerng, with this DNSBL :

Here is the URL of that feed :

Whne I copied https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts in mybrowser, I saw the list.

I tested with two host names :

[23.01-RELEASE][admin@pfSense.myplace.tld]/root: host ck.getcookiestxt.com
ck.getcookiestxt.com has address 0.0.0.0
Host ck.getcookiestxt.com not found: 3(NXDOMAIN)
[23.01-RELEASE][admin@pfSense.myplace.tld]/root: host www.30-day-change.com
www.30-day-change.com has address 0.0.0.0

Now I disable pfBlocker :

[23.01-RELEASE][admin@pfSense.pfSense.myplace.tld]/root: host ck.getcookiestxt.com
Host ck.getcookiestxt.com not found: 3(NXDOMAIN)
[23.01-RELEASE][admin@pfSense.pfSense.myplace.tld]/root: host www.30-day-change.com
www.30-day-change.com has address 199.59.243.223

So, the first host name doesn't even exist anymore ;)
The second got resolved.

So,

Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes

Ok, I'm not using a 4100, not a 1100 - and I use 23.01 with all the official patches.
But still, never saw some one telling that was only for 'x100' and up, not '1100', so, without proving I still tend to say : pfBlockerNG does DNSBL on a 1100 when forwarding.

Btw : using 'huge' lists on Netgate's smallest device (not counting the now expires '1000' which I see more as a show-case device) will create issues.
Available RAM always counts.

The device you use, is it a PC ?
Can you run ipconfig /all ?
What is the DNS used by the device ?
MAC : a command showing the same info exists.
I mean : if your device is using for example 8.8.8.8 then it will bypass pfSense (and thus unbound, pfBlockerng etc).

edit :

I activated "UT1" and activated the first 'Adult' feed.
It stayed at this point for over two minutes :

and after this rather long delay it continued :

"TLD Analyses" took 'ages'.

This list is even 'to much' for my 4 Gb RAM Netgate '4100', although, it was loaded.
TLD Analyses" not doing its job isn't a big deal.

Btw : 4504216 in a list is pure madness. Running DNS smoothly with such lists to parse for every DNS request needs "Intel I9" stuff.

I re-post my message, and pfBlocker still hasn't finished doing its forced reload.
Memory allocation went up to

This is interessting :

[23.01-RELEASE][admin@pfSense.myplace.tld]/root: host xnxx.com
xnxx.com has address 0.0.0.0
xnxx.com mail is handled by 30 aspmx2.googlemail.com.
xnxx.com mail is handled by 30 aspmx3.googlemail.com.
xnxx.com mail is handled by 30 aspmx4.googlemail.com.
xnxx.com mail is handled by 30 aspmx5.googlemail.com.
xnxx.com mail is handled by 10 aspmx.l.google.com.
xnxx.com mail is handled by 20 alt1.aspmx.l.google.com.
xnxx.com mail is handled by 20 alt2.aspmx.l.google.com.

So, great : no more xnxx.com.

On a PC that I just fired up (so it has not cached locally xnxx.com) :

C:\Users\Gauche>nslookup xnxx.com.
Serveur :   pfSense.myplace.tld
Address:  2a01:dead:907:a6dc::1

Réponse ne faisant pas autorité :
Nom :    xnxx.com
Addresses:  185.88.181.60
          185.88.181.59
          185.88.181.58
          185.88.181.57
          185.88.181.56
          185.88.181.55
          185.88.181.54
          185.88.181.53

Ok, no panic :

ipconfig /all

But again : xnxx.com still resolved on my PC : wtf !

This is probably related to the fact that my pfBlockerng full reload can't finish !!! it's job anymore :

without any errors, it just times out / stops.

This means unbound didn't get restarted => the newly generated list wasn't taken in account etc.
Manually restating unbound does seem to help (a bit).

Conclusion : stay away from big lists with millions of entries.
Blocking adult sites needs adult equipment ;)

mpfrench

@Gertjan, I came to the same conclusion as you did regarding the use of the DNSBL UT1_Adult. It is imprudent to try running it on anything but a very powerful computer. This is the reason I want to forward DNS queries to a "family" service that blocks adult content for me.

I would like to see your results when you forward DNS queries to Cloudflare's family servers, 1.1.1.3 and 1.0.0.3.

After making the proper changes, let me know if you can see porn sites such as xnxx.com and xvideos.com while using Unbound in the forwarding mode.

I can't get this to work on my Netgate 1100 running the latest software versions using Unbound in the forwarding mode. However, it does work correctly using DNSmasq.

SteveITS

@mpfrench said in Redesign pfBlockerNG to Run DNSBLs Using Unbound in Both Resolver and Forwarder Modes:

Steve, if you would please, set your forwarded DNS to 1.1.1.3 and 1.0.0.3. Then try to browse to xnxx.com and xvideos.com. If you can see those sites, your system is NOT using those two DNS servers.

It returns 0.0.0.0 which is also what I get if I query them directly. Ensure there are no other servers listed on your System/General tab?

As above, you're testing with nslookup or dig, and not a browser?

mpfrench

@SteveITS , you did not answer the question I asked. Did you see the content of the sites xnxx.com and xvideos.com when you browsed to them after using 1.1.1.3 and 1.0.0.3?

You asked about my DNS settings. Here they are:

Thanks for your help.
Mike