pfBlockerNG not blocking domain after first DNS lookup attempt

jrey

Wow that's a large list

TLD finalize...
Original Matches Removed Final
3649180 1412714 1873986 1775194

Only reason I mention this is the one specific list you are providing hasn't been touched in something like 8 years. Many of the domain names in it don't even exist anymore. You might want to review your lists for being current and accurate

That said with settings similar to yours, I was able to recreate the issue, with a much smaller list set.
As I mentioned above I think it is a cache issue, and once I got the system into a state of working like yours, the only way I could get it functional again was:

(also flush the DNS cache on the clients)

on the netgate
Turn off CNAME Validation
Turn off Resolver Cache,
restart the system
let the system start fully, don't query anything.

I added the top 3 entries on this list (there are many others, I checked my upstream DNS and what it had been resolving) keep in mind these are all the result of outbound queries, and won't happen unless there is a specific DNS request for them).

Screen Shot 2023-12-01 at 7.10.32 AM.png
Once running, you can re-enable both
CNAME and Resolver Cache
Reload the DNSBL as shown above
then once that competes, wait a couple of minutes visit Status -> Services and restart unbound one more time.

Test.
I need to go back and setup the failure again, but I'd like to see your results when you do this.

are you running DHCP Server on the device?
also Status -> System Logs (System Tab/ DNS Resolver)
filter like this, Screen Shot 2023-12-01 at 7.39.14 AM.png

Thanks

jrey

@vjizzle

please note that once I completed the above steps,
all of these lookups are within the same session, with the steps taken on pfSenses noted within.

 nslookup
> server 10.168.1.1
Default server: 10.168.1.1
Address: 10.168.1.1#53
> 
> www.msn.com
Server:         10.168.1.1
Address:        10.168.1.1#53

Non-authoritative answer:
Name:   www.msn.com
Address: 0.0.0.0
** server can't find www.msn.com: SERVFAIL
 
* at this point I turned off the list, removed the 3 entries and force reload the DNSBL * 

> www.msn.com
Server:         10.168.1.1
Address:        10.168.1.1#53

Non-authoritative answer:
www.msn.com     canonical name = www-msn-com.a-0003.a-msedge.net.
www-msn-com.a-0003.a-msedge.net canonical name = a-0003.a-msedge.net.
Name:   a-0003.a-msedge.net
Address: 204.79.197.203

* at this point I turned the list on again, and added the three entries, force reload the DNSBL * 

> www.msn.com
Server:         10.168.1.1
Address:        10.168.1.1#53

Non-authoritative answer:
Name:   www.msn.com
Address: 0.0.0.0
** server can't find www.msn.com: SERVFAIL
> a.msedge.net
Server:         10.168.1.1
Address:        10.168.1.1#53

Non-authoritative answer:
Name:   a.msedge.net
Address: 0.0.0.0
** server can't find a.msedge.net: SERVFAIL
> a-msedge.net
Server:         10.168.1.1
Address:        10.168.1.1#53

Non-authoritative answer:
Name:   a-msedge.net
Address: 0.0.0.0
** server can't find a-msedge.net: SERVFAIL
> dc-msedge.net
Server:         10.168.1.1
Address:        10.168.1.1#53

Non-authoritative answer:
Name:   dc-msedge.net
Address: 0.0.0.0
** server can't find dc-msedge.net: SERVFAIL
>

all results above from the client are within the same nslookup session. (didn't exit and/or do anything special on the client) the system is just returning the expected results again.

vjizzle

@jrey
Hi. I did the steps you asked but still the same.

disabled cname and resolver cache
restarted pfsense
waited a good 5 minutes after restart
re-enabled cname and resolver cache
reloaded the pfblockerng config using the Update tab

The first query succeeds and returns 0.0.0.0 but after that it starts resolving.

Please find the screenshot below of the DNS Resolver log:

jrey

@vjizzle said in pfBlockerNG not blocking domain after first DNS lookup attempt:

the DNS Resolver log:

that's it one entry? I'm trying to get a feel for how often the resolver is starting/restarting..

did you add the 3 specific blocks on the custom list ? you're not clear on those being added. if you did not please do and force reload the DNSBL
flush the cache on the client (windows) and try again.

the results you provide suggest strongly that another DNS is providing the response in the chain of responses

above you provided

Please find an overview of my DNS Resolver and DNSBL settings in the screenshots below.

but I don't see in that screen capture the fields that are above the "strict outgoing network setting" the point where you start the screen capture

Thanks

vjizzle

@jrey said in pfBlockerNG not blocking domain after first DNS lookup attempt:

@vjizzle said in pfBlockerNG not blocking domain after first DNS lookup attempt:

the DNS Resolver log:

that's it one entry? I'm trying to get a feel for how often the resolver is starting/restarting..

did you add the 3 specific blocks on the custom list ? you're not clear on those being added. if you did not please do and force reload the DNSBL
flush the cache on the client (windows) and try again.

the results you provide suggest strongly that another DNS is providing the response in the chain of responses

above you provided

Please find an overview of my DNS Resolver and DNSBL settings in the screenshots below.

but I don't see in that screen capture the fields that are above the "strict outgoing network setting" the point where you start the screen capture

Thanks

I did add the entries to the custom blocklist on my DNSBL group, did the force reload and also the restart. Flushed the cache from my Windows and MacOS client, but then again when using nslookup via cli there is no cache, that's the whole point of using nslookup of course.

But here is the thing, when add them to my custom blocklist they are always blocked and everything is working as expected.

In the strict outgoing setting I have enforced the DNS resolver in pfSense to use my VPN. Now I know that several VPN providers are doing DNS intercept, mine is not. Checked and verified with them. Also, my pihole is using the same VPN route to go to the internet and the unbound resolver on my pihole in combination with pihole itself never misses a beat when blocking.

jrey

@vjizzle

so then if you add the three items I provided it blocks, but when they are not there it doesn't.

you realize that this actually puts those entries on the same effective DNSBL list.

from pfsense diagnostics ping - ping this
ad.doubleclick.net
what do you get?

from your screen image provided earlier, you have this checked, so what group policy do you have that might potential allows bypass.

Screen Shot 2023-12-01 at 12.20.53 PM.png

my pihole is using the same VPN route to go to the internet and the unbound resolver on my pihole in combination with pihole itself never misses a beat when blocking

sure - but how is the network interface configured on the pihole vs the pfsense
specifically in the section on the pfsense page just above the strict outgoing network, setting where your screen capture starts, what are the settings for
Network Interfaces and Outgoing Network Interfaces
drag the low right corner to expand the lists so we can see what is selected.

jrey

@vjizzle

here are the results of every name on the block list you provided. no issue with anything on the list. All are returning "blocked" even though some of the names don't even exist anymore, they would still be blocked first.

When blocked
DNSBL Results.txt

When not blocked (those names where the name no longer exists are returning NXDOMAIN and those on another list still return 0.0.0.0, and those names that are still active and not on another list return IP addresses.

DNSBL Not Blocked Results.txt

DNSBL is working as I would expect.

vjizzle

@jrey
So what you are seeing if domain names don’t exist anymore and are in fact returning a NXDOMAIN, this throws pfblockerng into despair as it then starts to answer?

If I compare Pihole with pfblockerng, it does not matter what its resolver is doing, or the forwarder or the actual dns record on the internet, when a domain name is in a blocklist…Pihole just blocks it.

jrey

@vjizzle said in pfBlockerNG not blocking domain after first DNS lookup attempt:

NXDOMAIN, this throws pfblockerng into despair as it then starts to answer

Not something I said, and not what happens. DNSBL is not in a state of despair and does not then start to answer on subsequent queries for a domain that does not exist.

for example corpext.msitadfs.glbdns2.microsoft.com

when the name is on an active list you get the DNSBL response of 0.0.0.0 as the non-authoritative answer:
(the DNSBL just blocked it, it was told to return 0.0.0.0 and it does)

> corpext.msitadfs.glbdns2.microsoft.com
Server:         10.168.1.1
Address:        10.168.1.1#53

Non-authoritative answer:
Name:   corpext.msitadfs.glbdns2.microsoft.com
Address: 0.0.0.0
** server can't find corpext.msitadfs.glbdns2.microsoft.com: SERVFAIL

When it is not on a DNSBL list and name does not exist you get a direct response, but there is no resolution (no IP can be provided)

> corpext.msitadfs.glbdns2.microsoft.com
Server:         10.168.1.1
Address:        10.168.1.1#53

** server can't find corpext.msitadfs.glbdns2.microsoft.com: NXDOMAIN

visit any online dnslookup and they will tell you the domain does not exist.
here is the log from a site that does DNS
Screen Shot 2023-12-01 at 7.23.15 PM.png

it does not matter what its resolver is doing, or the forwarder or the actual dns record on the internet

yeah, actually it does.

when a domain name is in a blocklist…Pihole just blocks

and so does DNSBL from the pfBlockerNG, when name is on an active list, as shown above, you get 0.0.0.0

of course, if you prefer pihole there is no reason you can't continue to use it for your DNS blocking/resolution, in combination with pfsense. There are 100's of how-to's on the subject.

I, like many others, have not had a problem with the DNSBL it just does what you tell it to do - block
and yes, I did say earlier that I can recreate the issue you see, but in order to do that I had to change settings and force the system to do bad things in the first place.

there are still a couple of outstanding requests for you to provide your configuration information, (parts that you haven't already provided) and you have not, so not sure how else we can continue. If your settings are correct it will work for you, just like it does for me.

Thanks
Have a great weekend.

vjizzle

I really appreciate all the help. Next week when I am at home I will dive into this again. In the meantime Pihole just the job.