DNSBL Stops DNS Service (Solved)

[Visseroth]

PfBlocker is stopping my DNS service.
So there was a problem I couldn't overcome nor put up with in the past with PfBlocker's DNS Blocker where it would constantly reload the DNS service if a new client connected or a DHCP address was renewed.
It was constantly causing DNS to not be available until it was done reloading, which took to long and was annoying, specially when you're surfing the web and suddenly DNS is down for the next 10 seconds or so until it reloaded again and was down again until it finished reloading.

After giving it another go I noticed this problem still persisted except now there was a option for "Resolver Live Sync".
Tried it to no effect, DNS would still reload causing pauses in DNS.
Then I tried Unbound python mode, which does seem to be working but this evening DNS stopped so I took a look at the logs and found 14 stops in what is available in the logs and right before the stop was...
"info: generate keytag query _ta-4f66. NULL IN"

I have no idea what that is all about. Generate what keytay query? and what is ta-4f66? I assume Null is dead but dead in what? No idea.

Obviously, the only way I could enable Unbound python mode was to enable python mode in DNS Resolver and for some reason you still can't have DHCP registration in DNS Resolver enabled at the same time as DNSBL, which kind of sucks for any DHCP device you want to contact via dns.

I do really like DNSBL and would like to keep it running but I'd rather not have to sack my DHCP DNS resolution to do it, it comes in handy every now and then.
Yes, any devices I contact regularly are staticly set but, as I said, DHCP DNS registration comes in handy periodically.

Anyhow, hopefully you get a idea as to what is going on.
My question is...
Is there a way to have DNSBlocker and DHCP DNS Registration enabled without causing DNS to reload all the time there by causing network interruptions?
Is there a better way to configure DNSBlocker than what I'm doing?

Granted you don't have the whole picture here so if you need to know something feel free to ask because I don't know what you need to know to get the bast answers possible but I think I painted a decent picture as to what is going on.

Constructive input and criticism welcome and appreciated.

[dma_pf]

@visseroth said in DNSBL Stops DNS Service:

Obviously, the only way I could enable Unbound python mode was to enable python mode in DNS Resolver and for some reason you still can't have DHCP registration in DNS Resolver enabled at the same time as DNSBL, which kind of sucks for any DHCP device you want to contact via dns

This is still an issue. My understanding is that it is an upstream issue with the way that unbound works which pfsesne's developer can't work around. Your only solution is to do what you were suggesting by assigning static ips to clients you need to reach regularly by their host name.

[Uglybrian]

@visseroth Hi,
If I understand your problem with correctly, if you have ''DHCP Registration'' checked in DNS Resolver General Settings you will need to uncheck it to stop the reload.

[NollipfSense]

@visseroth Do you have a firewall rule to force all LAN net to use your pfSense for all DNS request? if not, that's the solution.

[bmeeks]

@nollipfsense said in DNSBL Stops DNS Service:

@visseroth Do you have a firewall rule to force all LAN net to use your pfSense for all DNS request? if not, that's the solution.

How does this stop the unbound resolver from restarting? His problem is DHCP updates to DNS causing a restart of unbound, and because he has large lists of DNSBL domains, it takes a very long time for unbound to successfully restart. And during the restarting interval DNS resolution on the firewall is halted.

This has nothing at all to do with whether his clients are querying pfSense or not (except that if his clients queried another DNS server, then they would never know unbound was not working -- of course then all the features of DNSBL would not work either).

[Visseroth]

@uglybrian I did, I no longer have Register DHCP leases in the DNS Resolver checked which did stop the reloading.

@NollipfSense I do have DNS rules on my interfaces to force redirect DNS resolution at the firewall for ports 53 and 853. Obviously DNS on any other port with get by the rules.

[bmeeks]

@visseroth said in DNSBL Stops DNS Service:

@uglybrian I did, I no longer have Register DHCP leases in the DNS Resolver checked which did stop the reloading.

I believe another issue occurs when an IP or domain list is updated by the cron task. Although I don't use pfBlockerNG-devel personally, my understanding of the code is that when the lists are updated, the resolver is restarted to ingest the new lists. With large lists, the restart time can be long.

There are also certain events at the pfSense system level that result in pfSense issuing a "restart all packages" command. That command will result in the resolver being restarted, and again, if large domain and IP lists are in use, the restart time can be excessive.

[NollipfSense]

@bmeeks said in DNSBL Stops DNS Service:

@nollipfsense said in DNSBL Stops DNS Service:

@visseroth Do you have a firewall rule to force all LAN net to use your pfSense for all DNS request? if not, that's the solution.

How does this stop the unbound resolver from restarting? His problem is DHCP updates to DNS causing a restart of unbound, and because he has large lists of DNSBL domains, it takes a very long time for unbound to successfully restart. And during the restarting interval DNS resolution on the firewall is halted.

This has nothing at all to do with whether his clients are querying pfSense or not (except that if his clients queried another DNS server, then they would never know unbound was not working -- of course then all the features of DNSBL would not work either).

Okay, I was just asking though.

[bmeeks]

@nollipfsense said in DNSBL Stops DNS Service:

@bmeeks said in DNSBL Stops DNS Service:

@nollipfsense said in DNSBL Stops DNS Service:

@visseroth Do you have a firewall rule to force all LAN net to use your pfSense for all DNS request? if not, that's the solution.

How does this stop the unbound resolver from restarting? His problem is DHCP updates to DNS causing a restart of unbound, and because he has large lists of DNSBL domains, it takes a very long time for unbound to successfully restart. And during the restarting interval DNS resolution on the firewall is halted.

This has nothing at all to do with whether his clients are querying pfSense or not (except that if his clients queried another DNS server, then they would never know unbound was not working -- of course then all the features of DNSBL would not work either).

Okay, I was just asking though.

The last sentence of your post said: if not, that's the solution., so it sounded like you were giving the advice as the solution to his posted problem.

[Gertjan]

@visseroth said in DNSBL Stops DNS Service:

PfBlocker is stopping my DNS service.

PfBlocker by itself stops nothing.It's for, 95 % just a lot of GUI script that doesn't get executed when you do not visit the pfSense GUI.
But then the 'admin' of pfSense does 2 things :
She/He adds 'feeds' with IPs and DNSBL. Upon activation, these lists will be downloaded and 'put in place'. Then 'unbound' has to be made aware that it should should use these lists.
Well .... unbound has to be reloaded = stopped, and then started.
This only happens ones.
And now the admin sets up PfBlocker with these instructions : download the lists/feeds every hour again .....

Now I didn't check if PfBlocker is mart enough not to download a list that didn't change, I guess it doesn't.
But some lists (the ones that are actively maintained = change very often) will get re downloaded : new list : unbound has to be made aware .... well, you know what will happen.
Why not check even more often ?? Or - be smart : less often ?

So : there are choices to make : do you want unbound to get shot gunned every hour ?
Or do you prefer ones a day ? Or just use less lists/feeds, and have it restart one or twice a week.
It's up to you.
And even more important : you have to know it's up to you.

@visseroth said in DNSBL Stops DNS Service:

which kind of sucks for any DHCP device you want to contact via dns.

This was solved some decades ago.
The solution proposed didn't not change, so I'll consider it's still valid.

Most devices that connect occasionally to one of your local networks, the BYODs : you don't care what IP - or host name, they have. These are bandwidth 'consumers', and don't expose services to other connected devices. try to nmap your iPhone : not one port replies ( = 'open').
"Server" type devices like NAS, printer, camera etc, you do want to know what address they have, or better : you want to give them a "easy to remember" host name.
Assigning devices with static IP, host names etc won't help you here.
The DHCP server all by itself without any setup doesn't help you neither : it does what is paid to do : handing out a gateway, a DNS, an IP and 'network mask' to devices that ask for this info.

You should (that is : 'consider' or 'I advise') add all devices that you need to access here :

This will take you half a minute per device, and you only have to do this ones in a live time (of the device).
When done, on the Resolver settings page :

and no more useless resolver (DNS) restarts.

True, DHCP lease / Unbound handling is somewhat 'broken'. But even if it was working without the unbound restarts, I would add MAC DHCP leases for every important device in my network.

The Wifi devices on my non trusted networks like "iPhone", "iPhone", "iPhone", "iPhoneMark, "HUAWEI_Mate_10_lite-37800", "android-7c8c5ddff4b5ec3", "Galaxy-S9" etc (got hundreds of them) : I don't care. I never 'connect' to them.

I know, if you have hundreds of works stations in your trusted LAN's, this can be tedious to do. But some sort of 'map' or plan or list with known connected devices has to be made anyway. Well, do so in pfSense.

Btw : there is another resolver, called bind (or named) that can re reads it's config files like /etc/hosts all by itself when it detects that these files changed (because some other process changed them).
Well, nothings stops you from stopping the Resolver (unbound) and use 'bind'. It's a pfsense package.
But, there is a price to pay : bind, as a resolver, isn't as well integrated into pfSense as unbound- packages as pfBlockerNG won't work with bind. And last but not least : when you use bind you have to stop thinking that that you understand what 'DNS' is. You will know what 'DNS' is.
And when you know what DNS is, you'll stop using bind right away. Because live is already hard as it is ;)
You use bind if you can't do without it - like running your own domain name server, or because you want to handle these pesky DNSSEC records in your own zone yourself. Or because you have a lot (like a lot) time left and like to learn about "DNS" ...

@visseroth said in DNSBL Stops DNS Service:

Then I tried Unbound python mode

Stick with Python mode.
BBcan177 wrote somewhere why this 'python' approach is far better. He had to use it : no choice. Because NLnet LABS said so. They (NLnet) could have chosen LUA - or classic sh/bash shell scripting, or whatever.
It became "Python". This video shows us some of the "why" part.

With the python script, the entire inner working process of the DNS resolver became accessible. It's far more easier now to trace all these DNS requests and answers - and logging.
And : reacting (answering 'No') on requests that have a match in one of the loaded DNSBL lists. With "Python mode" it's the python scripts that loads and uses our DNSBL lists, not unbound.

---

Some words about :

@visseroth said in DNSBL Stops DNS Service:

"info: generate keytag query _ta-4f66. NULL IN"

I'll start at the bottom.
Look here : Trust Anchors and Keys.
Now look at the paper on the right, as the left dates from 2010, an has been replaced by the one on the right.
This one is signed, the old fashioned way.
Look closer :

The hash 20326 decimal is 4f66 in hexadecimal.

Before unbound starts, another application is start. Its called "unbound-anchor".
It loads the root trusted certificate file, so unbound can do "DNSSEC" for us, if needed.

To make a long story short, when you see :

"info: generate keytag query _ta-4f66. NULL IN"

in the log file, you know unbound is about to get started.

edit : The next time 4f66 changes for some other 4 digit hex code, big news media will warn you upfront ;)

[NollipfSense]

@bmeeks I forgot that part...was foolishly trying to help without fully reading the post and instead relying on the buzzword DNS.

[Visseroth]

@bmeeks Sorry bud, I just realized after reviewing this thread I completely ignored you unknowingly. I apologize.

To answer your question, yes, I have port 53 and 853 redirect rules in place on all LANs to grab all DNS traffic and send it to my firewall for resolution.

Something I did a while back to try to keep potentially infected machines from resolving anywhere other than the firewall.
That being said it is still possible to use another non-standard port or tunnel.
Hopefully Snort will catch potentially bad traffic and block it.

[Visseroth]

@gertjan That's a lot of good information!
Thanks for taking the time to post and clarify some things!
And yes, I do static map quite a few things, primarily any server style device (a device providing a service) and yes, static mapping makes sense and should be the standard.
It's something I have to do on printers all the time because people just plug it in and go and wonder why it doesn't work after it's been turned off all weekend.

But I digress, per the advice of this forum I have disabled DHCP registration and PfBlocker works great.
I haven't seen any ads on my network since, it's been NICE!
Still a few blocks to get sorted but it is a nice to have feature!

And to everyone else that chimed in, I thank you too! Even if you don't think you were helpful you were and I think most of us learned a few new things as well, if not I at least did.

[the other]

Hi there,
just to nail down that one question in @Gertjan *s helpful post:
pfblockerng_dev (do not know about the other one) does NOT reload a list from servers if there are noch changes.
It seems "smart" enough to recognize a change in the list.
No changed list > no download (at least that's what the log says...)
:)

[Gertjan]

@the-other said in DNSBL Stops DNS Service (Solved):

pfblockerng_dev (do not know about the other one) does NOT reload a list from servers if there are noch changes.
It seems "smart" enough to recognize a change in the list.
No changed list > no download (at least that's what the log says...

I hope so, I'm not so sure.

File attributes, size, last modified time stamp etc are needed before the file gets downloaded again.
But :
/usr/local/pkg/pfblockerng/pfblockerng.inc line 3373 :

			if (($fhandle = @fopen("{$file_dwn}.raw", 'w')) !== FALSE) {

The local destination file is opened for writing - so initial file size date etc are lost : CURL doesn't cache by itself : the file can only be re downloaded at this stage.

Also :
/usr/local/pkg/pfblockerng/pfblockerng.inc line 170 :

				CURLOPT_FRESH_CONNECT	=> true

Now read Is there a way to tell curl to not use cache

edit :
I forget something : most feeds are https://..... and default TLS web server caching is : no caching.
So even if you, on the receiving side, are ok to receive a cached version, you still get the entire file again.

Btw :less used download methods like rsync are version/date/time aware.