Unbound - If you enable the python module, is unbound restarting on you one or more times per day?

OffstageRoller

If you're running Unbound with the Python Module enabled, can you check your logs to see if Unbound is restarting?

You would see this in your logs:

info: service stopped (unbound 1.17.1).

I was running 22.05 with the Python Module enabled and using pfBlockgerNG-devel, and now I'm running 23.01 with pfBlockerNG. In both cases, I noticed in my unbound service would restart 1 to 11 times per day. It happens at, what appears to be, random times as well. So it's not like it's a pfBlockerNG cron job running at 2 AM.

I ship all my logs to Splunk, and here's a screenshot of my last 14 days looking for unbound restarting:

Can you guess when I turned off the Python Module :)? For the last 7 days, I've had 0 restarts of unbound. Prior to that, I'd get 1 to 11 restarts per day. I saw the same thing with pfSense 22.05 running Unbound version 1.15.0.

My problem is solved for now, but it would be nice to use the Python Module for additional functionality in pfBlockerNG.

I'm mostly posting this here to see if others that use that module can check their logs and also see if there are restarts. If there are, I can open a redmine ticket, but I don't want to waste anyones time if this ends up being unique to my setup and maybe how I have pfBlockerNG configured.

Gertjan

@offstageroller

Good question

But no. It doesn't.
I say this because I 'think' I can see what it does. And better, we can all see what it does, as it's written in Python, so very open source. Here it is : /var/unbound/pfb_unbound.py

The resolver, unbound, has many options, and one of them is : everybody can write 'plugins' (like a package) for it to extend it's functionality.
A guys called @BBcan177 said to himself : "wait, I want to filter DNs requests, how can I do this with unbound, pfSense's resolver ?"
Many iterations later, he wrote a plugin for unbound, the so called "python module".
It has some set up code, and a main loop that hooks into the main resolver loop, called for every type of DNS request.
It always terminates with two possible outcomes : yes, request accepted - or no, request denied.
pfb_unbound.py can not indicate to the parent, calling process, unbound : "you have to restart" as that is not it's job.

pfBlockerng is more then a tool that puts in place the file /var/unbound/pfb_unbound.py and inserts it into the unbound's config file :

.....
# Python Module
python:
python-script: pfb_unbound.py

pfBlockerng also downloads all the IP and DNSBL feeds, and re downloads them with the frequency you told it to do. If none were actually changed, pfBlockerng is smart enough to do nothing = not restart unbound.
Btw : IP feeds are merged in on big IP block file, stashed in one big IP alias file, and the firewall is reloaded.
DNSBL feeds : same thing, they are all placed in one file, doubles are removed, and if this file changed compared to the existing one, unbound is restarted.
So, pfBlockerng can restart unbound.
Unbound uses the python script file /var/unbound/pfb_unbound.py
And this python script file uses the big DNSBL list.

Non python mode means : it's unbound itself that reads in the big DNSBL list. It can only do so so upon process start.
"Python mode" has two major advantages :
It's way faster.
It permits to do more filtering then what unbound itself offer, like brute regex filtering, AAAA filtering etc etc etc.

unbound isn't only restarted by pfBlockerng.
if your WAN interface changed it's IP, or there was a NIC event : unbound gets restarted
Like : you hooked up a LAN port (laos a : unbound gets restarted.
You can even activate : on every LAN DHCP request, restart unbound
There are probably more events that can make pfSense decide to restart unbound, as pfSense can restart nearly any process if needed.
The same thing goes for the web server GUI, or NTP, or any other process.

It's not a bad thing that unbound restarts ones in a while.
When unbound restarts under the control of pfBlockerng, it was asked to dump it's DNS cache first to a fime. When done, it stops unbound, starts it again and informs it to read the DNS cache file back in, so precious cache info isn't lost.

Mine also doesn't restart for days, and then it restarted 4 times yesterday. Because I changed a setting ? A DNSBL list got updated ? An interface event ? I didn't check.

You can see here :

when pfBlockerNG restarts unbound.

You can test this :

@offstageroller said in Unbound - If you enable the python module, is unbound restarting on you one or more times per day?:

Can you guess when I turned off the Python Module :)? For the last 7 days, I've had 0 restarts of unbound. Prior to that, I'd get 1 to 11 restarts per day. I saw the same thing with pfSense 22.05 running Unbound version 1.15.0.

For every DNSBL feed : de activate the download frequency. Or stop all functionality that needs to download something from somewhere.
This means the master DNSBL list doesn't change anymore.
You'll see what happens : pfBlockerNG won't restart unbound anymore as there is no need to do so.
But this doesn't mean unbound won't restart anymore as other reason exists.

OffstageRoller

@gertjan Thank you for the detailed reply!

It's not a bad thing that unbound restarts ones in a while.

It is a bad thing though no? When the resolver is down, DNS doesn't resolve, and therefor the internet is "down" for users.

In many cases, this restart process takes 7-10 seconds, so I doubt most people notice it. But if you make a DNS request during one of these periods, you won't get a response. That's how I noticed this myself (unable to resolve DNS), and then I started looking into the logs and noticed the resolver had restarted.

I have still yet to have a restart of Unbound after disabling the python module. I do have the Resolver Live Sync option enabled in pfBlockerNG.

So if I disable the python module, unbound no longer reboots (hasn't happened in over a week). But if I enabled the python module, I get 1-11 restarts of unbound a day (where DNS will not resolve for my users).

Gertjan

@offstageroller

That there is no DNS aviable for pfSense and the LANs during resolver, is normal.
One of the reasons python mode was 'invented' was : it permits unbound to restart way faster.

When python mode isn't used, it's unbound itself, upon start, reads the flat file will all DNSBL host names. this file can be very big.
Again : python mode doesn't restart unbound more or less often. When used, it only permits a faster restart.

Btw : I restarted my OpenVPN client process : that also restarts unbound ...

@offstageroller said in Unbound - If you enable the python module, is unbound restarting on you one or more times per day?:

But if I enabled the python module, I get 1-11 restarts of unbound a day

I'm using python mode : my last restart was 3 days ago ;)

Your mission : why is unbound getting restarted ? Check the main system log and others for events that happen just before or at the moment unbound was restarted.

OffstageRoller

@gertjan Thank you again for the reply.

I was able to run pfBlockerNG in Unbound mode for over 2 weeks without a single restart of Unbound. This is with the live reload option checked.

I switched back to Unbound python mode for the last couple of days, and set my pfBlockerNG cron settings to Once a day and the start hour of 6 for 6 AM.

Sure enough, every day at around 6:02 AM, Unbound restarts. Yes, it restarts quickly (usually within 5-10 seconds), but there is a brief period of time where Unbound will not respond to DNS requests.

I don't see the option to live reload unbound when using Unbound python mode, but I believe that would address this issue. I would be able to keep Unbound running with live reloads when pfBlockerNG updates, while getting the other performance benefits python mode provides.

But as it stands right now, normal Unbound mode with live reload enabled, provides more reliable DNS service to my users. And when I use this, I can run my pfBlockerNG hourly like I used to, since I know I won't be breaking DNS for my users when there's an update. But with Unbound python mode, DNS will go down, so I need to limit this to once a day and do it when everyone is sleeping.

Not sure if @BBcan177 has any insight into this or could speak to live reloading being an option when running Unbound python mode.

hovnetworks

@offstageroller Hello

I ran into this issue and what fixed it for me was when I changed unbound to unbound python.

I noticed that whenever I made changes and needed to update or reload anything with PFBlocker, it would also do the same thing to me as well as the normal scheduled updates it does via cron job. I believe one of the main reasons is because it restarts/reloads everything a bit quicker than normal but whatever it is, this fixed it for me. No more losing connections whenever I make changes and updates nor have I seen anymore disconnections in the logs.