Unbound crashes periodically with signal 11

Salander27 0 · Feb 20, 2021, 9:43 PM

Alright, it's been about 2 and a half hours since I installed unbound-1.13.1 on my system. I have stress-tested it by reducing the DHCP lease time to 120 seconds and have since seen hundreds of DHCP renewals (and subsequent unbound HUPs). Were this unbound 1.13.0 I would have likely seen dozens of crashes, however unbound 1.13.1 has been completely stable in that time.

At this point I believe the best course of option is for additional people to test to see if their issue is resolved with the updated version. If you are willing to take system backups and accept the (IMO very low) risk that you may need to reinstall pfSense completely if something goes wrong (so don't do this on your prod systems) then please follow the instructions in my previous comment to install unbound-1.13.1.

hans109h · Feb 20, 2021, 10:33 PM

@salander27-0 I'm having similar results that you report. DHCP registration is turned back on, and no crashes so far. 1.13.1 seems to be an upstream solution.

noviceiii · Feb 20, 2021, 11:14 PM

I had a stable unbound service again after flushing all DHCP leases.

SOLUTION (in my case): FLUSH DHCP Leases

Details/ Follow-up is here:
https://forum.netgate.com/topic/161092/2-5-0-dns-service-stopping-randomly/5?_=1613861976462

(sorry, if we may have opened a very similar thread to this topic)

khuynh · Feb 21, 2021, 11:06 AM

@salander27-0 Thanks! This was really helpful. Seems to be working perfectly now.

noviceiii · Feb 21, 2021, 11:48 AM

@khuynh very well. Glad to help. Hit „like“ on the solution and spread the news.

fivetoedslothbear · Feb 21, 2021, 7:27 PM

@salander27-0 Thanks! That fixed it. I forced short leases to cause a lot of renewals, and since I installed unbound-1.13.1, I've had no crashes going on 18 hours now.

Salander27 0 · Feb 21, 2021, 8:20 PM

@fivetoedslothbear Yeah, I'm just past 24 hours myself without any crashes.

Also, to anyone who installed 1.13.1 please continue to follow this thread as you may need to manually install the patched unbound from the stable repositories if/when a patched version is pushed.

Dexypher · Feb 21, 2021, 10:33 PM

@salander27-0 Been having the same issue for the last 24 hours -- will try this solution and see if it works for me. I appreciate it mate. Have a good one.

slu · Feb 22, 2021, 10:22 AM

I see that also on two 2.5.0 CE machines.
Disable "Register DHCP leases in the DNS Resolver" now and will have an eye on it.

cnaumer · Feb 22, 2021, 3:23 PM

@salander27-0 :I tried this. Will report back. However, I never had "Register DHCP leases in the DNS Resolver" set.

jimp · Feb 22, 2021, 5:44 PM

Since the new version of Unbound fixes it, it's unlikely to actually be related to just that one setting (DHCP lease registration), but that is the fastest way to trigger it for some people.

I imagine others are/were hitting it as well in different ways. So there isn't a need to keep tracking potential causes now that we know the upgrade fixes it.

Salander27 0 · Feb 22, 2021, 8:30 PM

@jimp So is the plan then to push 1.13.1 to the stable repo or to try to bisect through the 1.13.1 release in order to find out which patch specifically fixes the issue and just apply that patch to the version in stable?

jimp · Feb 22, 2021, 8:33 PM

@salander27-0 said in Unbound crashes periodically with signal 11:

@jimp So is the plan then to push 1.13.1 to the stable repo or to try to bisect through the 1.13.1 release in order to find out which patch specifically fixes the issue and just apply that patch to the version in stable?

We'll bring in 1.13.1, there isn't a compelling case to do all the legwork to pick in partial changes at this point. 1.13.1 is a minor patch/bug fix release and the impact is low other than the fix for this which is highly beneficial.

Salander27 0 · Feb 22, 2021, 8:47 PM

@jimp Speak of the devil, looks like it's already been added to the stable repos.

For anyone coming into this thread now, you can run the following command to pull unbound 1.13.1 from the stable/2.5.x repo (I am unsure of how to update system packages from the UI, hopefully someone can chime in there):

pkg upgrade -f unbound

(you should see that it is installing unbound-1.13.1)
Make sure to restart unbound after this package installation.

It is probably a good idea for those who have installed the devel package to do this as well just to ensure that your systems are not a mix of devel and stable packages.

jimp · Feb 23, 2021, 9:35 AM

There isn't a good way to do that from the GUI, but you could run

pkg upgrade -fy unbound from Diagnostics > Command Prompt. Be sure to restart the Unbound service from Status > Services after.

From the CLI the easiest way to do both is

pkg upgrade -fy unbound; pfSsh.php playback svc restart unbound

cnaumer · Feb 23, 2021, 7:53 AM

@jimp : I can confirm that the new version fixes it for us.

gorgonbert · Feb 23, 2021, 8:35 AM

@salander27-0 fixed it for me too... thank you

slu · Feb 23, 2021, 9:35 AM

@jimp

thank you Jim.
This open and direct communication is really awesome!

johnt · Feb 23, 2021, 10:29 AM

Fixed for me too, 24hrs with no unbound restarts...

jmkofoed · Feb 23, 2021, 2:44 PM

Hi

After installing unbound-1.13.1 my unbound is still restarting from time to time.
My system have been unstable for a long time, but I haven't had the time to dig into it. Last week I figured out that it was the unbound which is restarting. So I updated to version 2.5. But still having issues.
I have "DHCP Registration", "Static DHCP" and "OpenVPN Clients" enabled in my DNS Resolver settings.
I will try to test further and report back
Kind regards
Jens M. Kofoed