Unbound crashes periodically with signal 11

jkv · Feb 20, 2021, 6:00 AM

I also had the “ Register DHCP leases in the DNS Resolver” option enabled when seeing this issue and disabling this option appears to have stopped the crash.

maverick_slo · Feb 20, 2021, 6:02 AM

@jkv +1
No packages at all...

Salander27 0 · Feb 20, 2021, 7:20 AM

@fry-kun I suspect you're seeing crashes approximately whenever certain devices renew their DHCP lease. Since the timing of this has more to do with when that device was turned on you would expect to see somewhat random delays between such renewals.

Question for those experiencing these crashes, do you have both "Register DHCP static mappings in the DNS Resolver" enabled AND DHCP Static Mappings where at least one mapping has something filled in in the hostname field?

Fry-kun · Feb 20, 2021, 7:27 AM

@salander27-0 said in Unbound crashes periodically with signal 11:

Question for those experiencing these crashes, do you have both "Register DHCP static mappings in the DNS Resolver" enabled AND DHCP Static Mappings where at least one mapping has something filled in in the hostname field?

Affirmative

tuxbox78 · Feb 20, 2021, 12:59 PM

@salander27-0 Yes, same here on my system..both were enabled and have quite some manual mappings in my system. After upgrade to 2.5 it seems unbound crashed about every ten minutes, after disabling register DHCP mapping it stopped crashing.

My System is based on a standalone AMD GX-412TC SOC having run pfSense stable for years now.

e1219 · Feb 20, 2021, 2:44 PM

@salander27-0 I can confirm this as well, I have this option set with a handful of static mappings with hostname specified.

jkv · Feb 20, 2021, 3:11 PM

@salander27-0

confirmed

Salander27 0 · Feb 22, 2021, 8:50 PM

@jimp I have successfully downloaded and installed unbound-1.13.1 from the devel repo now that the package has been built. No crashes yet, but none of the DHCP leases of my static leases (with hostname) have expired yet so I wouldn't expect crashes yet. I will update in a few hours once those leases start expiring.

OBSOLETE PLEASE SEE https://forum.netgate.com/post/966915

For anyone else who wishes to try unbound 1.13.1 (normal caveats about this being unsupported and at your own risk) I simplified the install command to the following:

pkg add -f https://files01.netgate.com/packages/pfSense_master_amd64-pfSense_devel/All/unbound-1.13.1.txz

You can roll back with:

pkg install -f unbound

After either of these commands you will need to restart the unbound service to pick up on the new binary.

OBSOLETE PLEASE SEE https://forum.netgate.com/post/966915

Fry-kun · Feb 20, 2021, 9:42 PM

@salander27-0 been running 1.13.1 for ~2hrs, no crashes yet!
1.13.0 crashed way more often, as evidenced in my earlier message

Salander27 0 · Feb 20, 2021, 9:43 PM

Alright, it's been about 2 and a half hours since I installed unbound-1.13.1 on my system. I have stress-tested it by reducing the DHCP lease time to 120 seconds and have since seen hundreds of DHCP renewals (and subsequent unbound HUPs). Were this unbound 1.13.0 I would have likely seen dozens of crashes, however unbound 1.13.1 has been completely stable in that time.

At this point I believe the best course of option is for additional people to test to see if their issue is resolved with the updated version. If you are willing to take system backups and accept the (IMO very low) risk that you may need to reinstall pfSense completely if something goes wrong (so don't do this on your prod systems) then please follow the instructions in my previous comment to install unbound-1.13.1.

hans109h · Feb 20, 2021, 10:33 PM

@salander27-0 I'm having similar results that you report. DHCP registration is turned back on, and no crashes so far. 1.13.1 seems to be an upstream solution.

noviceiii · Feb 20, 2021, 11:14 PM

I had a stable unbound service again after flushing all DHCP leases.

SOLUTION (in my case): FLUSH DHCP Leases

Details/ Follow-up is here:
https://forum.netgate.com/topic/161092/2-5-0-dns-service-stopping-randomly/5?_=1613861976462

(sorry, if we may have opened a very similar thread to this topic)

khuynh · Feb 21, 2021, 11:06 AM

@salander27-0 Thanks! This was really helpful. Seems to be working perfectly now.

noviceiii · Feb 21, 2021, 11:48 AM

@khuynh very well. Glad to help. Hit „like“ on the solution and spread the news.

fivetoedslothbear · Feb 21, 2021, 7:27 PM

@salander27-0 Thanks! That fixed it. I forced short leases to cause a lot of renewals, and since I installed unbound-1.13.1, I've had no crashes going on 18 hours now.

Salander27 0 · Feb 21, 2021, 8:20 PM

@fivetoedslothbear Yeah, I'm just past 24 hours myself without any crashes.

Also, to anyone who installed 1.13.1 please continue to follow this thread as you may need to manually install the patched unbound from the stable repositories if/when a patched version is pushed.

Dexypher · Feb 21, 2021, 10:33 PM

@salander27-0 Been having the same issue for the last 24 hours -- will try this solution and see if it works for me. I appreciate it mate. Have a good one.

slu · Feb 22, 2021, 10:22 AM

I see that also on two 2.5.0 CE machines.
Disable "Register DHCP leases in the DNS Resolver" now and will have an eye on it.

cnaumer · Feb 22, 2021, 3:23 PM

@salander27-0 :I tried this. Will report back. However, I never had "Register DHCP leases in the DNS Resolver" set.

jimp · Feb 22, 2021, 5:44 PM

Since the new version of Unbound fixes it, it's unlikely to actually be related to just that one setting (DHCP lease registration), but that is the fastest way to trigger it for some people.

I imagine others are/were hitting it as well in different ways. So there isn't a need to keep tracking potential causes now that we know the upgrade fixes it.