Upgraded to 2.5.1 - Unbound DNS stops working

mods

Roughly 24hr after upgrading from 2.5.0 to 2.5.1, Unbound stopped working (red X in services status widget).
Only 1x package installed : OpenVPN Export
I have DNSSEC and register DHCP/OVPN clients enabled.

The last log entries before Unbound service stopped from Status > System Logs > System > DNS Resolver:

Apr 14 14:20:56 unbound 28876 [28876:2] error: read (in tcp s): Connection refused for 199.249.120.1 port 53
Apr 14 14:20:56 unbound 28876 [28876:2] error: read (in tcp s): Connection refused for 199.249.120.1 port 53
Apr 14 14:20:55 unbound 28876 [28876:2] error: read (in tcp s): Connection refused for 199.249.120.1 port 53
Apr 14 14:20:55 unbound 28876 [28876:2] error: read (in tcp s): Connection refused for 199.249.112.1 port 53

Very interesting because @GregBinSD and @CTMarsh report the exact same issue even same odd external IP (199.249.112.1) in their logs in below thread (again, I do not have pfBlocker installed):

https://forum.netgate.com/topic/161707/pfblockerng-devel-v3-0-0-10-causes-internet-outage-on-sg-3100-at-school

Gertjan

@mods said in Upgraded to 2.5.1 - Unbound DNS stops working:

read (in tcp s): Connection refused for 199.249.120.1 port 53

and 199.249.120.1 is b2.org.afilias-nst.org. : that not just somebody, it's a root DNS server or one of it's CDN.
These do not speak TLS (SSL).

Yep : https://github.com/NLnetLabs/unbound/issues/360

Strange, why does unbound wants to speak TLS to such a DNS server ....

Are you forwarding ? If so, shut down a zillion lines of code, by deactivating DNSSSEC. You have a MITM, so DNSSEC is useless anyway.

@mods said in Upgraded to 2.5.1 - Unbound DNS stops working:

and register DHCP/OVPN clients enabled.

Short answer : don't. This option shoots unbound in the head for every lease that comes in.
Probably ok if you have 1 or 2 devices in your network(s).
If you have this DHCP clients that rail-guns DHCP requests, you blow your DNS (unbound) out of the water.
Static DHCP leases have no issues with unbound.

GregBinSD

@gertjan @mods

Gertjan, thanks for the advice on not registering hosts. I'm trying it now, hope it helps.

Stewart

@gregbinsd

Did any of this help? I have a unit that I just installed with 2.5.1 having this issue.

GregBinSD

@stewart
I am still running 2.4.5-RELEASE-p1 because of resolver issues on the SG-3100 for the newer supported releases. Netgate support helped me back out the 21.02 version for ARM CPUs.

So I am happy to stay on the old version because of it's stability. However, I had turned on a couple of features that I thought would be helpful, but was experiencing occasional DNS outages that lasted several seconds. I searched the forum for similar issues and found this topic. I turned off the 2 features, and now there are no more intermittent DNS outages, so I popped off a post to say thanks to Gertjan for his good advice.

planedrop

Just wanted to comment here to say I'm having the same issue on a custom install, seems Unbound can't access that IP and it's bombarding it periodically which seems to trigger Unbound to crash.

Going to try disabling registration of DHCP leases and see if that makes it a bit more stable but I don't think that is the primary issue here.

Stewart

@planedrop I disabled those and added Unbound to the watchdog. Client hasn't called and complained since. Not sure if it fixed it but it at least fixed it enough that it's working. I see there is a regression for Unbound in the next version. May be related.

planedrop

@stewart Good to know, I will go ahead and give this a shot then.

mods

@Stewart @planedrop
Sort of based on @Gertjan suggestion...
I disabled DNSSEC, and enabled Forwarding and SSL/TLS.
I believe changing to forwarding mode is what resolved the issue.
All other options are still enabled - registering DHCP/Reservations/OpenVPN clients, and have not seen the issue again across 4 different pfSense deployments.

SteveITS

If you're still on 2.5.1, note there is a stability fix for unbound in 2.5.2. (and 21.05)

Edit: I was thinking of the 21.05 release notes, I guess 2.5.2 isn't quite out yet but apparently soon...

planedrop

@mods I definitely prefer to use root servers for my setup personally. If disabling DHCP registration and then having the Watchdog keep track of it works then I'm OK with it personally. If I still have issues I will try this.

I imagine changing to forwarding mode helps, as it's getting such a huge log file built up of that one root IP not being accessible.

But if watchdog can restart it when it goes down then things should be ok.

planedrop

@steveits This is good to see, I guess I could try the RC here soon as this isn't on a prod firewall.

Stewart

@steveits said in Upgraded to 2.5.1 - Unbound DNS stops working:

note there is a [stability fix for unbound in 2.5.2]

That's what I was referring to but it isn't ready yet from what I can see.

planedrop

@stewart yeah I might give it a shot anyway since it's RC and this is non-prod. Not sure yet though as stability does still matter to me quite a lot.