23.01 Upgrade unbound Issue

SteveITS

@stephenw10 I thought I'd try to replicate it with linkedin.com just now and couldn't, at least by turning enabling DNSSEC and immediately (without waiting an hour or two) running nslookup. It was failing the other day because I was testing and resolved it.

Of course disabling DNSSEC also restarts unbound so now there's a question of what the problem actually is...does it show up after time? On certain queries? When the domain's NS has something misconfigured?

If it is a bug, then it occurs to me there's a question on the correct solution:

A) resolve the bug
B) automatically disable/hide DNSSEC when forwarding is enabled

Option B sounds like it is "more correct"...and maybe something that should be done regardless of any bug/not bug.

johnpoz

@steveits said in 23.01 Upgrade unbound Issue:

does it show up after time?

Most likely this - if your asking for dnssec when where you forward to is already doing dnssec.. And you get something returned via cache - because your forwarding, etc. As quad9 step 3 says about turning it off "can cause false dnssec failures" if dnssec fails then what your looking for would fail, etc.

If your forwarding, clearly you trust where your forwarding too. They are already doing dnssec as a resolver, which is where it is meant to be done, etc. I see no point in setting your unbound to try and do dnssec as well.. Its going to be flawed when your not resolving. At best its causing extra queries, at worse stuff is going to fail to resolve. There is no reason to do it.

edit: btw I have been saying to turn off dnssec if your forwarding for YEARS... here is a post from 2016, where I state to turn it off if your forwarding ;)

https://forum.netgate.com/post/627755

Might be posts from before that, prob all the way back to when unbound was just a package in pfsense and not built in.. dnssec has no use in forwarding mode..

SteveITS

@johnpoz said in 23.01 Upgrade unbound Issue:

I see no point in setting your unbound to try and do dnssec as well

Oh I'm not debating whether it should be off. :) I'm suggesting that since it should be off, when forwarding is enabled, then pfSense should turn it off. IOW, avoid foot-shootery.

I've seen maybe 10ish people complaining about DNS problems after installing 23.01 and it seems like it's DNSSEC and forwarding.

johnpoz

@steveits we are on the same page, and I understood your suggesting it should be off if you enable forwarding.. Completely agree with you.. Maybe we crossed some wires or something.. We are in lock step here - if the user clicks forwarding mode, the checkbox for dnssec should be unchecked and grayed out to be honest.

But that would draw complaints most likely as well ;) Maybe pop up another check box that says, by clicking this I understand what I am doing and want to do forwarding with dnssec.. I will not post any questions or threads about dns issues if I have this checked ;)

SteveITS

@johnpoz @stephenw10 I made a Redmine feature request.

I can't easily generate a default config file right now (wife is on a conference call ) but is DNSSEC enabled by default? I am pretty sure it is not, but it still seems safer to turn it off when forwarding is enabled.

jimp

The default configuration has Forwarding off and DNSSEC on. Someone enabling forwarding should probably disable DNSSEC manually but that isn't rejected by validation or changed automatically.

Gertjan

Forwarding is probably several msecs faster.
Are there other major advantages for not doing resolving ?
And thus not doing DNSSEC ?

I've just checked https://dnsviz.net/d/linkedin.com/dnssec/ - so they do DNSSEC.

I've asked my unbound : dig @127.0.0.1 linkedin.com +trace and saw that 'dnssec' flaf was set : the result was 100 % valid.

I mean, I was happy the day I didn't need to use these ISP DNS servers any more.
I could, finally, use the real thing : the root servers ! The ones that define DNS, and are close to core Internet. Resolving means : I need nobody, I mean not needed external resources, for 'DNS' to work. DNSSEC is just a nice bonus.

Who wants to inform me why I should consider 'forwarding' ?

johnpoz

@gertjan said in 23.01 Upgrade unbound Issue:

Who wants to inform me why I should consider 'forwarding' ?

Won't be me, you couldn't get me to forward my dns to anyone to be honest.. There is zero point to it if you can resolve. Only reason would might be if your on a horrible connection and just can not reliably resolve - say sat or something, or your on a connection that is messing or blocking dns other than to them, etc. vpn might do that - you know for your own good <rolleyes>

Forwarding is probably several msecs faster.

Who says - might there be a few ms longer to resolve something from scratch, ok.. sure.. But once have the gltds cached, and for that matter the authoritative ns for any specific domain. When I need to look up say host.somedomain.tld would go direct to that domains ns.. Which might/could be faster or slower than say some forwarders ns.

But that I can tell you for sure, is every query to some forwarder is going to take X ms, at best - where maybe slower when they have to resolve it, because its not in their cache, etc.

using a forwarder for faster lookups is not really a valid reason to use them.. At best your talking a few ms anyway - who cares? if you think 10ms here or there is making a difference in your overall experience in internet browsing - your not understanding how any of it works to be honest. Or how fast 10ms is ;) That is ms, not minutes not seconds.. .010 seconds..

Cylosoft

@gertjan It's another layer of filtering/security. Encrypted connection out to a known DNS provider. For filtering we have a couple of different tiers based on how much filtering we want the location to have. Quad9 for example does some basic DNS blocking. Some sites get NextDNS connections so IT can manage rules centrally there rather than running pfBlocker locally. Some get wide open Cloudflare DNS servers. Some get filtered Cloudflare.

jimp

Some ISPs break acting as a resovler in various ways as well, either rate limiting or address limiting who you can reach on common DNS ports, or other similar shenanigans.

DNS over TLS/DNS over HTTPS are OK if you really trust the provider on the other end. Those are for privacy and not for authentication/validation, though. If you forward you have to trust that the upstream DNS servers are also not changing your query results in unexpected ways.

SteveITS

@jimp I understand your logic. Is it therefore a regression in unbound, if it worked with DNSSEC enabled in prior pfSense versions? If the answer is, "it's not supposed to work" then I can understand that too. I'm just trying to help people out. Worst case, after people upgrade to 23.01 and have a problem then they won't have it enabled for future upgrades anyway. Perhaps a "if you are using forwarding and have problems disable this option" note on that setting, if it's not too much handholding.

jimp

The answer really depends entirely on the upstream resolver behavior, so there is no way to know. If we put a note on the option then it's likely to be missed since they'd have to already know which option they'd need to disable to see that or to know their problem is related to forwarding, which isn't obvious. Bit of a chicken/egg issue there.

This is already covered in the docs under troubleshooting DNS issues:

https://docs.netgate.com/pfsense/en/latest/troubleshooting/dns.html#check-dns-service

SteveITS

@jimp It just seemed like a lot of people were now/newly hitting it on 23.01 so unbound behavior apparently changed. I need to check that setting when we get to updating the rest of our, and all our clients, routers.
(Always good to read the docs. Not sure I cited it this thread but I have in others.)

stephenw10

Mmm, it does seem like something has changed there causing people to hit this who weren't before. I wonder if it's some secondary effect though like they were always hitting it but cached values were hiding it until the upgrade.

johnpoz

@stephenw10 I have not had time to look through all the changes in unbound. But were we not on like 1.15.something, and now we are on 1.17.1 - have to assume lots of changes to unbound in such a big jump in version numbers.

is it also possible that something changed with say quad9.. I had not noticed their actual recommendation to disable dnssec if you forward to them until recently where it was pointed out in a thread. Has that been their standing recommendation for a long time, or did they possible make some changes that now make it the dnssec failures hit more often? If using 1.17.1 unbound?

bingo600

I haven't noticed any issues on my 23.01 "Test Box"
And i even use DNSSEC w. Forwarding.
Making my 20 meters from the pfSense to my two linux DNS servers "super secure" ...

/Bingo

stephenw10

That's your own hosted DNS resolvers? Interesting. They support DNSSec I preseume.

It seems unlikely any upstream change, at Quad9, would have coincided with 23.01 release. But perhaps people just started checking after upgrading. I agree some behaviour change in Unbound seems most likely to me.

Steve

johnpoz

@stephenw10 I just did a breeze over of the changes and bug fixes in unbound from 1.15 to 1.17.1 and nothing jumped out at me.. But possible something that was fixed now triggers more failures that should of failed before but didn't, etc.

I despise forwarding, so while I will turn it on to show how it can be done. I have no desire to leave it on for any length of time to see if errors occur on specific domains I might be going to, etc. Or after a amount of time..

if someone could point out a specific query that fails when forwarding using dnssec and forwarding, but works fine with just forwarding to a dnssec resolver, be it over tls or just clear, etc. I would be up for looking into that.

bingo600

I'm only forwarding because:

1:
I already had a working linux DNS/DHCP setup , that does dynamic DHCP entries in DNS.

2:
Unbound DHCP adding "hosts", created unacceptable reload/delays.
I want my DHCP devices to be resolvable, and more important to have "sane names" in the DNS system.

DNSSEC all 20 meters on a local network ... Is kind'a "well it hasen't broke yet" , and my Bind9's are setup to accept it.

/Bingo

johnpoz

@bingo600 said in 23.01 Upgrade unbound Issue:

only forwarding because

Your forwarding to your own dns? What does it do then to resolve public dns. Forwarding to your own internal resolvers is different ball game than forwarding to some outside dns ;)