Slow DNS after 22.05

Kempain

Super interesting thread and a lot of knowledge here.

So if I understand correctly (which is a big if!), are these assumptions correct?

pfSense uses the 13 root servers by default and will only use the defined DNS servers (in general settings) for the firewall if local DNS fails AND you have fall back to remote set, OR if you have use remote ignore local set in DNS Resolution Behaviour?

DNS servers listed in general settings are only used by client devices if you enable DNS forwarding and/or DNS Query Forwarding (and your not using DNS Server Override)?

DNS Server Override will (typically) use your ISP's DNS servers rather than the 13 root servers unless you're using OpenVPN?

From a security standpoint I'm assuming it makes no sense to use anything other than local and to block remote failback?
The only question I'd have here is whether the 13 root servers all support DNSSEC etc.

Assuming one really only wanted to use external DNS servers such as Cloudflare or Quad9 for both the firewall and client devices, would the process would be to:

Add the DNS servers in general settings
Enable use remote, ignore local in resolution behaviour
Enable dns forwarding and dns query forwarding

Excuse anything I may have misunderstood above, just trying to get my head around it all as I definitely misunderstood these features previously (and may still do).

johnpoz

@kempain said in Slow DNS after 22.05:

The only question I'd have here is whether the 13 root servers all support DNSSEC etc.

Your not understanding the use of the 13 roots, those are just to get the ball rolling, they point to the gltld servers, ie which servers to talk to for the different tlds.

;com.                           IN      NS

;; ANSWER SECTION:
com.                    86400   IN      NS      d.gtld-servers.net.
com.                    86400   IN      NS      i.gtld-servers.net.
com.                    86400   IN      NS      c.gtld-servers.net.
com.                    86400   IN      NS      a.gtld-servers.net.
com.                    86400   IN      NS      e.gtld-servers.net.
com.                    86400   IN      NS      l.gtld-servers.net.
com.                    86400   IN      NS      j.gtld-servers.net.
com.                    86400   IN      NS      g.gtld-servers.net.
com.                    86400   IN      NS      h.gtld-servers.net.
com.                    86400   IN      NS      b.gtld-servers.net.
com.                    86400   IN      NS      k.gtld-servers.net.
com.                    86400   IN      NS      f.gtld-servers.net.
com.                    86400   IN      NS      m.gtld-servers.net.

;; QUESTION SECTION:
;org.                           IN      NS

;; ANSWER SECTION:
org.                    3600    IN      NS      b0.org.afilias-nst.org.
org.                    3600    IN      NS      b2.org.afilias-nst.org.
org.                    3600    IN      NS      c0.org.afilias-nst.info.
org.                    3600    IN      NS      d0.org.afilias-nst.org.
org.                    3600    IN      NS      a0.org.afilias-nst.info.
org.                    3600    IN      NS      a2.org.afilias-nst.info.

etc...

When you resolve you walk down the tree..

Hey roots who should I talk to for .net
hey .net servers who should I talk to for domain.net

Hey NS for domain.net what is the A record for www.domain.net

But yes roots have dnssec, but that doesn't mean all tlds support dnssec, nor does it mean the authoritative ns for domain.net has enabled dnssec.

If you want to use quad9, you wouldn't setup ignore local unless you didn't want to resolve any local sources. But yeah you would have to setup forwarding in unbound, or use the forwarder vs unbound.

As to the roots all supporting dnssec..

https://www.stackscale.com/blog/root-dnssec-ksk-ceremony/

The 13 roots servers are only authoritative for the root zone " . " this zone has the NS records for all the tlds NS..

Keep in mind it doesn't have to do this every time you ask for whatever.somedomain.tld - once it has asked the roots for NS of .tld then it caches those, and doesn't have to ask roots again for somethingelse.tld, it just asks .tld servers for ns of whateverdomain.tld until the TTL of those NS has expired.

Once it knows the NSers for whateverdomain.tld, it just ask one of the NS when you ask for www.whateverdomain.tld or serverX.whateverdomain.tld, etc.

The "roots" are only talked to get the NS for the .tld your asking about.

Gertjan

60 minutes to understand how the most important part of the internet works.

Remove the "how you think it works", replace it for : how it really works.

As soon as you understand something, you can 'see' where an issue is, change it, and do other, more interesting things.
= You win !!

If there are still 'DNS' questions or issues after these video's : stop dealing with DNS. Become a painter, or make bread, or help Boeing make the MAX or TLS better. Take your pick.

Most of these are understandable for a 12 year old. Most videos are available in most languages.

@Kempain I promise you : it's way more easier as you think. It's 'old' technology from the '60 and '70, last century. No nuclear fusion tricks are involved. There is a boatload of 'keep it simple' going on here.

johnpoz

@gertjan said in Slow DNS after 22.05:

It's 'old' technology from the '60 and '70

Its not quite that old hehe, 1983 was when dns was created.. But yeah it's been around long time..

Gertjan

@johnpoz

True. Should have to consult that low number RFC again ;)
Most probably 'they' have taken a phone book, read it, and uses it as an example.
Family names => phone numbers.
Easy.

They've add some other stuff also.

johnpoz

@gertjan yeah in a nutshell sure dns is like a phone book.

Just different books.. The roots being the first book, saying hey if your looking for .tld X go look in book 1, if your looking for .tld Y go look in book 2

This next book lists the domain part of the fqdn, which says hey if your looking domainX look in book A1, if your looking for domainY go look in book A2, etc.

In that book will be the host part of the fqdn,

There might only be 3 books you need to look in, or there could be 6 books, depending on delegation, if cname that references a completely different book all the way back up to the tld, etc.

Generally speaking dns is not all that complicated, but there are quite a few moving parts. But sure the analogy of a phone book is pretty freaking close. DNS is the phonebook of the internet.

Forwarding is you just asking billy, hey could you look up kevins phone number for me.. Sure I could just look in the books myself, but I am lazy ;)

Gertjan

And the day Billy isn't up to the task, not present, not reachable, on holiday, on strike, or whatever, you won't be calling any one any more.

When Billy looks up a number, but send you another one (to error is human) you have an issue.

Added to that : Billy knows who you are in contact with ;)

johnpoz

@gertjan yup all true statements

Kempain

Anyone experiencing this issue got anywhere with fixing it?
I've tried reverting to OOTB DNS settings but still experience the same issue.

I'm going to try a full re-install tonight to see if that helps at all.

tentpiglet

I am also encountering this problem with DNS running on a pfSense VM running on my Proxmox server. Randomly hostnames, like facebook.com, will stop resolving, and then start up again after a brief period of time. No rhyme or reason. No pattern I can find.

As a temporary workaround, I installed a pihole container on proxmox to provide DNS services to my LAN, serve the pihole DNS address to my clients via DHCP. No issues at all after that, everything works fine.

Definitely something in the pfSense DNS.

Mikymike82

I encountered the same issues wit the upgrade to 22.05. After trying some suggestions ont this forum topic and reverting back after comming to the conlusion that noting realy resorted the problem, i have enabled the option: DNS Query Forwarding. After i set this my problems seemed to have been resolved. Any other setting is set to the standard "pfsense" defaults.

Gertjan

@mikymike82 @Kempain @tentpiglet

If you have to forward to some DNS resolver then the quality of your DNS depends on what this DNS server is answering you.

If you decide to use the pfSense default DNS setting, pfSense resolves for you using the 13 ( ! ) available root servers, then DNS will work fine.
I can tell, as I'm using the settings Netgate proposes by default : never had an issue for the last 10+ years.

These settings (first and third) are not default :

"for historical reasons".

tentpiglet

@gertjan

My pfSense DNS resolver settings were the 'default' ones provided out of the box. I made no special modifications. DNS resolution was incredibly unstable.

Switching to the pi-hole container resolved my issues.

johnpoz

@tentpiglet said in Slow DNS after 22.05:

DNS resolution was incredibly unstable.

Do you have IPv6? Other thread around here seems that setting unbound not to use IPv6 in its queries has helped..

server:
do-ip6: no

Gertjan

@tentpiglet said in Slow DNS after 22.05:

pfSense VM running on my Proxmox server

22.05 or 2.6.0 ?

UNstable means :
You found it 'not running', and no process 'unbound' was present when you list processes ?
Or the process unbound was there, but didn't answer ?
An nslookup from a PC on a LAN failed ?

C:\Users\gwkro>nslookup
Server default :   pfSense.main.net
Address:  192.168.1.1

> facebook.com
Server :   pfSense.main.net
Address:  192.168.1.1

Réponse ne faisant pas autorité :
Nom :    facebook.com
Addresses:  2a03:2880:f15a:83:face:b00c:0:25de
          31.13.77.35

>

edit : how many times a day unbound restarts ?
Just ask your pfSense :

grep 'start' /var/log/resolver.log
......
<30>1 2022-07-28T09:31:43.284577+02:00 pfSense.main.net unbound 60716 - - [60716:0] info: start of service (unbound 1.15.0).
<30>1 2022-07-29T14:35:14.991568+02:00 pfSense.main.net unbound 47871 - - [47871:0] info: start of service (unbound 1.15.0).
<30>1 2022-07-29T14:39:18.227880+02:00 pfSense.main.net unbound 94551 - - [94551:0] info: start of service (unbound 1.15.0).
<30>1 2022-08-01T00:00:42.825487+02:00 pfSense.main.net unbound 7853 - - [7853:0] info: start of service (unbound 1.15.0).
<30>1 2022-08-03T10:22:14.251824+02:00 pfSense.main.net unbound 20236 - - [20236:0] info: start of service (unbound 1.15.0).
<30>1 2022-08-03T10:22:52.446334+02:00 pfSense.main.net unbound 82360 - - [82360:0] info: start of service (unbound 1.15.0).
<30>1 2022-08-03T11:04:04.591690+02:00 pfSense.main.net unbound 88564 - - [88564:0] info: start of service (unbound 1.15.0).
<30>1 2022-08-04T08:02:35.997090+02:00 pfSense.main.net unbound 14697 - - [14697:0] info: start of service (unbound 1.15.0).
<30>1 2022-08-04T10:59:24.370800+02:00 pfSense.main.net unbound 59771 - - [59771:0] info: start of service (unbound 1.15.0).
<30>1 2022-08-06T10:09:10.552646+02:00 pfSense.main.net unbound 67115 - - [67115:0] info: start of service (unbound 1.15.0).
<30>1 2022-08-08T00:00:44.809011+02:00 pfSense.main.net unbound 9808 - - [9808:0] info: start of service (unbound 1.15.0).
<30>1 2022-08-09T10:35:25.013811+02:00 pfSense.main.net unbound 44307 - - [44307:0] info: start of service (unbound 1.15.0).

Mikymike82

@gertjan My PFsense was until the upgrade to 22.05 in "standard config" I have never changed anything to my DNS. I even did a complete new installation to replicate the issue. 22.01 no problems and after upgrading to 22.05 my dns resolution was a complete mess.

Jax

Really, I think it's clear something broke in 22.05 and there's not any point in pretending it's a user problem. We've all experienced this and found some work arounds. Perhaps the maintainers should take a serious look at the code and try to figure out why so many users experienced the same phenomenon.

Mikymike82

@tentpiglet I think my solution might (temporarily) help you out with the DNS issues you and others in this topic are experiencing. 1,5 weeks of stable connection so far.

Gertjan

@mikymike82 said in Slow DNS after 22.05:

my dns resolution was a complete mess

What about answering questions ?

Btw : I used a bare bone system (old Dell desktop, added a Quad Intel NIC) for nearly 10 years.
Since last Juin I'm using a SG 4100. Works great.
I have DNS statistics.

@jax said in Slow DNS after 22.05:

why so many users experienced the same phenomenon

120 thousand plus active users - all without DNS ?
That is, I can tell that my DNS works. edit : showing another way to see the resolver restarting.

@mikymike82 said in Slow DNS after 22.05:

22.01 no problems and after upgrading to 22.05

If you don't use IPv6, do what @johnpoz said.

Btw : I never used 22.01 - 22.01 was pre installed on my 4100, but I upgraded the day I received the unit.

johnpoz

@jax said in Slow DNS after 22.05:

We've all experienced

We are??? I am not having any issues..

why so many users experienced the same phenomenon.

So many users? like the handful posted here? Where are these users posting that they are having issues? The 1000's and 1000's of them that are running 22.05? All having issues? Some how I have seem to have missed that it was such a wide spread problem - very odd since I am on the board like all day pretty much every day.. And some how I have missed that its everyone having the issue..