DNS Resolver Root Server Question

jpvonhemel · Apr 11, 2022, 2:22 PM

Hello,

I am in the process of benchmarking DNS using Steve Gibson's DNS Benchmark software. I was looking to optimize the servers I use in DNS Resolver. I use my pfsense ip as my DNS Server for all of my hosts.

Since DNS resolver queries root and authoritative DNS servers, what is the reason to include a dns server like comcast, google, or cloudflare? Unbound resolves without any ISP or added DNS servers, or maybe I am seeing cached performance. Is this mostly for regional performance benefits or am I missing something.

Thank you.
J.

*Resolver
A DNS server acting as a resolver will accept recursive queries from clients and/or forwarders and perform DNS resolution by making iterative queries to root and authoritative DNS servers in search of an answer. Like a forwarder it will typically cache results for increased performance.

A resolver maintains a list of root servers in its “root hints” list. Resolvers will typically ship with a stock list but update it periodically from a trusted source.

Root Server
Authoritative name servers which serve queries for the DNS root zone (.). Though it is held as a short list of servers (a.root-servers.net through m.root-servers.net) there are hundreds of servers worldwide handling the requests.

These servers are a starting point for queries and will direct resolvers to authoritative DNS servers for zones which can answer their queries.*

johnpoz · Apr 11, 2022, 2:23 PM

@jpvonhemel said in DNS Resolver Root Server Question:

what is the reason to include a dns server like comcast, google, or cloudflare?

There isn't a reason..

jpvonhemel · Apr 11, 2022, 2:30 PM

@johnpoz Really, mind blown. Unbound must be really special in the FreeBSD realm.

johnpoz · Apr 11, 2022, 2:50 PM

@jpvonhemel unbound isn't anything special - its just a resolver.. There is zero reason for any other dns service (which are just resolvers) when you run your own.

When you want to say lookup www.somedomain.tld

When you forward (ask google or cloudflare) what do you think happens? If they do not have it cached, it is resolved just the same way unbound does it.

They ask the roots, hey what is NS for the .tld, ok thanks - goes ask one of those ns, hey what is the NS for somedomain.tld, ok thanks. Hey NS for somedomain.tld what is the A record for www.somedomain.tld

Your just cutting out the middle man when you run your own resolver.. If you want to see how something is resolved, run a simple dig +trace..

example

$ dig forum.netgate.com +trace +nodnssec

; <<>> DiG 9.16.27 <<>> forum.netgate.com +trace +nodnssec
;; global options: +cmd
.                       84355   IN      NS      k.root-servers.net.
.                       84355   IN      NS      l.root-servers.net.
.                       84355   IN      NS      m.root-servers.net.
.                       84355   IN      NS      b.root-servers.net.
.                       84355   IN      NS      c.root-servers.net.
.                       84355   IN      NS      d.root-servers.net.
.                       84355   IN      NS      e.root-servers.net.
.                       84355   IN      NS      f.root-servers.net.
.                       84355   IN      NS      g.root-servers.net.
.                       84355   IN      NS      h.root-servers.net.
.                       84355   IN      NS      a.root-servers.net.
.                       84355   IN      NS      i.root-servers.net.
.                       84355   IN      NS      j.root-servers.net.
;; Received 239 bytes from 192.168.9.253#53(192.168.9.253) in 0 ms

com.                    172800  IN      NS      e.gtld-servers.net.
com.                    172800  IN      NS      b.gtld-servers.net.
com.                    172800  IN      NS      j.gtld-servers.net.
com.                    172800  IN      NS      m.gtld-servers.net.
com.                    172800  IN      NS      i.gtld-servers.net.
com.                    172800  IN      NS      f.gtld-servers.net.
com.                    172800  IN      NS      a.gtld-servers.net.
com.                    172800  IN      NS      g.gtld-servers.net.
com.                    172800  IN      NS      h.gtld-servers.net.
com.                    172800  IN      NS      l.gtld-servers.net.
com.                    172800  IN      NS      k.gtld-servers.net.
com.                    172800  IN      NS      c.gtld-servers.net.
com.                    172800  IN      NS      d.gtld-servers.net.
;; Received 842 bytes from 198.41.0.4#53(a.root-servers.net) in 21 ms

netgate.com.            172800  IN      NS      ns1.netgate.com.
netgate.com.            172800  IN      NS      ns2.netgate.com.
;; Received 142 bytes from 192.33.14.30#53(b.gtld-servers.net) in 35 ms

forum.netgate.com.      3600    IN      A       208.123.73.199
netgate.com.            3600    IN      NS      ns2.netgate.com.
netgate.com.            3600    IN      NS      ns1.netgate.com.
;; Received 186 bytes from 208.123.73.80#53(ns1.netgate.com) in 39 ms

And all of the info it gets back is cached.. So if you ask for say otherthing.something.tld - it already knows the authoritative NSers for something.tld - so it just goes and asks one of them directly since it already knows that info.. no need to ask roots or gltd servers again, etc..

Same for all of the .tlds, .com, .net, .org - once it looks up those gltds servers from the "roots" it doesn't have to go ask roots again for any otherdomain.org, etc. it just goes and asks one of those authoriative for the .org tld hey what is the authoritative ns for otherdomain.org

Gertjan · Apr 11, 2022, 3:25 PM

@jpvonhemel said in DNS Resolver Root Server Question:

Unbound must be really special in the FreeBSD realm.

Noop.
I've tried Unbound using the 'other' (worlds most known) system : Debain. https://packages.debian.org/en/bullseye/unbound.
It's the same. Does the same thing.
It's a resolver, and because it resolves, it can do even DNSSEC.
Unbound can forward, if you have to wire your DNS traffic to some other resolver.

@johnpoz said in DNS Resolver Root Server Question:

There isn't a reason..

Have you seen [pick whatever social media] and search for : how to install pfSense ?

You'll see this :

People tend to think they have to give something for 'Primary and Secondary here.
The preceding text is always ( ?) overlooked :

The default behaviour of the DNS Resolver will ignore manually configured DNS servers for client queries and query root DNS servers directly. To use the manually configured DNS servers below for client queries, visit Services > DNS Resolver and enable DNS Query Forwarding after completing the wizard

..... and as said, after completing the wizard, the activate forward mode.

@johnpoz said in DNS Resolver Root Server Question:

unbound isn't anything special - its just a resolver

Keep in mind that that makes sense only if I know what a resolver is
Most people are brainwashed with "I have to forward to someone / something?" because that was what we all did in the old days.

thiasaef · Apr 12, 2022, 1:51 PM

@johnpoz said in DNS Resolver Root Server Question:

When you forward (ask google or cloudflare) what do you think happens? If they do not have it cached, it is resolved just the same way unbound does it.

It will most likely hit the cache and even if it doesn't it is still going to be faster than my own resolver:

Resolver (pfSense @home):

drill forum.netgate.com
;; Query time: 179 msec

Resolver (pfSense @vServer):

drill forum.netgate.com
;; Query time: 123 msec

Forward to Google (from home):

drill forum.netgate.com @8.8.8.8
;; Query time: 9 msec

Forward to Cloudflare (from home):

drill forum.netgate.com @1.1.1.1
;; Query time: 17 msec

And all of the info it gets back is cached.. So if you ask for say otherthing.something.tld - it already knows the authoritative NSers for something.tld - so it just goes and asks one of them directly since it already knows that info.. no need to ask roots or gltd servers again, etc..

One hour later ...

Resolver (pfSense @home):

drill forum.netgate.com
;; Query time: 170 msec

Resolver (pfSense @vServer):

drill forum.netgate.com
;; Query time: 99 msec

Forward to Cloudflare (from home):

drill forum.netgate.com @1.1.1.1
;; Query time: 98 msec

And the frequent Unbound restarts caused by https://redmine.pfsense.org/issues/12613 et al. result in even lower cache hit rates than ever before.

johnpoz · Apr 12, 2022, 2:05 PM

@thiasaef if you want to forward then forward.. Restarting of unbound, yes clears the cache.

I have not seen such issues, my unbound normally runs for days and days - restarts normally when I do something that restarts it, etc.


[22.01-RELEASE][admin@sg4860.local.lan]/root: unbound-control -c /var/unbound/unbound.conf status
version: 1.13.2
verbosity: 1
threads: 4
modules: 2 [ validator iterator ]
uptime: 323100 seconds
options: control(ssl)
unbound (pid 97316) is running...
[22.01-RELEASE][admin@sg4860.local.lan]/root:

What is that like almost 90 hours..

If you think 1 or 2 ms has anything to do with anything.. Then sure ok, forward if you want.. But I will resolve.. I have no want of sending all my queries to some service.

The only possible reason I could see forwarding, is if they provided something like filtering that I wanted to leverage. Or your on a internet connection that has bad latency or peering, etc.

If your unbound is restarting all the time - then cache is not going to be very good..

But response from when cached is going to be less than 1 ms, your best time there is 9ms too google..

You can also tweak unbound to do prefetching, so stuff you normally go too is more likely to be cached. And you can set to serve 0, so even if past the ttl you get the last IP of that fqdn, and then its relooked up in the background, etc. etc..

You do you.. Which is why there are many options and configurations possible for a user to decide what they want to do, etc.

Gertjan · Apr 12, 2022, 2:37 PM

@thiasaef said in DNS Resolver Root Server Question:

Resolver (pfSense @home):

drill forum.netgate.com
;; Query time: 179 msec

Ah lol. Let me try that one (using the resolver as a resolver == pfSense default DNS settings ) :

drill forum.netgate.com | grep 'Query'
;; Query time: 0 msec

There.
0 or zero.

Because : I've just read your forum post, so "forum.netgate.com" lices in my unbound cache. I guess it lives forever in my cache as I do what JP already said :

edit : 8.8.8.8 or 1.1.1.1 couldn't do this. If they would, after a day or so, they would have to cache all the existing host names on the internet. Just keeping that updated is already impossible. Indexing a cache that big (penta bytes cache size ? Bigger ?) wouldn't be a "x ms" thing any more.
You've demonstrated it : after one hours, it really looked like that forum.netgate.com wasn't in the 1.1.1.1 cache anymore, so 1.1.1.1 had to resolve for you.
You, on the other hand, woun't visit "every possible domain" on the Intranet every day, your DNS footprint will be small, maybe a couple of thousand host names ? and that is something unbound can handle just fine, and keep it updated.
Let's check : Status > DNS Resolver : 700 entries.

edit in edit :

Restart the unbound service (see arrow) and then refresh the page : the cache won't be empty, as the cache was exported to a file before stop, and read back upon start

so I let unbound populate his cache, and unbound keeps it updated. So it stays valid and up to date when I need it.
If 'time' consumed for DNS handling was an issue for me, I guess I nailed it.

And yes, I tend to keep unbound running as long as possible ;)

johnpoz · Apr 12, 2022, 2:50 PM

@gertjan said in DNS Resolver Root Server Question:

as the cache was exported to a file before stop, and read back upon start

While I know this is possible - I did not think pfsense unbound was doing that?? Is that something pfblocker does or can do?

[22.01-RELEASE][admin@sg4860.local.lan]/root: unbound-control -c /var/unbound/unbound.conf stats_noreset | grep cache.count
msg.cache.count=5022
rrset.cache.count=8956
infra.cache.count=5704
key.cache.count=379

restart unbound

[22.01-RELEASE][admin@sg4860.local.lan]/root: unbound-control -c /var/unbound/unbound.conf status
version: 1.13.2
verbosity: 1
threads: 4
modules: 2 [ validator iterator ]
uptime: 66 seconds
options: control(ssl)
unbound (pid 10009) is running...
[22.01-RELEASE][admin@sg4860.local.lan]/root:

[22.01-RELEASE][admin@sg4860.local.lan]/root: unbound-control -c /var/unbound/unbound.conf stats_noreset | grep cache.count
msg.cache.count=29
rrset.cache.count=129
infra.cache.count=64
key.cache.count=5
[22.01-RELEASE][admin@sg4860.local.lan]/root:

While if unbound was restarting every few minutes or something - sure that would be problematic - more so you trying to look up something while it wasn't actually up.

But even when doing a full resolve of something all the way down from roots, your taking a few extra ms.. If there is nothing wrong means nothing more than a few extra ms.. If your obsessing over a few ms in query time.. That can not even be noticed by human.. Lets say it takes 200 ms to resolve something... That is .2 of a second.. Do you really think that is going to cause a page load to be noticeable slower??

thiasaef · Apr 12, 2022, 2:57 PM

@johnpoz said in DNS Resolver Root Server Question:

If you think 1 or 2 ms has anything to do with anything..

No, but 170ms does and that's why I personally stopped using the resolver mode shortly after upgrading from 2.4.5-p1 to 2.5.2+, simply because I couldn't stand the performance impact any longer - not because I would prefer it.

But response from when cached is going to be less than 1 ms, your best time there is 9ms too google..

The initial response from google will be cached locally as well ...

You can also tweak unbound to do prefetching ... And you can set to serve 0

Yes, that works well if Unbound does not restart frequently - which it does if you run 2.5.2+ and have multiple end user devices directly attached to the firewall without an intermediary switch.

You do you..

Yes, but you were the one that said: "There is zero reason for any other dns service", which implies that anyone who does is stupid.

@gertjan said in DNS Resolver Root Server Question:

I guess it lives forever in my cache as I do what JP already said

Only if you access the forum more often than once every hour.

@gertjan said in DNS Resolver Root Server Question:

edit : 8.8.8.8 or 1.1.1.1 couldn't do this ...

Sorry, but you obviously don't understand what enabling "Prefetch Support" actually does.

johnpoz · Apr 12, 2022, 2:57 PM

@thiasaef said in DNS Resolver Root Server Question:

but 170ms does

no not really ;) 0.17 seconds... Your talking the blink of an eye time...

That is a full resolve time.. Happens now and then when your looking up something that needs to be fully resolved..

Again you do you.. But being worried about a few ms here or there is not very productive... Your not flash trading stocks ;) You a high frequency stock trader worried about nano seconds ;) heheh

thiasaef · Apr 12, 2022, 3:17 PM

@johnpoz said in DNS Resolver Root Server Question:

0.17 seconds... Your talking the blink of an eye time...

times the number of sequentially executed queries per page. Example (cnn.com):

... more than 60 DNS queries in total.

johnpoz · Apr 12, 2022, 3:23 PM

@thiasaef queries are not sequential. If you load up a website that has say 4 or 5 different fqdn to load stuff from.

It doesn't have to wait 170 ms for fqdn1 to be resolved before it asks for fqdn2, etc..

If you look at first query there, and then last query your talking a total of .471 seconds..

Your dns query time is going to be a very very small fraction of the load time of that site..

thiasaef · Apr 12, 2022, 3:33 PM

@johnpoz said in DNS Resolver Root Server Question:

@thiasaef queries are not sequential.

Agreed, but some are definitely interdependent.

Your dns query time is going to be a very very small fraction of the load time of that site..

Same test with Unbound in resolver mode:

Gertjan · Apr 12, 2022, 3:35 PM

... and yes, if you want to look at cnn.com, then half the planet has to made aware of that.
Every big iNet player does that these days.

On the other side : cnn.com isn't doing DNSSEC ( "DNS fact checking" ?! ).
If the did, you could see a triple the number of queries, and some of them will be to big for UDP, so TCP (slower) will kick in.

jpvonhemel · Apr 12, 2022, 7:56 PM

@johnpoz said in DNS Resolver Root Server Question:

I have no want of sending all my queries to some service.

I too do not want my ISP or another third party, ie google, comcast, or cloudflare building a browsing profile and profiting, but I also would like performance to be optimal at the same time. Seems fair to sai that unbound enhances privacy out of the box.

Privacy seems all but lost these days, but I want to limit the loss and avoid being someone's product.

thiasaef · Apr 13, 2022, 12:38 PM

Actual page load time (PerformanceTiming.domComplete - PerformanceTiming.navigationStart) of www.bbc.com:

Forward [1st run || 2nd run] || Resolve [1st run || 2nd run]
          0.87 s || 0.89 s   ||           1.65 s || 0.84 s

@gertjan said in DNS Resolver Root Server Question:

On the other side : cnn.com isn't doing DNSSEC

97 % of all .com domains are unsigned as of now: https://rick.eng.br/dnssecstat/

@gertjan said in DNS Resolver Root Server Question:

If the did, you could see a triple the number of queries

ftp.isc.org supports all bells and whistles related to DNSSEC:

Resolve [1st run || 2nd run] || Resolve +DNSSEC [1st run || 2nd run]
          1.05 s || 0.95 s   ||                   1.28 s || 0.96 s