DNS Resolver in forwarding mode slow replies

lumens · Oct 18, 2021, 10:08 AM

Hello everyone. I have used pfSense for many years and really enjoy the features and the platform. Thank you to the team.

I have a new clean pfSense deployment (Gen2 guest vm in a Hyper-V 2019 Server. 2gb RAM, 2 vCPU), with a 100/10 VDSL2 connection. pfSense version is 2.5.2-RELEASE (amd64), installed from iso.

This setup is for home use with 2 users and most important aspect is speed, so i have set DNS resolver in forwarding mode:

Added quad9 dns servers and hostnames in general setup
Selected Enable Forwarding Mode in DNS Resolver General Settings
Please see the screenshots.

Problem is, browsing feels slow, and when checking with DNS benchmark, i see average retrieval speed of 200+ ms while the quad9 resolvers resolve in 50-100ms. To verify, i go to Diagnostigs -> DNS Lookup, and enter some uncached domains, and get results like:

garmin.com
127.0.0.1 3256 msec
9.9.9.10 56 msec
149.112.112.10 55 msec

anker.de
127.0.0.1 252 msec
9.9.9.10 61 msec
149.112.112.10 56 msec

lipton.com
127.0.0.1 255 msec
9.9.9.10 88 msec
149.112.112.10 87 msec

ford.com
127.0.0.1 222 msec
9.9.9.10 57 msec
149.112.112.10 55 msec

etc

Here is a level 4 log for resolving dell.com: pastebin

So the question is, why 127.0.0.1 is so much slower than the dns servers? I also have a deployment of a 2.4.5 with the exact same dns settings, and the response from 127.0.0.1 almost always is 0ms (for uncached domains of course).

Since i tried all the combinations i could think of, any help would be appreciated. Thank you!

Screenshots:

Gertjan · Oct 18, 2021, 4:31 PM

@lumens said in DNS Resolver in forwarding mode slow replies:

So the question is, why 127.0.0.1 is so much slower than the dns servers

178 msec for me for a "dell.com".

If "dell.com" wasn't in the local "unbound" DNS cache, the, resolving it might take some time.

I guess it's understandable that using a big "DNS Provider" like 9.9.9.9 has "dell.com" always in cache.
If not, it has to resolve for you, and that takes time.

Btw : when you use the resolver, you'll get a original answer from of the dell's "dell.com" authoritative name servers. Because dell.com uses DNSSEC, that's also verified.
When you forward, you get a copy of the cache of the forwarder if it was present in their cache. If not, they will resolve for you.

To see what a resolver needs to to get an answer :

dig dell.com +trace

lumens · Oct 18, 2021, 4:31 PM

@gertjan Thank you for your answer. I understand that resolving needs to query the hierarchy from the root servers down to the requested domain name, and this has an increased latency, which is to be expected.

But since i have configured my DNS Resolver in "Forwardind Mode", i would expect that the query to localhost would be comparable to the query to the dns server configured in the "General Setup" section (quad9 nameservers in my case).

What i don't understand is, when i use dig, from within pfsense, to query directly quad9 for let's say apple.com, i have a response time of 50ms. When i query localhost (unbound in forwarding mode), i have a response time 200-250ms. I would expect that unbound, since configured in forwarding mode would answer almost as quick as querying the nameservers directly.

The thing is, in the other deployment that i have with a 2.4.4 version, the previous statement holds, dig shows similar results when querying directly 9.9.9.10 or 127.0.0.1 (unbound in forwarding mode).

Querying 9.9.9.10 (quad9) directly:

; <<>> DiG 9.16.16 <<>> linux.org @9.9.9.10
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52691
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;linux.org.			IN	A

;; ANSWER SECTION:
linux.org.		300	IN	A	104.21.62.194
linux.org.		300	IN	A	172.67.138.190

;; Query time: 58 msec
;; SERVER: 9.9.9.10#53(9.9.9.10)
;; WHEN: Mon Oct 18 18:28:56 EEST 2021
;; MSG SIZE  rcvd: 70

Querying localhost (unbound in forwarding mode):

; <<>> DiG 9.16.16 <<>> linux.org @127.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23152
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;linux.org.			IN	A

;; ANSWER SECTION:
linux.org.		300	IN	A	172.67.138.190
linux.org.		300	IN	A	104.21.62.194

;; Query time: 228 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Oct 18 18:29:20 EEST 2021
;; MSG SIZE  rcvd: 70

Any explanation would be appreciated

Gertjan · Oct 19, 2021, 8:33 AM

@lumens said in DNS Resolver in forwarding mode slow replies:

But since i have configured my DNS Resolver in "Forwardind Mode", i would expect that the query to localhost would be comparable to the query to the dns server configured in the "General Setup" section (quad9 nameservers in my case).

and unbound, using forwarder mode, is using port 853 and encrypts the traffic (TLS).

Probably normal ( ? ), but unbound (forwarder) also asks for the AAAA, the NS, and CNAME, and also requests for dell.com.lum1.lan.
I couldn't find the "A" request ....

Btw : Why 9.9.9.10 as its for experts only ? What about 9.9.9.9 or maybe 9.9.9.11.

edit :
what happens when you ask for "www.micosoft.com." instead of "www.micosoft.com" ?