Slow Root DNS Servers?
-
I've been having trouble with all things Google for a bit over the last week. There have been reports of problems at Down Detector:
http://downdetector.com/status/google/map/
but it doesn't seem like there's been much complaint about it. Since my problems have been with all google.com pages (and nothing else), I thought I'd try seeing if I could check on how long the DNS root servers were taking to respond (I've got Unbound running in Resolver mode using DNSSEC). Below, is the result of a drill command to www.google.com. If I'm reading that right, the query to the root server is taking 5 seconds to respond and the query to the next level down is taking 10 seconds. Is that correct?
drill -V5 -T www.google.com ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;; . IN NS ;; ANSWER SECTION: ;; AUTHORITY SECTION: ;; ADDITIONAL SECTION: ;; Query time: 0 msec ;; WHEN: Sun Feb 4 09:03:09 2018 ;; MSG SIZE rcvd: 0 ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: rd ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;; 129.14.0.193.in-addr.arpa. IN PTR ;; ANSWER SECTION: ;; AUTHORITY SECTION: ;; ADDITIONAL SECTION: ;; Query time: 0 msec ;; WHEN: Sun Feb 4 09:03:14 2018 ;; MSG SIZE rcvd: 0 . 518400 IN NS a.root-servers.net. . 518400 IN NS b.root-servers.net. . 518400 IN NS c.root-servers.net. . 518400 IN NS d.root-servers.net. . 518400 IN NS e.root-servers.net. . 518400 IN NS f.root-servers.net. . 518400 IN NS g.root-servers.net. . 518400 IN NS h.root-servers.net. . 518400 IN NS i.root-servers.net. . 518400 IN NS j.root-servers.net. . 518400 IN NS k.root-servers.net. . 518400 IN NS l.root-servers.net. . 518400 IN NS m.root-servers.net. ;; Received 508 bytes from 193.0.14.129#53(k.root-servers.net.) in 5185 ms ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;; www.google.com. IN A ;; ANSWER SECTION: ;; AUTHORITY SECTION: ;; ADDITIONAL SECTION: ;; Query time: 0 msec ;; WHEN: Sun Feb 4 09:03:15 2018 ;; MSG SIZE rcvd: 0 com. 172800 IN NS a.gtld-servers.net. com. 172800 IN NS b.gtld-servers.net. com. 172800 IN NS c.gtld-servers.net. com. 172800 IN NS d.gtld-servers.net. com. 172800 IN NS e.gtld-servers.net. com. 172800 IN NS f.gtld-servers.net. com. 172800 IN NS g.gtld-servers.net. com. 172800 IN NS h.gtld-servers.net. com. 172800 IN NS i.gtld-servers.net. com. 172800 IN NS j.gtld-servers.net. com. 172800 IN NS k.gtld-servers.net. com. 172800 IN NS l.gtld-servers.net. com. 172800 IN NS m.gtld-servers.net. ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: rd ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;; 129.14.0.193.in-addr.arpa. IN PTR ;; ANSWER SECTION: ;; AUTHORITY SECTION: ;; ADDITIONAL SECTION: ;; Query time: 0 msec ;; WHEN: Sun Feb 4 09:03:40 2018 ;; MSG SIZE rcvd: 0 ;; Received 492 bytes from 193.0.14.129#53(k.root-servers.net.) in 10172 ms ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;; www.google.com. IN A ;; ANSWER SECTION: ;; AUTHORITY SECTION: ;; ADDITIONAL SECTION: ;; Query time: 0 msec ;; WHEN: Sun Feb 4 09:03:40 2018 ;; MSG SIZE rcvd: 0 google.com. 172800 IN NS ns2.google.com. google.com. 172800 IN NS ns1.google.com. google.com. 172800 IN NS ns3.google.com. google.com. 172800 IN NS ns4.google.com. ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: rd ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;; 30.94.12.192.in-addr.arpa. IN PTR ;; ANSWER SECTION: ;; AUTHORITY SECTION: ;; ADDITIONAL SECTION: ;; Query time: 0 msec ;; WHEN: Sun Feb 4 09:03:40 2018 ;; MSG SIZE rcvd: 0 ;; Received 196 bytes from 192.12.94.30#53(e.gtld-servers.net.) in 75 ms ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;; www.google.com. IN A ;; ANSWER SECTION: ;; AUTHORITY SECTION: ;; ADDITIONAL SECTION: ;; Query time: 0 msec ;; WHEN: Sun Feb 4 09:03:41 2018 ;; MSG SIZE rcvd: 0 ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;; www.google.com. IN A ;; ANSWER SECTION: ;; AUTHORITY SECTION: ;; ADDITIONAL SECTION: ;; Query time: 0 msec ;; WHEN: Sun Feb 4 09:03:41 2018 ;; MSG SIZE rcvd: 0 www.google.com. 300 IN A 216.58.204.36 ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: rd ; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;; 10.36.239.216.in-addr.arpa. IN PTR ;; ANSWER SECTION: ;; AUTHORITY SECTION: ;; ADDITIONAL SECTION: ;; Query time: 0 msec ;; WHEN: Sun Feb 4 09:03:41 2018 ;; MSG SIZE rcvd: 0 ;; Received 48 bytes from 216.239.36.10#53(ns3.google.com.) in 50 ms
-
If you are running the DNS Resolver it keeps internal stats about the upstream servers it contacts with more accurate info:
unbound-control -c /var/unbound/unbound.conf dump_infra
-
What is supposed to happen when you issue this command? For me it just returned to the shell prompt.
-
you would get the stats of your NS that unbound has used… example
[2.4.2-RELEASE][root@sg4860.local.lan]/root: unbound-control -c /var/unbound/unbound.conf dump_infra
208.84.2.53 microsoft.com. ttl 883 ping 3 var 60 rtt 243 rto 243 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2001:503:a83e::2:30 net. ttl 796 ping 7 var 87 rtt 355 rto 355 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2600:1480:1::43 akam.net. ttl 883 ping 8 var 87 rtt 356 rto 356 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
192.5.6.30 net. ttl 795 ping 2 var 76 rtt 306 rto 306 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
205.251.193.77 x.dropbox.com. ttl 394 ping 4 var 79 rtt 320 rto 320 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
208.78.70.31 amazon.com. ttl 766 ping 1 var 75 rtt 301 rto 301 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
104.91.167.75 dspg.akamaiedge.net. ttl 884 ping 1 var 74 rtt 297 rto 297 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
192.55.83.30 com. ttl 265 ping 4 var 80 rtt 324 rto 324 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
195.244.245.25 t-internal.com. ttl 867 ping 30 var 105 rtt 450 rto 450 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
213.248.220.1 co.uk. ttl 796 ping 13 var 96 rtt 397 rto 397 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2001:503:39c1::30 net. ttl 796 ping 8 var 89 rtt 364 rto 364 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2001:4860:4802:34::a google.com. ttl 265 ping 6 var 84 rtt 342 rto 342 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2620:0:37::53 msft.net. ttl 883 ping 9 var 72 rtt 297 rto 297 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
<snipped multiple="" pages="" and="">https://www.unbound.net/documentation/info_timeout.html
The dump_infra command dumps the entire contents of the infra-cache, a snapshot of the ping-times of the servers on the internet that unbound has contacted.If your not seeing anything - I would take it your not actually using unbound..</snipped>
-
Thanks, Jimp. Unfortunately, I don't have any idea how to read the result. Looking at just the lines for google.com in there, I get:
Search "google.com" (8 hits in 1 file) new 1 (8 hits) Line 15: 2001:4860:4802:36::a google.com. ttl 100 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0 Line 274: 216.239.32.10 google.com. ttl 40 ping 60 var 9 rtt 96 rto 96 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0 Line 339: 2001:4860:4802:34::a google.com. ttl 39 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0 Line 521: 216.239.34.10 google.com. ttl 75 ping 61 var 11 rtt 105 rto 105 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0 Line 765: 216.239.36.10 google.com. ttl 585 ping 40 var 23 rtt 132 rto 132 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0 Line 791: 216.239.38.10 google.com. ttl 585 ping 46 var 35 rtt 186 rto 186 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0 Line 979: 2001:4860:4802:32::a google.com. ttl 75 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0 Line 1048: 2001:4860:4802:38::a google.com. ttl 585 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
Nothing leaps out and hits me in the eye as being bad. But, my issues with google.com pages has gotten much better lately
It's unrelated, but those results bring up a question: I thought I had IPv6 support turned off throughout pfSense, yet I see some of those lines using IPv6. Does that mean I missed a setting somewhere?
-
Does your pfsense have ipv6 on its wan interface? Do you have unbound set to be able to use say an IPv6 tunnel you setup? That is why you see some in mine.. I had unbound able to use my HE interface for outbound. Which I have removed and flushed the infra cache.. Will give it a few minutes and and look at in a few and correct there should be no IPv6 listed.
Also NS with IPv6 could be listed but if your seeing RTT of 376 which is the default value - it never actually talked to it..
here is one of your examples
"Line 339: 2001:4860:4802:34::a google.com. ttl 39 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0"
And you notice the ping value is 0.. That is just a NS it could use - but never talked too.. All of your IPv6 entries look like never talked too.
See how some of mine have ping values and RTT other than the 376 default value
"2001:503:39c1::30 net. ttl 796 ping 8 var 89 rtt 364 rto 364 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0"
If your curious what NS and their specific entries in your infra cache prob better to use lookup and the domain, example
[2.4.2-RELEASE][root@sg4860.local.lan]/root: unbound-control -c /var/unbound/unbound.conf lookup google.com
The following name servers are used for lookup of google.com.
;rrset 85631 4 0 2 0
google.com. 172031 IN NS ns2.google.com.
google.com. 172031 IN NS ns1.google.com.
google.com. 172031 IN NS ns3.google.com.
google.com. 172031 IN NS ns4.google.com.
;rrset 85631 1 0 1 0
ns4.google.com. 172031 IN A 216.239.38.10
;rrset 85631 1 0 1 0
ns4.google.com. 172031 IN AAAA 2001:4860:4802:38::a
;rrset 85631 1 0 1 0
ns3.google.com. 172031 IN A 216.239.36.10
;rrset 85631 1 0 8 0
ns3.google.com. 344831 IN AAAA 2001:4860:4802:36::a
;rrset 85631 1 0 1 0
ns1.google.com. 172031 IN A 216.239.32.10
;rrset 85631 1 0 8 0
ns1.google.com. 344831 IN AAAA 2001:4860:4802:32::a
;rrset 85631 1 0 1 0
ns2.google.com. 172031 IN A 216.239.34.10
;rrset 85631 1 0 8 0
ns2.google.com. 344831 IN AAAA 2001:4860:4802:34::a
Delegation with 4 names, of which 0 can be examined to query further addresses.
It provides 8 IP addresses.
2001:4860:4802:34::a rto 376 msec, ttl 131, ping 0 var 94 rtt 376, tA 0, tAAAA 0, tother 0, EDNS 0 assumed.
216.239.34.10 rto 162 msec, ttl 131, ping 10 var 38 rtt 162, tA 0, tAAAA 0, tother 0, EDNS 0 probed.
2001:4860:4802:32::a rto 376 msec, ttl 131, ping 0 var 94 rtt 376, tA 0, tAAAA 0, tother 0, EDNS 0 assumed.
216.239.32.10 rto 219 msec, ttl 131, ping 7 var 53 rtt 219, tA 0, tAAAA 0, tother 0, EDNS 0 probed.
2001:4860:4802:36::a rto 376 msec, ttl 131, ping 0 var 94 rtt 376, tA 0, tAAAA 0, tother 0, EDNS 0 assumed.
216.239.36.10 rto 333 msec, ttl 131, ping 5 var 82 rtt 333, tA 0, tAAAA 0, tother 0, EDNS 0 probed.
2001:4860:4802:38::a not in infra cache.
216.239.38.10 rto 198 msec, ttl 131, ping 10 var 47 rtt 198, tA 0, tAAAA 0, tother 0, EDNS 0 probed.
[2.4.2-RELEASE][root@sg4860.local.lan]/root:Notice no IPv6 used here - I removed HE interface from outbound on my unbound setup. All their RTT are 376
-
All my interfaces have IPv6 Configuration Type set to None. So, I went through some more settings and, under System > Advanced > Networking > IPv6 Options (the very first option), found the following to be turned on:
Allow IPv6 All IPv6 traffic will be blocked by the firewall unless this box is checked
NOTE: This does not disable any IPv6 features on the firewall, it only blocks traffic.I swear I had that turned off. I can't come up with any reason why I would have left it on. I'll turn it off and see what happens over time.
Thanks.
-
flush your cache… Remember your still going to see NS listed with IPv6 for domains that have IPv6 NS... But if pfsense has no IPv6 address that unbound can use for outbound queries then there would be no way for pfsense to talk to them... So in the cache you will see none have been talked too..