PfSense not using right DNS servers
-
Hi, I'm having an issue with pfSense using the wrong DNS servers. I have 3 DNS servers configured, 9.9.9.9, 8.8.8.8, and 8.8.4.4. Also I'm using Unbound DNS resolver. But when I do a packet capture on the WAN interface, I can see that pfSense is not using these. I'm just wondering why and if it has something to do with forwarding mode vs. resolver?
-
"Also I'm using Unbound DNS resolver."
The resolve does not foward - it resolves… You could setup a hundred ns.. If your using unbound its resolving.. Unless you changed it to forwarder mode With the check box..
To be honest not sure why anyone would do this... If you want a forwarder, just use the forwarder and turn off unbound.. Dnsmasq is better forwarder - you can have it forward to all of your NS and use the fastest response, etc.
Why not just resolve, why do you want to forward to those?
-
I'm seeing the exact same thing. I added the quad 9 DNS as my primary server, but my replies were still coming back from 8.8.8.8. It appears this is essentially a race issue. The DNS servers are not queried in order as you might expect and instead ask simultaneously with the first response providing the answer. For example, in the case of quad 9 and querying isitblocked.org, the Google DNS is often just a titch faster so it answers before the NX Domain response is returned from quad 9. Hence, the IP address is resolved and the second response of NX Domain is discarded.
This apparently is an ongoing question/concern from other users in the forums so it looks like it is here to stay. In the meantime, I would suggest using quad 9's secondary DNS – 149.112.112.112. If you are using 9.9.9.9 as your primary and anything else as your secondary, what you will receive is likely not what you are expecting. For the record, the DNS lookup in the web GUI does query sequentially so it will respond "properly" every time (see attached). BTW, I'm using DNSBL so the suggestion to turn off unbound is not possible... dnsmasq exhibits the exact same behavior anyway.
15:48:30.593813 IP MYIP.48067 > 9.9.9.9.53: UDP, length 54
15:48:30.618897 IP 9.9.9.9.53 > MYIP.48067: UDP, length 122
15:48:30.619107 IP MYIP.47358 > 9.9.9.9.53: UDP, length 59
15:48:30.692827 IP 9.9.9.9.53 > MYIP.47358: UDP, length 96
15:48:30.693039 IP MYIP.63765 > 8.8.8.8.53: UDP, length 49
15:48:30.723946 IP 8.8.8.8.53 > MYIP.63765: UDP, length 65
15:48:30.724171 IP MYIP.28155 > 8.8.8.8.53: UDP, length 42
15:48:30.753318 IP 8.8.8.8.53 > MYIP.28155: UDP, length 763 <-- FIRST RESPONSE
15:48:31.689338 IP MYIP.42062 > 9.9.9.9.53: UDP, length 56
15:48:31.714287 IP 9.9.9.9.53 > MYIP.42062: UDP, length 1048 <-- SECOND RESPONSE
-
If you want the protection of 9.9.9.9, you don't want to mix it with 8.8.8.8 and 8.8.4.4 anyway. Resolver can't read your mind. You told it to use 8.8.8.8 as well.
You need to switch the resolver to forwarding mode and it will forward to the servers defined in System > General.
They specifically state they support DNSSEC so you should be able to leave that enabled in this case.
This apparently is an ongoing question/concern from other users in the forums so it looks like it is here to stay.
I would hope so since it is behaving exactly how it has been programmed by the user to behave.
-
I stated that because the general thought (including my own) is the DNS servers are queried in order, not simultaneously, i.e. if the primary DNS is unavailable, try the secondary, etc. I don't completely agree with it, but now that I know what it is doing I'll work around it.
-
Guide to configuring, testing, and troubleshooting Quad9 on pfSense
https://www.linuxincluded.com/configuring-quad9-on-pfsense/
-
You might as well drop the secondary forwarder and use only 9.9.9.9 to avoid a situation where 9.9.9.9 is not responding and the forwarder (unbound in forwarding mode) then tries the unsecure secondary. There is absolutely no requirement to have two forwarders configured, one is just fine.
-
That is the secure secondary DNS as specified by Quad9. I get it that anycast means it technically isn't a single server, but more than one forwarder should still be used for redundancy. Granted, I would have preferred using a different provider for better redundancy, but that issue is what led me here. FWIW, you have the same incorrect understanding of "sequential queries" as myself, the original poster, and numerous others on this forum. If you configure multiple DNS, it is a race – all DNS are queried simultaneously and the first responder wins. In my research, dnsmasq is the same although I did not test it. Normally, this is absolutely what you want to ensure the fastest response. This is not the desired behavior when DNS servers such as Quad9 and Google might respond differently.
-
Not sure where you go this idea of a "race".. Its not a race with unbound.. Its a race with dnsmasq unless you set sequential queries.
But with unbound, yes it will ask the next forwarder in the list - if the response is not fast enough.. You need to understand what unbound understands about the RTT to a specific NS be a forwarder or a NS listed as authoritative for a domain.
This might help
https://www.unbound.net/documentation/info_timeout.htmlStart doing queries for different domains.. See when unbound also sends to next NS on the list, when it retrans the same query to the same NS, etc.
Here I set 8.8.8.8 and 9.9.9.10 as forwarders… Did a few queries and it liked to use 8.8.8.8 until it didn't respond fast enough then it switched over to 9.9.9.10 - if it doesn't respond fast enough then sure it will send to 8.8.8.8, etc.
There is a whole bunch of stuff that happens in the background - see the above link for some details. And the BIG thing to understand here is that be it your taking about a dns client or a resolver/forwarder be it unbound, dnsmasq, powerdns, etc..
Just because you list NS 1, and NS 2, NS 3, etc.. Does not mean they will only ever talk to 1 unless it does not answer in like seconds. You might have 50ms as timeout before it will ask again or send to next NS on the list... You need to fully understand all the background stuff the software your using is doing to figure out which NS to use.. This goes for your OS, your dns client the dns fowarder/resolver your using, etc..
Here I took a look at what unbound was showing a few min apart - see how the numbers changed.
[2.4.2-RELEASE][root@pfsense.local.lan]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_infra
9.9.9.10 . ttl 234 ping 20 var 6 rtt 50 rto 50 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
8.8.8.8 . ttl 234 ping 39 var 16 rtt 103 rto 103 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0[2.4.2-RELEASE][root@pfsense.local.lan]/var/unbound: unbound-control -c /var/unbound/unbound.conf dump_infra
9.9.9.10 . ttl 390 ping 20 var 17 rtt 88 rto 88 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
8.8.8.8 . ttl 390 ping 36 var 11 rtt 80 rto 80 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
[2.4.2-RELEASE][root@pfsense.local.lan]/var/unbound:It has never been no matter what product or OS your using to set NS for either the client or the forwarder to use NS that do not respond with the same info.. Normally your talking about common user mistake of pointing a client to a public dns and a local dns.. Since you can never be sure when the client will use any ns you list.. Just because its the top of the list or "primary" does not mean the client will always ask that NS, etc.
So in the case pointing to NS that will block domains that are "bad" and not answer, and then pointing/forwarding to some other NS that does not block "bad" is the same bad practice.. Be it you put them in some arbitrary order you "think" they will be used in.. Your pointing/using NS that could respond with different info - just like 8.8.8.8 is not going to know shit about your active directory domain, so you don't point your clients to 8.8.8.8 and your local AD dns.. You don't set your forwarder to use NS where one might stop you from looking up a bad/malware site and the other will not - since you can never be sure which one your forwarder is going to look for..
But this is not "race" where it asks all of them and uses the fastest response for every query.. dnsmasq can do this, it will ask every single ns listed every single query - and then just uses the first response.. Next query - again ask ALL NS listed, use the fastest response. That is not what unbound does.
-
I got specific configrmation that these are the "filtering" DNS servers for Quad 9:
9.9.9.9
149.112.112.1122620:fe::fe
2620:fe::9 -
John is correct. It is a race, but unbound does not send simultaneous requests.
Testing against randomly generated domain names, the "primary" server flip flops frequently and more often than not, the second server is queried. Perhaps the query time is right on the cusp of "acceptable" to unbound causing the frequent changes? In previous tests, the 1st query was sent and it didn't come back fast enough so Google DNS was queried. Surprisingly, Google DNS still answered first in some instances despite its later start. In full disclosure, I have DNSSEC enabled which may further slow some query/response speeds as well on the handful of domains that use it.
Thanks for the info on Quad9 IPv6 too.
-
You can tweak some of the values in unbound…
Example this prob could help a lot in keeping it only talking to 1 of your NS
infra-cache-min-rtt: <msec>Lower limit for dynamic retransmit timeout calculation in infra-structure cache. Default is 50 milliseconds. Increase this value if using forwarders needing more time to do recursive name resolution.
The reason unbound does this is to help speed up resolving.. Because normally its keeping this info about authoritative NS.. not just forwarders..
[2.4.2-RELEASE][root@pfsense.local.lan]/root: unbound-control -c /var/unbound/unbound.conf dump_infra
2001:660:3005:1::1:2 nl. ttl 401 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
184.85.248.67 akam.net. ttl 227 ping 10 var 74 rtt 306 rto 306 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
204.61.216.50 arin.net. ttl 278 ping 3 var 77 rtt 311 rto 311 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
205.251.197.228 plex.tv. ttl 405 ping 24 var 97 rtt 412 rto 412 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
205.251.197.208 awsdns-13.co.uk. ttl 66 ping 14 var 99 rtt 410 rto 410 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2600:9000:5305:5600::1 awsdns-22.net. ttl 165 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
208.97.182.10 dreamhost.com. ttl 124 ping 4 var 80 rtt 324 rto 324 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
95.100.174.130 akadns.org. ttl 107 ping 4 var 80 rtt 324 rto 324 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2600:9000:5302:6f00::1 awsdns-47.com. ttl 165 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2620:0:30::53 msft.net. ttl 653 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2600:9000:5304:ec00::1 prod.mozaws.net. ttl 227 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
205.251.194.190 plex.tv. ttl 328 ping 4 var 79 rtt 320 rto 320 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
216.239.38.10 googleusercontent.com. ttl 49 ping 4 var 79 rtt 320 rto 320 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
176.97.158.104 inwx.net. ttl 636 ping 2 var 75 rtt 302 rto 302 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2a00:11c0:1010::1 teamviewer.com. ttl 653 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
205.251.193.226 plex.bz. ttl 405 ping 7 var 67 rtt 275 rto 275 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2400:cb00:2049:1::adf5:3b60 inwx.de. ttl 636 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
205.251.198.123 awsdns-59.org. ttl 164 ping 4 var 80 rtt 324 rto 324 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
208.76.45.53 azure-dns.info. ttl 49 ping 4 var 80 rtt 324 rto 324 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
192.43.172.30 net. ttl 44 ping 25 var 98 rtt 417 rto 417 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
2001:503:ba3e::2:30 . ttl 124 ping 0 var 94 rtt 376 rto 376 tA 0 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
205.251.196.49 awsdns-46.org. ttl 166 ping 14 var 99 rtt 410 rto 410 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0
205.251.194.230 awsdns-36.org. ttl 44 ping 9 var 70 rtt 289 rto 289 tA 0 tAAAA 0 tother 0 ednsknown 1 edns 0 delay 0 lame dnssec 0 rec 0 A 0 other 0<snipped the="" pages="" and="" of="" more="" ns="" info="">This way when it has to ask the NS something.com about www again after the ttl expires it can ask the one that responds the fastest, etc.
"I have DNSSEC enabled which may further slow some query/response speeds as well on the handful of domains that use it. "
To be honest unless your actually resolving I don't see any point to this.. The bigger question here is the resolver your upstream forwarder is using, be it quad9, google, open etc. Are they doing dnssec.. if they are you shouldn't have to specifically ask for that info..
At some point in a chain there is a resolver involved.. This is when dnssec becomes viable - since the resolver is going to be the one talking to the listed authoritative NS for the tree down to record your looking for.. It is the one that needs to validate that signed records, etc. All that should happen for the end user is should not get an answer to something they asked for if dnssec fails..
So client asks for www.domain.com, he sends it to his local forwarder, which in turn sends it to quad9 for example. Now quad9 is either forwarding somewhere else in their own infrastructure to the actual resolver or shoot they could be forwarding to googledns for all you know ;) But at some point in the chain there will be a resolver.. This is where the dnssec happens..
Your client asked for www.domain.com.. It doesn't really give 2 shits about dnssec.. It just wants the IP for www.domain.com… If dnssec fails at the resolver, then the record should not be handed back to whoever asked for the record.. So quad9 doesn't have record for your www.domain.com, so your forwarder your client is using doesn't get an answer. So in the end your client doesn't get an answer.. So what is the point actually of having your forwarder even ask for dnssec info?
So per https://www.quad9.net/#/faq#does-quad9-implement-dnssec
Yes. Quad9 provides DNSSEC validation on our 9.9.9.9 resolver. This means that for domains that implement DNSSEC security, the Quad9 system will cryptographically ensure that the response provided matches the intended response of the domain operator. In the event of a cryptographic failure, our system will not return an answer at all.
So in that case, what dnssec info is your unbound going to be getting back? What is the point of asking your forwarder for it - since be it you ask for dnssec info or not, the resolver is doing it for you.. So there is no need to have it enabled in unbound when forwarding to be honest.</snipped></msec>
-
After decades of doing this, I have found it is simply better to just be sure all configured name servers on a client give the same answers to the same requests. You can never count on them being queried in any particular order and, even if you can, if the "primary" is down and you get a bad answer from the "secondary" you are now looking at bad cache on that client.
-
^ exactly.. My whole point with just more wording ;)
