Unbound DNS Resolver - Not caching?
-
@kpa:
You're missing the fact that the records have a TTL (time to live) of only 300 seconds (the second column in the answer section) as set by the authoritative name server for cnn.com. After that time has passed the records are considered invalid and dropped from the cache.
Thank you, this makes perfect sense. I was under the impression that Unbound would cache for a longer time, than specified by the "dig" command.
However I still have no idea why nothing is being cached when I use my clients to do the lookup :-[
-
Try different clients and with different FQDNs, windows has its own DNS cache service that might skew the results.
-
" I was under the impression that Unbound would cache for a longer time"
that would really break the whole point of running a resolver and getting your info right from the authoritative server.. Caching nameserver that doesn't respect the TTL is BAD nameserver..
As mentioned your clients are going to have their own cache, which will in turn reflect the ttl..
So if unbound resolves say www.cnn.com and from the authoritative server it got 300 seconds for TTL.. That is how long it will last, so lets say 60 seconds later client 2 asks unbound for www.cnn.com It will see a TTL of 240 seconds and the local dns client will only cache it for that amount of time.
Now client 3 asks unbound for www.cnn.com another 60 seconds later, that ttl for that query would only be 180 seconds.
If your wondering about the cache on your local clients for your own hosts that are in your dhcp or overrides, just do a dig to pfsense for whatever record your trying to look up, I believe its 3600 seconds.
Now back to min TTL you can change that in the advanced section, so for example your min ttl would be 1 hour vs say 300 seconds like some sites. But keep in mind these sites with very short ttls normally have them that way for a reason. Like the ips are very likely to change, etc..
if your saying when you query something on a client its not being cached - maybe your clients not asking unbound on pfsense?
So I just duplicated your test.. I check the cache for cnn.com = nothing, I then on a client did a query to unbound asking for cnn.com got a response, then checked the cache again and yup its in there.
If you notice the long response time.. I am remote vpn into my network, so that query to unbound was my machine connected via a vpn, on a hotspot off my phone connection here in TN, to my home in Chicagoland..
You will also notice the TTL time… Looks like it took me 7 seconds from the time I did the query on the client until I checked the cache ;)
Notice pic 2 is after the 300 had expired, and now only the DS and NS records are left because they have longer TTLs
-
Thanks a lot for your in depth reply :)
I seem to have it working now, but I don't understand why I have so many "extra" entries when I do the "dump-cache" | grep cnn.com command, compared to your screenshot.
my domain is "int.xxx.com", and seems to be all over the place…
[2.3.1-RELEASE][root@pfSense.int.xxx.com]/var/log: unbound-control -c /var/unbound/unbound.conf dump_cache | grep cnn.com cnn.com. 172798 IN NS ns3.timewarner.net. cnn.com. 172798 IN NS ns1.timewarner.net. cnn.com. 172798 IN NS ns1.p42.dynect.net. cnn.com. 172798 IN NS ns2.p42.dynect.net. cnn.com. 298 IN A 157.166.226.26 cnn.com. 298 IN A 157.166.226.25 cnn.com. 3598 IN SOA ns1.timewarner.net. hostmaster.turner.com. 2016051603 28800 7200 604800 3600 msg cnn.com.xxx.com. IN AAAA 32899 1 3598 3 0 1 0 msg cnn.com.xxx.com. IN A 32899 1 3598 3 0 1 0 msg cnn.com.int.xxx.com. IN A 32899 1 3598 3 0 1 0 msg cnn.com. IN AAAA 32896 1 3598 3 0 1 0 cnn.com. IN SOA 4 msg cnn.com. IN A 32896 1 298 3 1 1 0 cnn.com. IN A 0 cnn.com. IN NS 0 msg cnn.com.int.xxx.com. IN AAAA 32899 1 3598 3 0 1 0EDIT: If I "dig cnn.com" directly from pfsense console, like this "dig @10.11.12.1 cnn.com" the "msg" entries does NOT appear in the cache. On the other hand, they do appear when I use a client, ie. "nslookup cnn.com".
-
did you snip out something.. or is that really xxx.com ?
The AAAA is easy, whatever you were using to go to cnn.com did a AAAA query (ipv6)..
as to the xxx.com I would guess you got something working through a suffix search and adding that to your queries.. And the suffix your using is actually resolvable on the public side
So your suffix is prob int.xxx.com, so it did query for those, and it also did query for the parent of that xxx.com which it attaches to what you really looked for.. Which is why you see
cnn.com.int.xxx.com
cnn.com.xxx.comWith both A and AAAA
I would guess your using int.xxx.com as your internal domain. But is that really xxx or did you snip what it really was?
-
Ah yes, that makes sense.
I'm using int.domain.com as my internal home domain, while domain.com is an actual resolvable domain. That's probably why.
Does this mean that I'm doing a public lookup for my internal domain every time I make a lookup? Won't that double my response time on DNS queries?
I guess this is normal behavior and I shouldn't do anything about it?Unfortunately I just snipped the actual domain, I don't own xxx.com :'( :'( :'( ::) ;D
EDIT: "msg" entries only being added when using "nslookup". So browsing and other services does not seem to create these entries.
-
Well depends on your dns client and how you have your suffix search setup.. Depending on version of nslookup and you don't put the trailing . on your query then sure it might walk up your suffix list.
You could setup debug on it and watch what it does for example.
at your nslookup prompt do a
set debug
or better even for more info
set d2Now you will see what it asks and what it gets back. Browsers normally wont walk your suffix list, other applications might all depends.
If you really want to have some fun.. Do a wireshark so you can actually see the queries and responses. This is fun on the wan of pfsense so you can watch it walk from the roots down as it finds what your looking for.
DNS is one of my favorite things to play with… If you have any questions - I could talk about dns for hours and hours ;) heheheh especially if you get some beer and bourbon in me...
To be honest not a real fan of nslookup and its limited info and features. Dig is better tool, you can install on windows machine as well.. So you don't have to be limited to nslookup.
As to doing external queries.. Since unbound is not an authoritative server, and it has no record for what you did a query for - yes it would try and find it via the roots, and walk down.. Normally those suffix searches would be rare, but guess depending on your client sure if asking unbound for cnn.com.int.domain.com it would go and try and find that. One way to limit how far down the tree that would go would be to use a non public tld, for example i use .lan.. So as if I looked for cnn.com.local.lan unbound would get a NX back right away from roots when it was looking for the NS of .lan
But there should be a way to keep unbound from even asking for that if it does happen. Have to take a look see..
-
Thanks again, set d2 is excellent :)
It does indeed query for my domain 4 times. I'll mess around with this, quite interesting, as you point out
I'll swoosh by the states with a gallon of bourbon, it's merely an 8 hour flight ;D
-
see my edit.. I add some stuff about those queries going outbound.. I am heading back to the hotel, but will take a look tonight on if that really goes out and how to stop your local domain from going outbound.
I know for sure that my .lan tld would get past asking roots for it.. But just from wanting to be efficient and dnsleaking security concerns depending how tight your tinfoil hat is, if you try and search for blablah without a domain you wouldn't want it asking all the outside ns for blahblah.int.domain.com and then blahblah.doman.com
There might be something already preventing that, but will double check it.
-
Thanks a lot for your great replies, really helps me understand whats going on!
It would be nice if it would ignore the queries for the internal domain.
I tried setting```
server:private-domain: "int.domain.com"It might have something to do with this autogenerated line? **local-zone: "int.domain.com" transparent** Maybe it should be "static" in stead?? -
You are correct, static will return NX for a query that in your local domain that does not exist. To be honest this is prob a better default than transparent for the tinfoil hat dns guys that are concerned with leaking queries ;)
If set to the default transparent and you query something in your local domain, ie I did a query for noname.local.lan I got back NX from root with SOA.. You can verify that with simple sniff on your wan.
If you set this to static, you just get back NX from pfsense for stuff you query that is not in your local domain data. So yes this should stop queries outbound when suffix search that includes your local domain. You could also set it up to return your SOA if you wanted for that domain.. Just have to create the local data I believe.
But since you have a local subdomain setup, ie yours is int.domain.com, my take would be this would stop the query for cnn.com.int.domain.com, but I am not 100% sure on your query cnn.com.domain.com as you walk down your suffix search. I would test it to be sure.
Is there some specific reason you went with a subdomin, and use of a public tld? I find it better to use non public tld for local domains. But each their own, maybe you have a specific reason.. Lets say you owned domain.com, for me locally I would use say domain.lan, using of .local is problematic since apple ruined it for everyone ;)
But .locallan wold be fine.. anything really as your tld that is not public.. Which there are more and more of all the time. I find it unlikely that they would ever allow .lan - but you never know now that they let you register your own TLD if you have the $ maybe someone will scoop that one up?
-
I took the plunge and renamed my domain to "domain.lan". There was no particular reason for the "int.domain.com" domain name, so I decided to change it, to avoid further confusion.
It didn't remove the "msg" entries though, they're still there, just with the .lan extension in stead.
I have tried the different "local-zone" types (static, typetransparent), without any change in the way the results are returned. I've been using the unbound.conf(5) man page for reference.
I guess the answer here is that nslookup on Windows will include the DNS suffix search to the upstream NS. Unbound sees this as a legitimate query and forwards it. No harm done really, since it's not affecting a standard DNS->IP lookup using browsers, tablets, phones, etc.
-
If u change to static it will not send upstream. Keep in mind u will have cache so once u change u have to query new host or wait til ttl expires
-
Using "Static":
orange.com. 86398 IN A 185.63.192.20 orange.com. 3598 IN SOA a4.nstld.com. dnssupport.verisign-grs.com. 2284794609 28800 7200 1209600 86400 orange.com. 86398 IN NS j4.nstld.com. orange.com. 86398 IN NS k4.nstld.com. orange.com. 86398 IN NS a4.nstld.com. orange.com. 86398 IN NS l4.nstld.com. orange.com. 86398 IN NS h4.nstld.com. orange.com. 86398 IN NS g4.nstld.com. orange.com. 86398 IN NS f4.nstld.com. msg orange.com. IN A 32896 1 86398 3 1 1 0 orange.com. IN A 0 orange.com. IN NS 0 msg orange.com. IN AAAA 32896 1 3598 3 0 1 0 orange.com. IN SOA 4Using "Transparent":
blue.com. 1794 IN A 104.236.7.74 blue.com. 1794 IN SOA ns1.digitalocean.com. hostmaster.blue.com. 1429203160 10800 3600 604800 1800 blue.com. 172794 IN NS ns1.digitalocean.com. blue.com. 172794 IN NS ns2.digitalocean.com. blue.com. 172794 IN NS ns3.digitalocean.com. msg blue.com.domain.lan. IN AAAA 32899 1 3594 4 0 3 0 msg blue.com.domain.lan. IN A 32899 1 3594 4 0 3 0 msg blue.com. IN A 32896 1 1794 3 1 0 0 blue.com. IN A 0 msg blue.com. IN AAAA 32896 1 1794 3 0 1 0 blue.com. IN SOA 4It still creates the "msg" entries, but omits the domain.lan when using "Static". Do you suggest I keep it at "Static"??
-
Huh?? What does static have to do with looking up domains? on the public internet??
What is your concern with msg? Yes you still did a query for blue.com.domain.com to unbound.. That is your client, that has nothing to do with unound.. What you don't want is unbound sending that query blue.com.domain.com upstream.. Which as static it will not, as transparent it will.
-
I get it. Was just wondering if static would be a better choice for me, rather than transparent.
Thanks for you assistance :)
-
Depends.. If you you have domain.lan, and you have something that adds suffix do you want that searching going out to the roots to try and look up?
So for example if your client asks for cnn.com.domain.lan do you want unbound to try and find that? Or do you want it to respond with NX since it has no record for it?
If you look for nohostyouhave.domain.lan do you want unbound to ask roots trying to find it?
So I run static, with SOA entry.. So when I try and lookup something that is not in my dns but in my domain local.lan I get this.. so if something gets looked up and for whatever reason the app or whatever adds my local suffix it to it, unbound just sends back NX vs sending it on anywhere. 2nd pic example
All comes down to what you want..
-
I would like to have it configured, so I don't send those request out. So I guess "Static" with SOA is the way to go.
How to add SOA for my domain in unbound?
-
Another thing, that I'm hoping you might have the answer to :)
After enabling the DNS resolver, I'm getting hundreds of firewall entries concerning DNS.
The source IP's seem to be DNS servers operated by my ISP, but I don't know why they're trying to return a query to me on a non-established port. I'm not using the ISP DNS servers, I should be querying root servers only.
I have attached a screenshot of the firewall log.
-
You can create a SOA record in the advanced option.. Attached you will see mine.
As to those blocks.. I would sniff on your wan and see what is going on.. Grab those packets and open them up in wireshark, etc.. What is the data in them? If something was actually creating the queries, then they shouldn't be blocked they should be allowed, etc. So yeah its a bit odd.. Maybe your isp is trying to do some sort of intercept on your dns queries and sending them back? So your not actually talking to roots.. But normally in this case queries fail..
As I said before watching dns traffic on your wan with a sniffer can be quite enlightening to what is actually going on.
The include is something you don't need, that is me just loading up domains I redirect to loopback to block ad domains.. pfblocker does such a thing, but it redirects to service running on pfsense that serves up 1x1 image, not something I need/want.. So I just send back loopback if anything tries to lookup a ad domain, etc.
> dig something.zmedia.com ; <<>> DiG 9.10.3-P4 <<>> something.zmedia.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30873 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;something.zmedia.com. IN A ;; ANSWER SECTION: something.zmedia.com. 3600 IN A 127.0.0.1 ;; Query time: 0 msec ;; SERVER: 192.168.9.253#53(192.168.9.253) ;; WHEN: Sun May 22 06:55:20 Central Daylight Time 2016 ;; MSG SIZE rcvd: 65
