Unbound DNS Resolver - Not caching?

jim82

Dear community,

I'm using Unbound in Resolver mode on v2.3.1, where the point is to query root servers only and then cache the results(AFAIK).

However I have a suspicion that mine is not working as intended.

Here's what I'm doing:

1) From pfsense console

unbound-control -c /var/unbound/unbound.conf dump_cache | grep cnn.com

This returns nothing….

From windows host with only pfsense as DNS

nslookup cnn.com
Server:  pfsense-vlan12gw.xxx.com
Address:  10.11.12.1

Non-authoritative answer:
Name:    cnn.com
Addresses:  157.166.226.26
          157.166.226.25

This resolves just fine

From pfsense console

unbound-control -c /var/unbound/unbound.conf dump_cache | grep cnn.com

This returns no results. I was under the impression that Unbound would cache the result I performed on my Windows host, for future uses, until the TTL expired.

Am I missing something here? Check also my screenshots below for General page and Unbound page

The only way to populate the cache with entries is to perform a "dig" directly on the console, ie.


[2.3.1-RELEASE][root@pfSense.xxx.com]/root: unbound-control -c /var/unbound/unbound.conf dump_cache | grep cnn.com
[2.3.1-RELEASE][root@pfSense.xxx.com]/root: dig cnn.com

; <<>> DiG 9.10.3-P4 <<>> cnn.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30949
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 4, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;cnn.com.                       IN      A

;; ANSWER SECTION:
cnn.com.                300     IN      A       157.166.226.26
cnn.com.                300     IN      A       157.166.226.25

;; AUTHORITY SECTION:
cnn.com.                172800  IN      NS      ns3.timewarner.net.
cnn.com.                172800  IN      NS      ns1.timewarner.net.
cnn.com.                172800  IN      NS      ns2.p42.dynect.net.
cnn.com.                172800  IN      NS      ns1.p42.dynect.net.

;; Query time: 187 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Thu May 19 21:01:43 CEST 2016
;; MSG SIZE  rcvd: 165

[2.3.1-RELEASE][root@pfSense.xxx.com]/root: unbound-control -c /var/unbound/unbound.conf dump_cache | grep cnn.com
cnn.com.        172797  IN      NS      ns3.timewarner.net.
cnn.com.        172797  IN      NS      ns1.timewarner.net.
cnn.com.        172797  IN      NS      ns2.p42.dynect.net.
cnn.com.        172797  IN      NS      ns1.p42.dynect.net.
cnn.com.        297     IN      A       157.166.226.26
cnn.com.        297     IN      A       157.166.226.25
msg cnn.com. IN A 32896 1 297 3 1 1 0
cnn.com. IN A 0
cnn.com. IN NS 0

*Additional confusion for me is that even this cached result disappears from the cache after 2-5 minutes :-*

Unbound.conf:


##########################
# Unbound Configuration
##########################

##
# Server configuration
##
server:

chroot: /var/unbound
username: "unbound"
directory: "/var/unbound"
pidfile: "/var/run/unbound.pid"
use-syslog: yes
port: 53
verbosity: 1
hide-identity: yes
hide-version: yes
harden-glue: yes
do-ip4: yes
do-ip6: yes
do-udp: yes
do-tcp: yes
do-daemonize: yes
module-config: "validator iterator"
unwanted-reply-threshold: 0
num-queries-per-thread: 512
jostle-timeout: 200
infra-host-ttl: 900
infra-cache-numhosts: 20000
outgoing-num-tcp: 10
incoming-num-tcp: 10
edns-buffer-size: 4096
cache-max-ttl: 86400
cache-min-ttl: 0
harden-dnssec-stripped: yes
msg-cache-size: 50m
rrset-cache-size: 100m

num-threads: 4
msg-cache-slabs: 4
rrset-cache-slabs: 4
infra-cache-slabs: 4
key-cache-slabs: 4
outgoing-range: 4096
#so-rcvbuf: 4m
auto-trust-anchor-file: /var/unbound/root.key
prefetch: yes
prefetch-key: yes
use-caps-for-id: no
# Statistics
# Unbound Statistics
statistics-interval: 0
extended-statistics: yes
statistics-cumulative: yes

# Interface IP(s) to bind to
interface: 0.0.0.0
interface: ::0
interface-automatic: yes

# Outgoing interfaces to be used
outgoing-interface: 80.197.130.157

# DNS Rebinding
# For DNS Rebinding prevention
private-address: 10.0.0.0/8
private-address: 172.16.0.0/12
private-address: 169.254.0.0/16
private-address: 192.168.0.0/16
private-address: fd00::/8
private-address: fe80::/10

# Access lists
include: /var/unbound/access_lists.conf

# Static host entries
include: /var/unbound/host_entries.conf

# dhcp lease entries
include: /var/unbound/dhcpleases_entries.conf

# Domain overrides
include: /var/unbound/domainoverrides.conf

# Unbound custom options
server:private-domain: "plex.direct"

###
# Remote Control Config
###
include: /var/unbound/remotecontrol.conf

Stats output


[2.3.1-RELEASE][root@pfSense.xxx.com]/root: unbound-control -c /var/unbound/unbound.conf stats
thread0.num.queries=87
thread0.num.cachehits=36
thread0.num.cachemiss=51
thread0.num.prefetch=0
thread0.num.recursivereplies=51
thread0.requestlist.avg=1.31373
thread0.requestlist.max=17
thread0.requestlist.overwritten=0
thread0.requestlist.exceeded=0
thread0.requestlist.current.all=0
thread0.requestlist.current.user=0
thread0.recursion.time.avg=0.111007
thread0.recursion.time.median=0.055296
thread0.tcpusage=0
thread1.num.queries=73
thread1.num.cachehits=30
thread1.num.cachemiss=43
thread1.num.prefetch=1
thread1.num.recursivereplies=43
thread1.requestlist.avg=1.25
thread1.requestlist.max=11
thread1.requestlist.overwritten=0
thread1.requestlist.exceeded=0
thread1.requestlist.current.all=0
thread1.requestlist.current.user=0
thread1.recursion.time.avg=0.103791
thread1.recursion.time.median=0.0321378
thread1.tcpusage=0
thread2.num.queries=15
thread2.num.cachehits=6
thread2.num.cachemiss=9
thread2.num.prefetch=0
thread2.num.recursivereplies=9
thread2.requestlist.avg=0.777778
thread2.requestlist.max=3
thread2.requestlist.overwritten=0
thread2.requestlist.exceeded=0
thread2.requestlist.current.all=0
thread2.requestlist.current.user=0
thread2.recursion.time.avg=0.211279
thread2.recursion.time.median=0.196608
thread2.tcpusage=0
thread3.num.queries=27
thread3.num.cachehits=13
thread3.num.cachemiss=14
thread3.num.prefetch=0
thread3.num.recursivereplies=14
thread3.requestlist.avg=0.714286
thread3.requestlist.max=7
thread3.requestlist.overwritten=0
thread3.requestlist.exceeded=0
thread3.requestlist.current.all=0
thread3.requestlist.current.user=0
thread3.recursion.time.avg=0.087440
thread3.recursion.time.median=0.032768
thread3.tcpusage=0
total.num.queries=202
total.num.cachehits=85
total.num.cachemiss=117
total.num.prefetch=1
total.num.recursivereplies=117
total.requestlist.avg=1.17797
total.requestlist.max=17
total.requestlist.overwritten=0
total.requestlist.exceeded=0
total.requestlist.current.all=0
total.requestlist.current.user=0
total.recursion.time.avg=0.113248
total.recursion.time.median=0.0792025
total.tcpusage=0
time.now=1463684351.062592
time.up=3650.290381
time.elapsed=249.783344
mem.total.sbrk=0
mem.cache.rrset=285981
mem.cache.message=152744
mem.mod.iterator=16548
mem.mod.validator=91970
histogram.000000.000000.to.000000.000001=18
histogram.000000.000001.to.000000.000002=0
histogram.000000.000002.to.000000.000004=0
histogram.000000.000004.to.000000.000008=0
histogram.000000.000008.to.000000.000016=0
histogram.000000.000016.to.000000.000032=0
histogram.000000.000032.to.000000.000064=0
histogram.000000.000064.to.000000.000128=0
histogram.000000.000128.to.000000.000256=0
histogram.000000.000256.to.000000.000512=0
histogram.000000.000512.to.000000.001024=0
histogram.000000.001024.to.000000.002048=0
histogram.000000.002048.to.000000.004096=0
histogram.000000.004096.to.000000.008192=1
histogram.000000.008192.to.000000.016384=6
histogram.000000.016384.to.000000.032768=25
histogram.000000.032768.to.000000.065536=15
histogram.000000.065536.to.000000.131072=14
histogram.000000.131072.to.000000.262144=17
histogram.000000.262144.to.000000.524288=20
histogram.000000.524288.to.000001.000000=1
histogram.000001.000000.to.000002.000000=0
histogram.000002.000000.to.000004.000000=0
histogram.000004.000000.to.000008.000000=0
histogram.000008.000000.to.000016.000000=0
histogram.000016.000000.to.000032.000000=0
histogram.000032.000000.to.000064.000000=0
histogram.000064.000000.to.000128.000000=0
histogram.000128.000000.to.000256.000000=0
histogram.000256.000000.to.000512.000000=0
histogram.000512.000000.to.001024.000000=0
histogram.001024.000000.to.002048.000000=0
histogram.002048.000000.to.004096.000000=0
histogram.004096.000000.to.008192.000000=0
histogram.008192.000000.to.016384.000000=0
histogram.016384.000000.to.032768.000000=0
histogram.032768.000000.to.065536.000000=0
histogram.065536.000000.to.131072.000000=0
histogram.131072.000000.to.262144.000000=0
histogram.262144.000000.to.524288.000000=0
num.query.type.A=130
num.query.type.PTR=12
num.query.type.AAAA=60
num.query.class.IN=202
num.query.opcode.QUERY=202
num.query.tcp=0
num.query.tcpout=0
num.query.ipv6=0
num.query.flags.QR=0
num.query.flags.AA=0
num.query.flags.TC=0
num.query.flags.RD=202
num.query.flags.RA=0
num.query.flags.Z=0
num.query.flags.AD=0
num.query.flags.CD=0
num.query.edns.present=0
num.query.edns.DO=0
num.answer.rcode.NOERROR=163
num.answer.rcode.FORMERR=0
num.answer.rcode.SERVFAIL=0
num.answer.rcode.NXDOMAIN=39
num.answer.rcode.NOTIMPL=0
num.answer.rcode.REFUSED=0
num.answer.rcode.nodata=34
num.answer.secure=0
num.answer.bogus=0
num.rrset.bogus=0
unwanted.queries=0
unwanted.replies=0
msg.cache.count=396
rrset.cache.count=812
infra.cache.count=5019
key.cache.count=30

unbound_pfsweb.PNG_thumb

pfsweb_general.PNG_thumb

unbound_advanced1.PNG_thumb

unbound_advanced2.PNG_thumb

kpa

You're missing the fact that the records have a TTL (time to live) of only 300 seconds (the second column in the answer section) as set by the authoritative name server for cnn.com. After that time has passed the records are considered invalid and dropped from the cache.

jim82

@kpa:

You're missing the fact that the records have a TTL (time to live) of only 300 seconds (the second column in the answer section) as set by the authoritative name server for cnn.com. After that time has passed the records are considered invalid and dropped from the cache.

Thank you, this makes perfect sense. I was under the impression that Unbound would cache for a longer time, than specified by the "dig" command.

However I still have no idea why nothing is being cached when I use my clients to do the lookup :-[

kpa

Try different clients and with different FQDNs, windows has its own DNS cache service that might skew the results.

johnpoz

" I was under the impression that Unbound would cache for a longer time"

that would really break the whole point of running a resolver and getting your info right from the authoritative server.. Caching nameserver that doesn't respect the TTL is BAD nameserver..

As mentioned your clients are going to have their own cache, which will in turn reflect the ttl..

So if unbound resolves say www.cnn.com and from the authoritative server it got 300 seconds for TTL.. That is how long it will last, so lets say 60 seconds later client 2 asks unbound for www.cnn.com It will see a TTL of 240 seconds and the local dns client will only cache it for that amount of time.

Now client 3 asks unbound for www.cnn.com another 60 seconds later, that ttl for that query would only be 180 seconds.

If your wondering about the cache on your local clients for your own hosts that are in your dhcp or overrides, just do a dig to pfsense for whatever record your trying to look up, I believe its 3600 seconds.

Now back to min TTL you can change that in the advanced section, so for example your min ttl would be 1 hour vs say 300 seconds like some sites. But keep in mind these sites with very short ttls normally have them that way for a reason. Like the ips are very likely to change, etc..

if your saying when you query something on a client its not being cached - maybe your clients not asking unbound on pfsense?

So I just duplicated your test.. I check the cache for cnn.com = nothing, I then on a client did a query to unbound asking for cnn.com got a response, then checked the cache again and yup its in there.

If you notice the long response time.. I am remote vpn into my network, so that query to unbound was my machine connected via a vpn, on a hotspot off my phone connection here in TN, to my home in Chicagoland..

You will also notice the TTL time… Looks like it took me 7 seconds from the time I did the query on the client until I checked the cache ;)

Notice pic 2 is after the 300 had expired, and now only the DS and NS records are left because they have longer TTLs

cache.png_thumb

after300seconds.png_thumb

jim82

Thanks a lot for your in depth reply :)

I seem to have it working now, but I don't understand why I have so many "extra" entries when I do the "dump-cache" | grep cnn.com command, compared to your screenshot.

my domain is "int.xxx.com", and seems to be all over the place…

[2.3.1-RELEASE][root@pfSense.int.xxx.com]/var/log: unbound-control -c /var/unbound/unbound.conf dump_cache | grep cnn.com
cnn.com.        172798  IN      NS      ns3.timewarner.net.
cnn.com.        172798  IN      NS      ns1.timewarner.net.
cnn.com.        172798  IN      NS      ns1.p42.dynect.net.
cnn.com.        172798  IN      NS      ns2.p42.dynect.net.
cnn.com.        298     IN      A       157.166.226.26
cnn.com.        298     IN      A       157.166.226.25
cnn.com.        3598    IN      SOA     ns1.timewarner.net. hostmaster.turner.com. 2016051603 28800 7200 604800 3600
msg cnn.com.xxx.com. IN AAAA 32899 1 3598 3 0 1 0
msg cnn.com.xxx.com. IN A 32899 1 3598 3 0 1 0
msg cnn.com.int.xxx.com. IN A 32899 1 3598 3 0 1 0
msg cnn.com. IN AAAA 32896 1 3598 3 0 1 0
cnn.com. IN SOA 4
msg cnn.com. IN A 32896 1 298 3 1 1 0
cnn.com. IN A 0
cnn.com. IN NS 0
msg cnn.com.int.xxx.com. IN AAAA 32899 1 3598 3 0 1 0

EDIT: If I "dig cnn.com" directly from pfsense console, like this "dig @10.11.12.1 cnn.com" the "msg" entries does NOT appear in the cache. On the other hand, they do appear when I use a client, ie. "nslookup cnn.com".

johnpoz

did you snip out something.. or is that really xxx.com ?

The AAAA is easy, whatever you were using to go to cnn.com did a AAAA query (ipv6)..

as to the xxx.com I would guess you got something working through a suffix search and adding that to your queries.. And the suffix your using is actually resolvable on the public side

So your suffix is prob int.xxx.com, so it did query for those, and it also did query for the parent of that xxx.com which it attaches to what you really looked for.. Which is why you see

cnn.com.int.xxx.com
cnn.com.xxx.com

With both A and AAAA

I would guess your using int.xxx.com as your internal domain. But is that really xxx or did you snip what it really was?

jim82

Ah yes, that makes sense.

I'm using int.domain.com as my internal home domain, while domain.com is an actual resolvable domain. That's probably why.

Does this mean that I'm doing a public lookup for my internal domain every time I make a lookup? Won't that double my response time on DNS queries?
I guess this is normal behavior and I shouldn't do anything about it?

Unfortunately I just snipped the actual domain, I don't own xxx.com :'( :'( :'( ::) ;D

EDIT: "msg" entries only being added when using "nslookup". So browsing and other services does not seem to create these entries.

johnpoz

Well depends on your dns client and how you have your suffix search setup.. Depending on version of nslookup and you don't put the trailing . on your query then sure it might walk up your suffix list.

You could setup debug on it and watch what it does for example.

at your nslookup prompt do a
set debug
or better even for more info
set d2

Now you will see what it asks and what it gets back. Browsers normally wont walk your suffix list, other applications might all depends.

If you really want to have some fun.. Do a wireshark so you can actually see the queries and responses. This is fun on the wan of pfsense so you can watch it walk from the roots down as it finds what your looking for.

DNS is one of my favorite things to play with… If you have any questions - I could talk about dns for hours and hours ;) heheheh especially if you get some beer and bourbon in me...

To be honest not a real fan of nslookup and its limited info and features. Dig is better tool, you can install on windows machine as well.. So you don't have to be limited to nslookup.

As to doing external queries.. Since unbound is not an authoritative server, and it has no record for what you did a query for - yes it would try and find it via the roots, and walk down.. Normally those suffix searches would be rare, but guess depending on your client sure if asking unbound for cnn.com.int.domain.com it would go and try and find that. One way to limit how far down the tree that would go would be to use a non public tld, for example i use .lan.. So as if I looked for cnn.com.local.lan unbound would get a NX back right away from roots when it was looking for the NS of .lan

But there should be a way to keep unbound from even asking for that if it does happen. Have to take a look see..

jim82

Thanks again, set d2 is excellent :)

It does indeed query for my domain 4 times. I'll mess around with this, quite interesting, as you point out

I'll swoosh by the states with a gallon of bourbon, it's merely an 8 hour flight ;D

johnpoz

see my edit.. I add some stuff about those queries going outbound.. I am heading back to the hotel, but will take a look tonight on if that really goes out and how to stop your local domain from going outbound.

I know for sure that my .lan tld would get past asking roots for it.. But just from wanting to be efficient and dnsleaking security concerns depending how tight your tinfoil hat is, if you try and search for blablah without a domain you wouldn't want it asking all the outside ns for blahblah.int.domain.com and then blahblah.doman.com

There might be something already preventing that, but will double check it.

jim82

Thanks a lot for your great replies, really helps me understand whats going on!

It would be nice if it would ignore the queries for the internal domain.

I tried setting```
server:private-domain: "int.domain.com"


It might have something to do with this autogenerated line?

**local-zone: "int.domain.com" transparent**

Maybe it should be "static" in stead??

johnpoz

You are correct, static will return NX for a query that in your local domain that does not exist. To be honest this is prob a better default than transparent for the tinfoil hat dns guys that are concerned with leaking queries ;)

If set to the default transparent and you query something in your local domain, ie I did a query for noname.local.lan I got back NX from root with SOA.. You can verify that with simple sniff on your wan.

If you set this to static, you just get back NX from pfsense for stuff you query that is not in your local domain data. So yes this should stop queries outbound when suffix search that includes your local domain. You could also set it up to return your SOA if you wanted for that domain.. Just have to create the local data I believe.

But since you have a local subdomain setup, ie yours is int.domain.com, my take would be this would stop the query for cnn.com.int.domain.com, but I am not 100% sure on your query cnn.com.domain.com as you walk down your suffix search. I would test it to be sure.

Is there some specific reason you went with a subdomin, and use of a public tld? I find it better to use non public tld for local domains. But each their own, maybe you have a specific reason.. Lets say you owned domain.com, for me locally I would use say domain.lan, using of .local is problematic since apple ruined it for everyone ;)

But .locallan wold be fine.. anything really as your tld that is not public.. Which there are more and more of all the time. I find it unlikely that they would ever allow .lan - but you never know now that they let you register your own TLD if you have the $ maybe someone will scoop that one up?

jim82

I took the plunge and renamed my domain to "domain.lan". There was no particular reason for the "int.domain.com" domain name, so I decided to change it, to avoid further confusion.

It didn't remove the "msg" entries though, they're still there, just with the .lan extension in stead.

I have tried the different "local-zone" types (static, typetransparent), without any change in the way the results are returned. I've been using the unbound.conf(5) man page for reference.

I guess the answer here is that nslookup on Windows will include the DNS suffix search to the upstream NS. Unbound sees this as a legitimate query and forwards it. No harm done really, since it's not affecting a standard DNS->IP lookup using browsers, tablets, phones, etc.

johnpoz

If u change to static it will not send upstream. Keep in mind u will have cache so once u change u have to query new host or wait til ttl expires

jim82

Using "Static":

orange.com.     86398   IN      A       185.63.192.20
orange.com.     3598    IN      SOA     a4.nstld.com. dnssupport.verisign-grs.com. 2284794609 28800 7200 1209600 86400
orange.com.     86398   IN      NS      j4.nstld.com.
orange.com.     86398   IN      NS      k4.nstld.com.
orange.com.     86398   IN      NS      a4.nstld.com.
orange.com.     86398   IN      NS      l4.nstld.com.
orange.com.     86398   IN      NS      h4.nstld.com.
orange.com.     86398   IN      NS      g4.nstld.com.
orange.com.     86398   IN      NS      f4.nstld.com.
msg orange.com. IN A 32896 1 86398 3 1 1 0
orange.com. IN A 0
orange.com. IN NS 0
msg orange.com. IN AAAA 32896 1 3598 3 0 1 0
orange.com. IN SOA 4

Using "Transparent":


blue.com.       1794    IN      A       104.236.7.74
blue.com.       1794    IN      SOA     ns1.digitalocean.com. hostmaster.blue.com. 1429203160 10800 3600 604800 1800
blue.com.       172794  IN      NS      ns1.digitalocean.com.
blue.com.       172794  IN      NS      ns2.digitalocean.com.
blue.com.       172794  IN      NS      ns3.digitalocean.com.
msg blue.com.domain.lan. IN AAAA 32899 1 3594 4 0 3 0
msg blue.com.domain.lan. IN A 32899 1 3594 4 0 3 0
msg blue.com. IN A 32896 1 1794 3 1 0 0
blue.com. IN A 0
msg blue.com. IN AAAA 32896 1 1794 3 0 1 0
blue.com. IN SOA 4

It still creates the "msg" entries, but omits the domain.lan when using "Static". Do you suggest I keep it at "Static"??

johnpoz

Huh?? What does static have to do with looking up domains? on the public internet??

What is your concern with msg? Yes you still did a query for blue.com.domain.com to unbound.. That is your client, that has nothing to do with unound.. What you don't want is unbound sending that query blue.com.domain.com upstream.. Which as static it will not, as transparent it will.

jim82

I get it. Was just wondering if static would be a better choice for me, rather than transparent.

Thanks for you assistance :)

johnpoz

Depends.. If you you have domain.lan, and you have something that adds suffix do you want that searching going out to the roots to try and look up?

So for example if your client asks for cnn.com.domain.lan do you want unbound to try and find that? Or do you want it to respond with NX since it has no record for it?

If you look for nohostyouhave.domain.lan do you want unbound to ask roots trying to find it?

So I run static, with SOA entry.. So when I try and lookup something that is not in my dns but in my domain local.lan I get this.. so if something gets looked up and for whatever reason the app or whatever adds my local suffix it to it, unbound just sends back NX vs sending it on anywhere. 2nd pic example

All comes down to what you want..

examplestatic.png_thumb

2nexamplestatic.png_thumb

jim82

I would like to have it configured, so I don't send those request out. So I guess "Static" with SOA is the way to go.

How to add SOA for my domain in unbound?