DNS over TLS forwarding howto

PertFlavus

EDIT:
Better info can be found here:
https://www.netgate.com/blog/dns-over-tls-with-pfsense.html

Hey all,

I was curious about dns over tls, so I figured I'd try it out.

To get it to work with pfsense using DNS Resolver I made the following changes:

disable forwarding mode
Add these custom options:


server:
ssl-upstream: yes
do-tcp: yes
forward-zone:
  name: "."
  forward-addr: {ipv4address}@853
  forward-addr: {ipv6address}@853

You can find servers to try out here:
https://dnsprivacy.org/wiki/display/DP/DNS+Privacy+Test+Servers

You can also roll your own as I did, which they also provide guides for.

If you have ideas of a better way to do this let me know! cheers!

chrcoluk

this is still on my todo list for my network :) currently using dnscrypt but I expect doing it natively in unbound will be faster.

Thanks for your information. :)

johnpoz

So let me get this right… So your concerned with your privacy so your going to forward all your dns queries to some random dns server on the public net because it was on a list in some wiki article? But your more secure because your isp can not see your queries? Ok yeah see how that makes a lot of sense <rolleyes>This might be valid idea if every NS on the planet supported tls, and then you could actually resolve via tls.. Talk about extra overhead to something that is suppose to be QUICK! and small - why its udp..

If your worried about someone watching and logging all your dns queries - forwarding for sure is not what you should be doing, because your just handing them every single thing you want to lookup.. So why would you not just resolve? Like unbound does out of the box on pfsense. Now the root servers would know when you look up something for specific tld, but you wouldn't go ask them again for anything until you looked up something with a new tld, or ttl expired for the gTLD servers that own that tld.. The gTLD servers for your TLD would know what your looking for but like root they are run by multiple different orgs.. All over the planet.. And then they only get part of what you look for, just the stuff in a specific TLD.. And then only the first query for something in a domain, since once you have that cached you don't go ask them again for records that full under that specific domain, etc.

Its nice of you to post how to do it.. But I just don't get why anyone would want too..</rolleyes>

Velcro

I too have privacy concerns and struggled with trusting OpenDNS vs using the resolver and opted to use resolver and have my DNS requests go thru my VPN only.

I went to Services -> DNS Resolver -> General Settings tab -> Outgoing Network Interfaces -> Selected only my VPN interface (vs WAN and VPN Interface).

Wouldn't this encrypt my DNS traffic? How is using DNScrypt or TLS any better?

Thanks for posting PertFlavus…good discussion!

johnpoz

Sending your dns through your vpn would hide it from your ISP, it would prevent your isp trying to do dns interception. But once the query leaves your vpn endpoint it would be in the clear still. Your vpn provider could be logging all the dns if they so desired..

When you use a vpn, your trusting them not be logging or looking at your traffic because..

dnscrypt just encrypts validates that your asking the forwarder your linking to.. Still run into the problem that you are giving them all of your dns queries, and have to trust them that they are giving you good data, etc. All that does just like the tls thing is hide it from your isp.. And again could prevent dns interception that your isp would be doing.

I really do not get all this concern over dns leaks or god forbid my isp sees my that I looked up www.domain.tld

So do you not have a smart phone - because shoot your provider and really anyone else with access using that phone knows exactly where you are 24/7

Do you not use CC? They know exactly what and where and when you bought a box of condoms and what beer you like, etc.

Do you where a mask when you go outside, since the the camera's that are every where could be doing facial recognition on you. Do you not use automatic tolling because the toll company knows exactly when you pass every toll booth.. For that matter they can track you in your car as you drive around the city via your license plate and all the speed camera's

There is the impression of privacy, then their is reality of it all.. I personally don't really give two shits that my ISP knows that I went to forum.pfsense.org etc.. But that is just me.. What I would be more worried about is the place I get sent to for forum.pfsense.org is actually that - per the domain owners signing their records via dnssec and directly asking the listed authoritative NS.. Which brings a valid point pfsense.org be using dnssec which it is not.

Should prob bring that up to them.. Since they provide unbound using dnssec as their default deployment, would be nice if their own domains were dnssec ;) Same goes for netgate.com which I see also that they are missing their AAAA glue as well.

BBcan177

Can also add "qname-minimisation | -strict" to reduce what gets sent during the resolving process… Should probably be an option in the pfSense Unbound GUI...

https://www.unbound.net/documentation/unbound.conf.html
https://ripe72.ripe.net/archives/video/219/

qname-minimisation: <yes or="" no="">Send minimum amount of information to upstream servers to
enhance privacy. Only sent minimum required labels of the QNAME
and set QTYPE to NS when possible. Best effort approach; full
QNAME and original QTYPE will be sent when upstream replies with
a RCODE other than NOERROR, except when receiving NXDOMAIN from
a DNSSEC signed zone. Default is off.

qname-minimisation-strict: <yes or="" no="">QNAME minimisation in strict mode. Do not fall-back to sending
full QNAME to potentially broken nameservers. A lot of domains
will not be resolvable when this option in enabled. Only use if
you know what you are doing. This option only has effect when
qname-minimisation is enabled. Default is off.</yes></yes>

PertFlavus

@johnpoz:

Sending your dns through your vpn would hide it from your ISP, it would prevent your isp trying to do dns interception. But once the query leaves your vpn endpoint it would be in the clear still. Your vpn provider could be logging all the dns if they so desired..

When you use a vpn, your trusting them not be logging or looking at your traffic because..

dnscrypt just encrypts validates that your asking the forwarder your linking to.. Still run into the problem that you are giving them all of your dns queries, and have to trust them that they are giving you good data, etc. All that does just like the tls thing is hide it from your isp.. And again could prevent dns interception that your isp would be doing.

I really do not get all this concern over dns leaks or god forbid my isp sees my that I looked up www.domain.tld

So do you not have a smart phone - because shoot your provider and really anyone else with access using that phone knows exactly where you are 24/7

Do you not use CC? They know exactly what and where and when you bought a box of condoms and what beer you like, etc.

Do you where a mask when you go outside, since the the camera's that are every where could be doing facial recognition on you. Do you not use automatic tolling because the toll company knows exactly when you pass every toll booth.. For that matter they can track you in your car as you drive around the city via your license plate and all the speed camera's

There is the impression of privacy, then their is reality of it all.. I personally don't really give two shits that my ISP knows that I went to forum.pfsense.org etc.. But that is just me.. What I would be more worried about is the place I get sent to for forum.pfsense.org is actually that - per the domain owners signing their records via dnssec and directly asking the listed authoritative NS.. Which brings a valid point pfsense.org be using dnssec which it is not.

Should prob bring that up to them.. Since they provide unbound using dnssec as their default deployment, would be nice if their own domains were dnssec ;) Same goes for netgate.com which I see also that they are missing their AAAA glue as well.

To be clear, I did it because I could. It was fun. I also was able to host my own dns server, which wouldn't normally be possible, as it uses a different port/tcp. My VPS provider isn't as incentivised to scrape my traffic for ad revenue like google or my ISP are. Encrypting DNS by itself does nothing to keep your internet use private by itself though because https sends the hostname in a SNI request over plain text. That might change with TLS 1.3.

Baby steps John, baby steps. Google's going to get their info one way or another anyhow. (And probably through DNS over TLS! https://www.xda-developers.com/android-dns-over-tls-website-privacy/ ) glares at Android phone

I've considered just sending my internet connection through my VPS using OpenVPN but I don't think it's performance is up to snuff. I also turned off dns over tls for now because of the above points. I just figured I'd share how to do it to show it's possible, and without an FR for support.

johnpoz

I see this use case scenario as prob not all that bad..

I run a vps somewhere. On this vps out in the cloud I run a resolver.. I use dns over tls to talk to this resolver via forwarding from my location..

This hides your dns traffic from your isp.. This also hides your IP from from roots, and the authoritative servers even.

This allows you to use a resolver that you trust - your freaking running it ;) Hides your dns traffic from your isp.. And prevents any so called dns leaks…

PertFlavus

It's not bad, so far but you'll need more than one. Linode has $5 nodes, so if you get two in different data centers you're good to go.

Latency on lookup is noticeable. Quarter secondish. My servers don't support tcp fast open and they're uncached queries, so I couldn't tell you which is causing the delay.

I can post my unbound server configs if you want to give it a try.

@BBcan177
qname-minimization appears to work fine, so I've updated the config. Thank you. =)

johnpoz

"I can post my unbound server configs if you want to give it a try."

Thanks.. But have no real desire to try this at all.. Like I said I don't really care that my isp or roots can see that I go to forum.pfsense.org ;)

Its nice of you to share you info with the shinyhats out there.. But to me this is just waste of time.. You mean it slows down my dns.. Well sure sign me up! ;) lets give that a run hehehehehe

But why do you need two vps nodes? As long as the node is up, you sure don't need 2 of them.

Now what bcan posted about qname-minimisation, this is good way to help out the shinyhat wearers and not add complexity and layers and latency to your dns I would think.. I might play with that a bit to see if have any issues resolving stuff I go to..

PertFlavus

@johnpoz:

Why do you need two vps nodes? As long as the node is up, you sure don't need 2 of them.

Because, at some point, it won't be, and there's not a damned thing you can do about that. It's another con to the waste of time.

In the above config your internet will totally stop working if the dns server you forward to is inaccessible because, say, your vps provider gets ddos'd ;)

johnpoz

My internet could also go down, I could loose power.. There could be a zombie Apocalypse as well ;)

As to the damn thing I could do about sure, if that vps goes down I just resolve normally… No reason to pay for extra vps because I would be worried that my vps provider gets hit with a ddos ;) hehehe

I use 3 different hosts for vpses - none of them have gone down because of ddos ;) tat I can recall They have had maint, sure.. But to be honest pretty freaking impressed with the uptime.. Especially the the main one I use where I have 4 different vps int 3 different data centers, etc.

But sure yes failover planning and redundancy is part of any system that needs to be taken into account sure.

chrcoluk

John personally I run my own dnscrypt endpoint, and I would do the same if I switched to unbound TLS.

In some parts of the world (UK especially) isp's actually intercept and filter DNS queries (yes this would also catch queries using pfsense as the resolver as its outbound port 53 to query authoritative servers) so there is net value to carrying out DNS privacy. So I think in that case even using a 3rd party server would be worthwhile.

johnpoz

If your isp is doing dns interception and doing any sort of injection or filtering to stop you from looking up something then by all means this makes sense.. Be it dnscrypt/tls tunnel - vpn, etc. To get your data past such network.

My guess is they are attempting to block p2p sites, etc. But doesn't matter what they are blocking - blocking whatever it is to me in violation to what they are suppose to be doing which is just providing you a net connection. If you want to lookup up p0rn, p2p, whatever - and its out there.. They shouldn't be messing with your ability to look up the IP that is for sure.

But if all they are doing is logging it.. Then I don't give 2 shits.. If they want to sell it to someone that I seem to like xyz I really don't care. But they better not mess with what is to be returned from the authoritative server.. If they were doing such a thing I would be on a different isp..

PertFlavus

@chrcoluk:

John personally I run my own dnscrypt endpoint, and I would do the same if I switched to unbound TLS.

In some parts of the world (UK especially) isp's actually intercept and filter DNS queries (yes this would also catch queries using pfsense as the resolver as its outbound port 53 to query authoritative servers) so there is net value to carrying out DNS privacy. So I think in that case even using a 3rd party server would be worthwhile.

Damn.. that's terrible.. but why do they stop at dns when they could also filter http/https? I don't suppose you know a good source that describes this? I'd be interested in learning about it. I'm really hoping tls 1.3 includes a way to encrypt sni.

chrcoluk

John yes UK isps commonly block p2p and other undesirable sites, there may be other motives for them to do so also. But it is common practice in the UK sadly on the major isps.

Just wanted to point out in some parts of the world on some isps there is a definite good reason to mask out DNS traffic. :)

johnpoz

So why don't you just run through a vpn and be done with it?

Here is a question for you.. Are they actually doing interception, or is their isp dns is just not returning the stuff they want you not to go to? Its a whole different ball game to just block specific dns in your dns that your running vs intercepting users dns, or blocking outbound on 53..

PertFlavus

@BBcan177:

Can also add "qname-minimisation | -strict" to reduce what gets sent during the resolving process… Should probably be an option in the pfSense Unbound GUI...

https://www.unbound.net/documentation/unbound.conf.html
https://ripe72.ripe.net/archives/video/219/

qname-minimisation: <yes or="" no="">Send minimum amount of information to upstream servers to
enhance privacy. Only sent minimum required labels of the QNAME
and set QTYPE to NS when possible. Best effort approach; full
QNAME and original QTYPE will be sent when upstream replies with
a RCODE other than NOERROR, except when receiving NXDOMAIN from
a DNSSEC signed zone. Default is off.

qname-minimisation-strict: <yes or="" no="">QNAME minimisation in strict mode. Do not fall-back to sending
full QNAME to potentially broken nameservers. A lot of domains
will not be resolvable when this option in enabled. Only use if
you know what you are doing. This option only has effect when
qname-minimisation is enabled. Default is off.</yes></yes>

This looks incredibly easy to implement in the unbound package. I'll see if I can get a pull request for this soon. I will most likely not include a -strict option though as I don't see a reason to have it.

edit: Maybe not so easy. I saw the files to edit in https://github.com/pfsense/pfsense-packages to edit, but I can't find the xml files or the inc files in the new repo, https://github.com/pfsense/FreeBSD-ports =/

kejianshi

For me, unbound solved most of my DNS issues since I get to be my own dns server and the info comes directly from the root servers.

The only way it could get better is if I typed in all the names and IPs by hand… My hands hurt just thinking about it.

tman222

Hi all,

I have been following this thread and reading up a bit more on qname-minimisation. Also found some info about the topic at this source:

https://tools.ietf.org/html/rfc7816

In order to enable this feature in pfSense DNS resolver, it is as simple as adding the appropriate line(s) to unbound.conf and then restarting Unbound? If so, where is unbound.conf located in pfSense?

One thing I'm not quite sure on: Does this still offer protection when the DNS Resolver in pfSense is enabled with forwarding (to e.g. OpenDNS or Google instead of going to the root DNS servers)? Or does it not offer any privacy enhancement in that case?

Thanks in advance for your help, I really appreciate it.

johnpoz

"forwarding (to e.g. OpenDNS or Google instead of going to the root DNS servers)?"

Thought you said you read the RFC?? When you use the forwarder you do not talk to roots, you would have to send the forwarder the FULL thing your looking for, not just the pieces of the fqdn you need to find the authoritative server. So you could ask it for the record..

So in this scenario instead of asking roots hey whats the NS for .com in www.domain.com - its just asks hey whats the NS for .com
Then asks hey NS for .com whats the NS for domain.com, vs asking for NS of www.domain.com

How would that work with a forwarder?

You do not need to edit the conf file directly.. Just add it to the custom options box.

here… This simple.. see attached.

I ask the resolver hey www.testthisdomain.com and it just ask the NS for .com for testthisdomain.com vs the www.testthisdomain.com see attached sniff pic.

dig -x 192.31.80.30 +short
d.gtld-servers.net.

;; QUESTION SECTION:
;com. IN NS

;; ANSWER SECTION:
com. 172800 IN NS f.gtld-servers.net.
com. 172800 IN NS e.gtld-servers.net.
com. 172800 IN NS j.gtld-servers.net.
com. 172800 IN NS k.gtld-servers.net.
com. 172800 IN NS c.gtld-servers.net.
com. 172800 IN NS i.gtld-servers.net.
com. 172800 IN NS m.gtld-servers.net.
com. 172800 IN NS l.gtld-servers.net.
com. 172800 IN NS h.gtld-servers.net.
com. 172800 IN NS a.gtld-servers.net.
com. 172800 IN NS b.gtld-servers.net.
com. 172800 IN NS g.gtld-servers.net.
com. 172800 IN NS d.gtld-servers.net.

PertFlavus

The spec was a good read. I removed it from the config above since forwarding to a DNS over TLS server would defeat the point, and I don't know if qname-minimisation is ignored automatically or not in this config. For now, DNS over TLS has to be explicitly enabled and does not work at all if the server's queried do not support it.

I also created an FR to add this as an option within the Advanced UI, so if anyone has anything to add… That's where I'd recommend doing it.

https://redmine.pfsense.org/issues/8028

edit: I wish i could edit these FR's for typos lol. How embarassing...

johnpoz

I added note to your FR, that I am now using the strict option as well.

If you or anyone else running the
server:
qname-minimisation: yes
qname-minimisation-strict: yes

Options in the custom option box find any domains your having a problem resolving - please post them so we can look resolving issue related to the settings or something, etc. I will post back after say a week or so if run into any problems.

PertFlavus

@johnpoz:

Options in the custom option box find any domains your having a problem resolving - please post them so we can look resolving issue related to the settings or something, etc. I will post back after say a week or so if run into any problems.

Well that didn't take long. go.microsoft.com fails to resolve with qname-minimisation-strict enabled.

johnpoz

really… Let me look.

Well its a shitty cname to cname nonsense... Not good practice..

go.microsoft.com. 2810 IN CNAME go.microsoft.com.edgekey.net.
go.microsoft.com.edgekey.net. 437 IN CNAME e11290.dspg.akamaiedge.net.
e11290.dspg.akamaiedge.net. 2 IN A 23.45.146.138

And if I had to guess, from my very limited understanding of the strict is the .com.edgekey.net is causing it issues.

Not like they don't warn you that strict could have issues with bad practice and naming conventions.

occamsrazor

@chrcoluk:

In some parts of the world (UK especially) isp's actually intercept and filter DNS queries (yes this would also catch queries using pfsense as the resolver as its outbound port 53 to query authoritative servers) so there is net value to carrying out DNS privacy. So I think in that case even using a 3rd party server would be worthwhile.

Not just UK… it happens in a number of developing countries I have lived in for various reasons. While a VPN is the best option it is not always ideal for all traffic and can cause certain issues. When I used to run Tomato firmware I used to use DNSCrypt with OpenDNS. At least this way I could be sure the DNS replies I was getting were not being intercepted and altered by my ISP. The solution to use unbound as a resolver isn't (I think) going to help against that at all.. is it?

I've read the various threads about DNSCrypt on here and don't really understand the reticence to implement it as an option. Most of the criticism seems to fall under "this isn't needed or useful against the threats in my use case scenario", because in most places ISPs are not actively intercepting…. but I believe it is useful in other use case scenarios. Personally I would like to see encryption of DNS traffic.

The desire (for me at least) is not so much about privacy but about the ability of the ISP to manipulate the DNS results and thus where I end up. I may not care that they know I'm visiting forum.pfsense.org, although I may care about some other sites in which case I'd use VPN, but I want to be sure that I'm not being sent instead to some fake site purporting to be forum.pfsense.org. I trust OpenDNS and pretty much any other major well-known western DNS provider a hell of a lot more than I trust my ISP to give clean results.

Disclaimer: I'm a newbie, be gentle :-)

johnpoz

To be honest, if on an isp that is intercepting your dns.. You should be using a vpn.. Because whatelse are they doing with your traffic if they are intercepting your dns?

If your in said country that is doing such things, really the only thing to do is vpn outside of that country before you send anything anywhere.. So that would be vpn solution.

Back to the strict setting.. Yeah seems doesn't like much of anything with MS that gets sent to the edgekey.net via cname… Couldn't get to blogs.technet.microsoft.com either I had to turn it off.

chrcoluk

@johnpoz:

So why don't you just run through a vpn and be done with it?

Here is a question for you.. Are they actually doing interception, or is their isp dns is just not returning the stuff they want you not to go to? Its a whole different ball game to just block specific dns in your dns that your running vs intercepting users dns, or blocking outbound on 53..

Performance reasons, I dont need to redirect all my traffic, just DNS queries.

You really dont seem to like DNS privacy. :)

Although I do route my iptv boxes through a VPN. I like pfsense's policy routing abilities.

I request that you please dont ask anymore why we are doing this, this thread was made by the OP to discuss a how to, not why we doing it. DNSCrypt discussions got derailed in the same way.

johnpoz

I like my dns privacy very much… Why I run my own resolver.. What I don't understand is people with their shiny hats thinking that forwarding traffic to some specific NS out on the public internet is some sort of privacy.. Just beecause they are inside a tunnel.. Passing off this sort of setup as "privacy" is BS plain and simple... Its not privacy.. It circumvention of someone interfering with your traffic, or listening in on it.. But sending all of your queries to 1 specific company does not in any way shape or form promote "privacy"

More than happy to stay out of your discussion on a public forum.. Here to promote discussion and debate on topics related to pfsense.. Have at it.. ;) As you can see trying to help with the part of this discussion that actually makes sense. Use the qname security, etc. Roots don't need my FQDN, they just need the part of the FQDN they are responsible for.. But seems companies like to break that with bad use of names, bad practice of cname to cname, etc.

Even thanked the poster a few times for the info... But seems with all your wanting of "privacy" you not up for discussion of actual point of doing such a thing.. Cuz you can state privacy all you want - its not that.. Sorry... Back to your tunneling your traffic to some specific name server and sending them every single query you want to resolve all in one place ;)

Velcro

I thought I would take a shot at this…I have attached a screenshot of my custom options in my DNS Resolver general settings. I am also running pfBlockerNG with DNSBL enabled, a pfB_DNSBL note was added when I enabled this.

I had to change the note a little("server:" had to have its own line), added "qname-minimisation: yes", restarted DNS resolver.

Does this enhance DNS privacy with just the "qname-minimisation: yes" or does one need to also add the "qname-minimisation-strict: yes"?
Does the order matter in terms of "include: /var/unbound/pfb_dnsbl.*conf" first then "qname-minimisation: yes" in this custom options?

I am using DNS resolver(unbound), with unbound "Outgoing Network Interfaces" having just my VPN provider(I deselected WAN).

I tend to be more of a "shiny hat" kinda person....not sure if it is just "go.microsoft.com.edgekey.net" sites that break but I think I can live with out these sites...does this break Linkedin? Skype as well?

Awesome conversation and post from the forum...thank you all!

![Screenshot-2017-11-4 pfSense localdomain - Services DNS Resolver General Settings.png](/public/imported_attachments/1/Screenshot-2017-11-4 pfSense localdomain - Services DNS Resolver General Settings.png)
![Screenshot-2017-11-4 pfSense localdomain - Services DNS Resolver General Settings.png_thumb](/public/imported_attachments/1/Screenshot-2017-11-4 pfSense localdomain - Services DNS Resolver General Settings.png_thumb)

tman222

@johnpoz:

"forwarding (to e.g. OpenDNS or Google instead of going to the root DNS servers)?"

Thought you said you read the RFC?? When you use the forwarder you do not talk to roots, you would have to send the forwarder the FULL thing your looking for, not just the pieces of the fqdn you need to find the authoritative server. So you could ask it for the record..

So in this scenario instead of asking roots hey whats the NS for .com in www.domain.com - its just asks hey whats the NS for .com
Then asks hey NS for .com whats the NS for domain.com, vs asking for NS of www.domain.com

How would that work with a forwarder?

You do not need to edit the conf file directly.. Just add it to the custom options box.

here… This simple.. see attached.

I ask the resolver hey www.testthisdomain.com and it just ask the NS for .com for testthisdomain.com vs the www.testthisdomain.com see attached sniff pic.

dig -x 192.31.80.30 +short
d.gtld-servers.net.

;; QUESTION SECTION:
;com. IN NS

;; ANSWER SECTION:
com. 172800 IN NS f.gtld-servers.net.
com. 172800 IN NS e.gtld-servers.net.
com. 172800 IN NS j.gtld-servers.net.
com. 172800 IN NS k.gtld-servers.net.
com. 172800 IN NS c.gtld-servers.net.
com. 172800 IN NS i.gtld-servers.net.
com. 172800 IN NS m.gtld-servers.net.
com. 172800 IN NS l.gtld-servers.net.
com. 172800 IN NS h.gtld-servers.net.
com. 172800 IN NS a.gtld-servers.net.
com. 172800 IN NS b.gtld-servers.net.
com. 172800 IN NS g.gtld-servers.net.
com. 172800 IN NS d.gtld-servers.net.

Thanks John, that makes a lot more sense to me now. I actually went ahead and tried this out today with qname-minimisation: yes enabled, and forwarding disabled and so far I haven't had any issues with DNS lookups. I won't enable strict mode just yet as I saw that you guys ran into some issues with it.

I do have more general question through regarding the Unbound resolver though: In the past I've had the resolver enabled with forwarding and used Google's public DNS servers as the upstream DNS servers to forward queries to in order to try to improve performance. Today I disabled forwarding to try out qname-minimisation for better privacy and to be honest I've haven't really found browsing to be any slower (despite the additional lookups that may be occurring). I'm curious if everyone else thinks that the added privacy is worth a bit of a performance hit in DNS lookup speed or whether one would be better of just using a the resolver together with forwarding to a fast public DNS (e.g. Google). What does everyone think?

Thanks in advance.

kejianshi

Hmmmm…

Assuming you are using a caching resolver like unbound and its getting its DNS with a bit of added encryption overhead for end users it would not be much of a performance hit at all because only the first query is via TLS. Future queries will pull from the cache as long as they remain valid. I'd want the root servers to be what I was connecting to though.

Now here is the issue I see with the encryption. The server load. Its going to be just crazy with TLS added. I mean, I have DNS over TLS right now since it all comes from my pfsense in the USA over vpn. This is done for many reasons, one of which is privacy.

It will be more meaningful if TLS is rolled into unbound and bind and all the goodness starts with the root servers. Technology advances may make this feasible and may have already. Not sure.. I'm 100% sure the feature was not included due to hardware load issues and bandwidth considerations as well as latency. Not because the people who designed bind and unbound don't know how to write the code.

johnpoz

"I've haven't really found browsing to be any slower (despite the additional lookups that may be occurring)."

Not sure where that little bit of FUD started that resolving is going to be slower.. In any sense that would matter.. Unless your on a some really high latency connection. The only time that there would be any sort of delay would be a cold lookup having to walk all the way down.

So you asking google for something.. What is that 30ms.. What does it take to fully resolve something? Lets call it 300ms worst case – lets just say.. But that is only on the COLD lookup.. Your talking less then 3 tenths of a second..

After that then your 1ms since the next lookups for www.domain.tld will come from cache.

Now keep in mind that was FULL cold lookup.. Had to talk to roots.. But you have looked up the NS for the tld in question, you no longer have to go ask roots every time. you look up newdomain.tld.. It will go straight their for the NS of newdomain.tld

Keep in mind that talking to the authoritative servers means you will always get the FULL TTL... not whatever google had left on its timer.. So Maybe you got 10 seconds left, and then have to do a query again in 10 seconds. While your resolver got the full 1 hour ttl, etc. So if you got two queries in 1 hour the forwarder would have to get asked twice while you resolver would have only had to do the 1 lookup.

Once something is cached it all becomes moot if your resolving or forwarding. Since its cached.. So why would you care about a few ms difference when looking up something cold.. Your telling me you can tell the difference in your page load be it it took 30 ms or 300ms.. Really?

Keep in mind that 300ms is most likely really high exaggeration... But you could always do some testing if you curious... Just clear your unbound cache and then do a simple dig.. How long does it take to resolve..

Unless your isp is only allowing you to talk to its ns, or your on really high latency internet - say sat or something. Resolving is going to be better all the way around, privacy, security and performance is a non issue to be honest.. Since once something has been looked up.. Your client caches it at the os, the browser caches it, your local dns caches it.. Your not cold resolving every single time all the way down from roots, etc.

Shoot resolving can even be faster.. Once you know the NS for domainX, looking up any records in domainX now just mean talking directly to that NS - which could even be closer to you in RTT or faster to respond than google.. Here just did a query direct to google for forum.pfsense.org took 64ms. While query direct to the NS for pfsense.org only took 52 ms.. And got back more data.. Why anyone would forward - especially if privacy is a concern.. And just giving someone all your info vs spreading it around to just the authoritative servers your wanting to go too and the roots.. That are global controlled setup.. You really think they are tracking billy's IP is going to sites xyz and selling that info?

edit:
Resolver also has a setting or prefetch.. Which can remove the full Colds or even the talking to the authoritative NS delay out... Since the resolver will resolve www.domain.tld if someone asks for it when there is 10% or less left on the TTL.. So lets say the TTL is 24 hours.. If someone asks for that record and there is less than 2.4 hours left on the TTL then the resolver will refresh that info by resolving it again and now the TTL is back to 24 hours. In the back ground, so next query comes by for that record. Its there waiting in the 1ms response cache. No need to even go ask google for it at 30ms, etc.

Velcro

I have tried using forwarding(to OpenDNS) and using unbound with no forwarding…I have found negligible difference in speed(didn't actually measure the speed, I just didn't notice while surfing), however after a reboot it can be very slow using unbound(John not sure that is what you mean by "Cold" lookup)...it then speeds up.

I did play with enable "qname-minimisation-strict" to see how it worked...I didn't do extensive testing but noticed it broke Skype while using the IOS app...

tman222

@johnpoz:

"I've haven't really found browsing to be any slower (despite the additional lookups that may be occurring)."

Not sure where that little bit of FUD started that resolving is going to be slower.. In any sense that would matter.. Unless your on a some really high latency connection. The only time that there would be any sort of delay would be a cold lookup having to walk all the way down.

So you asking google for something.. What is that 30ms.. What does it take to fully resolve something? Lets call it 300ms worst case – lets just say.. But that is only on the COLD lookup.. Your talking less then 3 tenths of a second..

After that then your 1ms since the next lookups for www.domain.tld will come from cache.

Now keep in mind that was FULL cold lookup.. Had to talk to roots.. But you have looked up the NS for the tld in question, you no longer have to go ask roots every time. you look up newdomain.tld.. It will go straight their for the NS of newdomain.tld

Keep in mind that talking to the authoritative servers means you will always get the FULL TTL... not whatever google had left on its timer.. So Maybe you got 10 seconds left, and then have to do a query again in 10 seconds. While your resolver got the full 1 hour ttl, etc. So if you got two queries in 1 hour the forwarder would have to get asked twice while you resolver would have only had to do the 1 lookup.

Once something is cached it all becomes moot if your resolving or forwarding. Since its cached.. So why would you care about a few ms difference when looking up something cold.. Your telling me you can tell the difference in your page load be it it took 30 ms or 300ms.. Really?

Keep in mind that 300ms is most likely really high exaggeration... But you could always do some testing if you curious... Just clear your unbound cache and then do a simple dig.. How long does it take to resolve..

Unless your isp is only allowing you to talk to its ns, or your on really high latency internet - say sat or something. Resolving is going to be better all the way around, privacy, security and performance is a non issue to be honest.. Since once something has been looked up.. Your client caches it at the os, the browser caches it, your local dns caches it.. Your not cold resolving every single time all the way down from roots, etc.

Shoot resolving can even be faster.. Once you know the NS for domainX, looking up any records in domainX now just mean talking directly to that NS - which could even be closer to you in RTT or faster to respond than google.. Here just did a query direct to google for forum.pfsense.org took 64ms. While query direct to the NS for pfsense.org only took 52 ms.. And got back more data.. Why anyone would forward - especially if privacy is a concern.. And just giving someone all your info vs spreading it around to just the authoritative servers your wanting to go too and the roots.. That are global controlled setup.. You really think they are tracking billy's IP is going to sites xyz and selling that info?

edit:
Resolver also has a setting or prefetch.. Which can remove the full Colds or even the talking to the authoritative NS delay out... Since the resolver will resolve www.domain.tld if someone asks for it when there is 10% or less left on the TTL.. So lets say the TTL is 24 hours.. If someone asks for that record and there is less than 2.4 hours left on the TTL then the resolver will refresh that info by resolving it again and now the TTL is back to 24 hours. In the back ground, so next query comes by for that record. Its there waiting in the 1ms response cache. No need to even go ask google for it at 30ms, etc.

Thanks John for taking the time to craft this very helpful response. After reading what you wrote and doing a bit more research on my own (looking through past threads on Unbound and performance) I understand the tradeoff's far better now. This thread was also very helpful:

https://forum.pfsense.org/index.php?topic=112160.0

I think a lot of the performance concerns can be traced back to the fact that a good number of domains today have pretty short TTL's (e.g. like www.cnn.com in the thread linked above). So on a small network (e.g. a home network), the TTL will expire after a few minutes necessitating another lookup the next time someone makes a DNS request for that same website. That lookup may/may not be faster than going against e.g. Google's DNS servers which likely already have the DNS record in question cached. I think that's probably where the main privacy/performance tradeoff comes in. In my opinion, adding up to a few tenths of a second of additional latency is worth it instead of sending all queries to one set of DNS servers on the internet. If speed is of upmost concern then using then using Unbound with forwarding enabled (to a fast DNS server with a large cache) is probably an option to consider.

Now it's possible to tweak the TTL values on Unbound to improve performance. Frankly, I wouldn't really want to touch the minimum TTL value (default is 0) since you run the risk of then having expired/bad records in the cache. However, what do you think about the max TTL value? I see that it is set to 86400 (1 day) by default. By running dig to a few sites on the net I saw that the expiry on records can be higher. Would it be bad idea to increase this value to two days (172800) instead to make sure that cache records don't expire prematurely?

–------------------

Anyway, I do not want to take this thread too far off-topic. I found some additional information regarding using Unbound over TLS:

https://calomel.org/unbound_dns.html

In general, I would agree with what kejianshi posted. Seems to me the additional overhead would add enough load on servers to slow down what was intended to be fast and lightweight protocol. That being said, hardware and connections speeds have continued to improve, so would be cool if this could be implemented at some point. For now, I think using the Unbound resolver with forwarding disabled (and the qname-minimisation option discussed in this thread enabled) is an easy to improve privacy by 1. limiting the amount of information that is shared during the recursive DNS lookup and 2) spreading the DNS request across multiple servers from multiple orgs vs. just one.

Thanks again to all for this great discussion.

PertFlavus

Unbound in particular makes for a pretty poor client for dns over tls at the moment. If you look at the implementation progress, it doesn't support reusing sessions or out of order queries. As a server these are no issue but as a client it will mean creating and tearing down a connection with each query. This matters because unbound is effectively the dns client in this example.

Freebsd seems really good at keeping up with unbound releases, so hopefully in the not too distant future this won't be the case.

tman222

I'm curious if anyone had an opinion on whether it might make sense to increase the max TTL value in the resolver from 1 to 2 days to potentially improve cache performance? Or is best to keep it at 1 day or lower? Thanks again.

PertFlavus

@tman222:

I'm curious if anyone had an opinion on whether it might make sense to increase the max TTL value in the resolver from 1 to 2 days to potentially improve cache performance? Or is best to keep it at 1 day or lower? Thanks again.

I expect not much to change with that. You should just enable prefetch support. When the 1 day period expires it'll check for an update to refresh the cache prior to you requesting it.

chrcoluk

guys what really helps in losing google dns caching hit rates, is the new unbound feature called serve-expired. It is now in the pfsense GUI as one of the advanced options, so you need only tick the box.

What it does is when a dns record is cached, even when TTL hits 0 (expired) a new lookup from the lan will serve the cached record but at the same time the cache is updated so the next lookup after is more up to date.

So you can enable the privacy stuff, use your own forwarder or do direct lookups from pfsense unbound, and this option mitigates most cold cache issues.

tman222

@chrcoluk:

guys what really helps in losing google dns caching hit rates, is the new unbound feature called serve-expired. It is now in the pfsense GUI as one of the advanced options, so you need only tick the box.

What it does is when a dns record is cached, even when TTL hits 0 (expired) a new lookup from the lan will serve the cached record but at the same time the cache is updated so the next lookup after is more up to date.

So you can enable the privacy stuff, use your own forwarder or do direct lookups from pfsense unbound, and this option mitigates most cold cache issues.

So I saw this option a couple weeks ago and started wondering if there would be any adverse impact to using it. I suppose there is the chance that the first lookup returns an incorrect result and an outdated page (or no page at all). A quick refresh/reload on the browser should fix that though (since the cache has refreshed in the background). Are there any security considerations one should be aware of when enabling this feature? I actually decided to try it (i.e. enable serve-expired) out and so far everything is working fine. I can see this option as potentially being beneficial on smaller networks with fewer clients.

This leads me to a more general question: Why do a lot of major sites have short TTL's these days? Is this being done for load balancing reasons?

Thanks in advance.