Just to clarify the use of DNS over TLS (DOT)

johnpoz · Jun 16, 2023, 3:18 PM

@marchand-guy that is exactly what I said ;) you just called your colum fqdn dest, vs server name..

that is your packet 443 traffic to yoru dot server.. sniff your other traffic to say amazon.com

Clearly you know how to find the info.. Your just not looking at your other traffic. If your 443 traffic to dot server shows the fqdn, why would you think some other client hello to say amazon.com or google.com or netgate.com would not also show the fqdn in the clear..

Which was my whole point. Where you go is in the clear, until such time that encrypted sni becomes a thing across the internet..

BTW: that is a lot of session setups to your dot server, it should be reusing the same session.. opening that many ssl sessions sure and the hell not very efficient. You have 10 different sessions being setup in 15 seconds?

I know there was an issue with old version of unbound that it didn't reuse the session, but that was a few versions back when they changed that. from my understanding from unbound docs, I believe its suppose to be able to reuse the session for like 60 seconds.. Creating a new session for every query would be horrible performance..

Jun 16, 2023, 3:38 PM

Damn! Now I understand what you were saying all along.
Sorry for being dense.
DNS resolution (encrypted or not) is one thing.
Accessing the destination after that, is another. And unless you use a VPN, HTTPS will always show the name of the destination (SNI) in the clear.

And thank you for spotting the communication issues I seem to have.
I will look into it further, because yes, I have some slowdowns...

Thank you, again.

johnpoz · Jun 16, 2023, 3:49 PM

@marchand-guy said in Just to clarify the use of DNS over TLS (DOT):

HTTPS will always show the name of the destination (SNI) in the clear.

Exactly - until such time that whole of the internet adopts some form of encrypted sni, there was an attempt with esni, that crashed and burned.. now the latest is ech (encrypted client hello).. But for that to work, every site hosting https services would have to enable it ;) So not going to happen anytime soon that is for sure hehehe

edit: the problem with any form of encrypted sni, is the hoster - why should they bother to enable it.. Its more work, it will be more resources their boxes have to expend to serve traffic, just one other thing that could break and take their site offline, which if site is selling something, or even just serving ads could mean lost revenue..

I don't really see it taking off like a rocket ship that is for sure, if adopted - its going to be a slow grind.. Some of the major hosters like cloudflare or something sure could make it so its default or even click a button. Like they do with dnssec on your domains. But for it to reach any sort of major deployment going to be a slow crawl that is for sure..

Shoot how many sites have moved to tls 1.3? even ;) If you want a slow crawl for change just look at ipv6..

dnssec should be on every domain, its % of use is horrible... Changes like this are slow to take hold.. dnssec is another one of those why should I enable it - its just something that could fail, and prevent my users from getting to my site.. So its adoption has been lack luster that is for sure.

Jun 16, 2023, 3:55 PM

I like your way of thinking.

I need to rethink my communications stategy. If using DOT with quad9 is causing me delays, and my HTTPS communications are showing my destinations anyway, maybe I should try cloudflare to try isolating the DOT problems to quad9, or drop DOT altogether and get back to resolver without forwarding and ticking the maximum of options available to minimize the infos going out.

edit: I could try going full VPN from my router, but that's opening a can of worms...

RobbieTT · Jun 16, 2023, 5:16 PM

@johnpoz said in Just to clarify the use of DNS over TLS (DOT):

@marchand-guy said in Just to clarify the use of DNS over TLS (DOT):

Query Name Minimization Send minimum amount of QNAME/QTYPE information to upstream servers to enhance privacy

That sure isn't going to be optimal if your forwarding - that only makes sense if your resolving.. That if would even work would require multiple queries.. You should not have that set if forwarding.

The default setting in unbound is to apply qname-minimisation: yes.

The monsters only bite you if you select the option of qname-minimisation-strict: yes but the unbound default is set to : no.

There is also the option to set tls-use-sni: no. The default setting is a more reasonable : yes.

️

Jun 16, 2023, 5:27 PM

@RobbieTT said in Just to clarify the use of DNS over TLS (DOT):

set tls-use-sni

No such option on my installation (pfsense 2.6 CE) in resolver settings

johnpoz · Jun 16, 2023, 5:58 PM

@RobbieTT said in Just to clarify the use of DNS over TLS (DOT):

The default setting in unbound is to apply qname-minimisation: yes.

since when? I just looked at clean install of 2.6 and that is not enabled.. Why it could be default, it sure wouldn't make sense if you were forwarding.

That might be the default for unbound, but default settings on pfsense would not to do that.. Unless they changed something recently?

tls-use-sni: no.

That would have zero to do with clients creating a connection to say amazon.com, etc., That would have more to do when forwarding to some sort of specific dot server..

RobbieTT · Jun 16, 2023, 5:57 PM

@marchand-guy You can go pure CLI for unbound configuration but the pfSense GUI gives you a custom options box where you can add configuration settings.

️

RobbieTT · Jun 16, 2023, 6:07 PM

@johnpoz said in Just to clarify the use of DNS over TLS (DOT):

since when? I just looked at clean install of 2.6 and that is not enabled.. Why it could be default, it sure wouldn't make sense if you were forwarding.

That might be the default for unbound, but default settings on pfsense would not to do that.. Unless they changed something recently?

As far as I tell, pfSense picks up the default unbound settings, unless there is a variance to them - this is why the pfSense unbound config list is so small. If the config option is not explicitly set in pfSense then the unbound defaults rule.

 qname-minimisation: <yes or no>
              Send  minimum  amount  of information to upstream servers to en-
              hance privacy.  Only send minimum required labels of  the  QNAME
              and  set  QTYPE  to  A when possible. Best effort approach; full
              QNAME and original QTYPE will be sent when upstream replies with
              a  RCODE other than NOERROR, except when receiving NXDOMAIN from
              a DNSSEC signed zone. Default is yes.

As ever, I'm willing to be corrected.

️

johnpoz · Jun 16, 2023, 7:12 PM

@RobbieTT look at the setting in the gui - its not set, so your saying in the gui being not checked to enable it - would actually mean its doing it.. That would be a bug that should be corrected then, because when not set it just doesn't put in the setting if unbound defaults to yes then there would be no way to turn it off.

Now if there is no gui option for, ok then it would make sense to use the unbound default.. But there is a clear gui setting for qname - and it defaults to not being enabled. If unbound was actually doing it, then it would be BUG that should be fixed..

So looking at my 2.6 config, it doesn't have it.. And it defaults to yes, then it would be on.. But how would I turn it off? Since just clicking enabled, and then disabled just removes the setting..

login-to-view

There is no setting in /var/unbound/unbound.conf for qname.. I have it enabled on my 23.05 and it places this in the conf

qname-minimisation: yes

If unbound now defaults that to yes.. Then yeah there needs to be a redmine put in - because there would be no way to disable from the gui... I just enabled and see the yes in the conf.. Then unchecked it and the whole qqname-minimisation line is gone. So if it is now defaulting to qname -- that is problem..

I will validate that default is now yes in unbound on version that is on 2.6, which is Version 1.13.2, then should prob look on 23.05 and 2.7 snap shots.. if unbound is now defaulting to yes on qname then the logic in the gui needs to be adjusted so you can actually turn it off..

 qname-minimisation: <yes or no>
              Send  minimum  amount  of information to upstream servers to en-
              hance privacy.  Only send minimum required labels of  the  QNAME
              and  set  QTYPE  to  A when possible. Best effort approach; full
              QNAME and original QTYPE will be sent when upstream replies with
              a  RCODE other than NOERROR, except when receiving NXDOMAIN from
              a DNSSEC signed zone. Default is yes.

Yeah its doing it by default.. If I put in in the options box then it doesn't do it..

server:
qname-minimisation: no

redmine created
https://redmine.pfsense.org/issues/14479

edit: ok just tested on 23.05, yeah its doing it even when you uncheck to do it in the gui, it just remove the line to do it in the conf.. which it then does it by default..

RobbieTT · Jun 16, 2023, 7:53 PM

@johnpoz Thanks, I hadn't noticed that at all; it does seem a bit odd.

Other settings via the Resolver GUI do populate faithfully across to the /var/unbound/unbound.conf file.

Learning as we go. No doubt there is more to be uncovered.

️

RobbieTT · Jun 16, 2023, 8:08 PM

@marchand-guy said in Just to clarify the use of DNS over TLS (DOT):

Accessing the destination after that, is another. And unless you use a VPN, HTTPS will always show the name of the destination (SNI) in the clear.

Things are improving with HTTP/3 as there is very little outside of the encryption, so the actual name has gone from clear text. Of course, you still need an address to route to and that will always be there. In IPv6 land the source and destination probably has a privacy address in place but the prefix may still paint a picture:

login-to-view

️

johnpoz · Jun 16, 2023, 8:32 PM

@RobbieTT said in Just to clarify the use of DNS over TLS (DOT):

Other settings via the Resolver GUI do populate faithfully across to the /var/unbound/unbound.conf file.

To be honest this qname thing is kind of a issue to be sure, hey if your resolving its not really a change in the number of queries.. But if your forwarding and you do qname min its going to cost many queries when there should only be 1..

If I want to look up say host.domain.tld.. and I want to ask quad9 for that - asking it just for .tld or just for domain.tld are wasted queries that only take time to not work, and then finally ask for host.domain.tld

qname min should never be enabled if your forwarding..

And if there is some other thing going on and your creating multiple tls sessions vs just 1 and using it for multiple queries.. It can add up to poor performance..

edit:
To your http3 or quic point.. While the first connection might be encrypted - pretty sure those keys are visible, so you can decode and still get the sni if you wanted to..

RobbieTT · Jun 16, 2023, 8:34 PM

@johnpoz
I use forwarding and it only sends a single* query with the default setting.

️

*Well, there is an unrelated problem with unbound that may cause 2 sequential queries to be forwarded...

johnpoz · Jun 16, 2023, 8:42 PM

@RobbieTT said in Just to clarify the use of DNS over TLS (DOT):

I use forwarding and it only sends a single* query with the default setting.

So your saying in forwarding mode - it disables that qname-min is default to yes? I don't forward so would have to test that.. But yeah that would for sure make sense not to enable qname-min when forwarding.

As to you quic thing though - yeah its still there in the clear.. If I recall they exchange keys when they first talk, but those keys are in the open so anyone that wants to decode can, and wireshark does this on its own, etc...

login-to-view

The qname thing is still a problem though - because there is no "gui" way to disable it, and user just looking at the settings would think its off, when its not.. The only way to turn it off when resolving is to use the custom option box and actually set it to no.

Jun 16, 2023, 9:43 PM

@johnpoz said in Just to clarify the use of DNS over TLS (DOT):

The only way to turn it off when resolving

Don't you mean the only way to turn it off when forwarding?
Won't you turn it on when resolving?

johnpoz · Jun 16, 2023, 11:03 PM

@marchand-guy said in Just to clarify the use of DNS over TLS (DOT):

Won't you turn it on when resolving?

maybe people don't want to do it.. It can cause some issues with cnames, especially if strict.. and now allow for fall back, etc.

When resolving I would have it on sure, but you should be able to disable it if you wanted it too.

As to forwarding seems like you can't enable it? Which would make sense I can not see a reason why anyone that forwards would ever want to use qname, its makes zero sense to do that if forwarding - so maybe when you enable forwardering qname because disabled completely?

I don't forward so would have to do a specific test for it to find out for sure.. But RobbieTT mentioned it doesn't do it, not sure what other issue he is talking about where 2 sequential queries?

RobbieTT · Jun 17, 2023, 2:41 PM

@johnpoz said in Just to clarify the use of DNS over TLS (DOT):

...not sure what other issue he is talking about where 2 sequential queries?

As said, it is an unrelated issue but at the moment when you have a mix of IPv4 & IPv6 servers in your forwarding list unbound treats the differing address types as 2 distinct query requests and (as an additional irritant) it completes both tasks in sequentially before providing any answer to the client.

Clearly the name servers are equally capable of delivering IPv4 & IPv6 addresses but using both together provides clients different pathways, which can be advantageous. Whilst unbound has consistently fought against a Dnsmasq-like all-servers option to avoid the additional loading (harumph!), they have ended-up sending double the queries, despite professing the need for a single query only. Add in the 400ms rules and the 900 (90% fast, 10% slow) rules and query times can become rather odd. Again, a bit off topic here.

Regarding HTTP/3 and encryption of the SNI - encrypted means just that, it is no longer sent in the clear. Of course, as a sender/observer or the target address it remains eminently visible on Wireshark.

It is also correct to say that the key exchange can be captured by an external entity (say for censorship) and worked on by software - but it is a difficult task to do at scale. Russian and Chinese censors have taken to blocking HTTP/3. A simple parry but I guess it proves some worth in HTTP/3.

Source/destination address, as I mentioned earlier, remains a vulnerability but it is not always as easy as people make out. A static IPv4 address to a static IPv4 tied to a single server is easy meat and is often the example people highlight. In reality things are often not that easy at all.

For example, many differing services routinely operate behind either a single or a brace of IPs. CDNs, reverse proxies or shared platforms complicate matters. Add in IPv6, privacy extensions and the general complications of BGP et al we quickly get to a more complicated point as to where the modern internet sits before we even contemplate node distribution, dark fibre or VPNs.

As ever, you cannot point at a single aspect of protocols, privacy and security and claim either their robustness or their fallibility. They all add layers and those layers, taken together, do add considerable value. I use what I can, when I can, whilst reaping the benefits of others paying little or no attention to such matters. Fodder for the cannon.

️

johnpoz · Jun 17, 2023, 2:41 PM

@RobbieTT said in Just to clarify the use of DNS over TLS (DOT):

but it is a difficult task to do at scale

Says who?? And depends on what your wanting to do with it. While I agree might be a bit harder to scale if what your looking to do is filter/censor..

But what if that is not what after, and what after is just a list of where they are going so can sell this info, etc. Ie what the dot and doh champions have been saying your isp is doing..

I don't need to in real time decode and then make a decision of if you can go there or not.. All that needs to be done is log the traffic and decode it (since the keys are in the clear) as some later time and provide a db that user xyz (ip address) when here and here and here at these times..

Just saying - don't let smoke and mirrors about can not be scaled think your hiding anything..

JonathanLee · Jul 6, 2023, 6:47 AM

Can unbound be set to resolve DoH ? The DoH packets should be the same except what the server it's handing that request right?

It's a DoH packet, leading to can Squid Proxy handle them and know what to look for, so it could just auto send it to unbound resolver when it sees a DoH request hit?

Right now I block a massive DoH lost.

Side thoughts: I think QBIC/HTTPS3 does the DoH over also just over UDP. That would be some epic code to write to make proxies work better.

Edited:

Unbound can resolve Dot and DoH per custom options

https://unbound.docs.nlnetlabs.nl/en/latest/topics/privacy/dns-over-https.html