FYI: DNS over TLS works for Cloudflare like a charme

iqjet

Hi,

Cloudflare DNS over TLS works like a charme by enabling the GUI
For Quad9 you need to add in the GUI User defined Option:
forward-addr: 9.9.9.9@853
forward-addr: 149.112.112.112@853
See: https://www.netgate.com/blog/dns-over-tls-with-pfsense.html.

Here we can check if everything is OK.
https://developers.cloudflare.com/1.1.1.1/dns-over-tls/

kdig -d @1.1.1.1 +tls-ca +tls-host=cloudflare-dns.com example.com

Answer from my System:

kdig -d @1.1.1.1 +tls-ca +tls-host=cloudflare-dns.com  example.com
;; DEBUG: Querying for owner(example.com.), class(1), type(1), server(1.1.1.1), port(853), protocol(TCP)
;; DEBUG: TLS, imported 148 system certificates
;; DEBUG: TLS, received certificate hierarchy:
;; DEBUG:  #1, C=US,ST=CA,L=San Francisco,O=Cloudflare\, Inc.,CN=*.cloudflare-dns.com
;; DEBUG:      SHA-256 PIN: yioEpqeR4WtDwE9YxNVnCEkTxIjx6EEIwFSQW+lJsbc=
;; DEBUG:  #2, C=US,O=DigiCert Inc,CN=DigiCert ECC Secure Server CA
;; DEBUG:      SHA-256 PIN: PZXN3lRAy+8tBKk2Ox6F7jIlnzr2Yzmwqc3JnyfXoCw=
;; DEBUG: TLS, skipping certificate PIN check
;; DEBUG: TLS, The certificate is trusted. 
;; TLS session (TLS1.2)-(ECDHE-ECDSA-SECP256R1)-(AES-256-GCM)
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 12062
;; Flags: qr rd ra; QUERY: 1; ANSWER: 1; AUTHORITY: 0; ADDITIONAL: 1

;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 1536 B; ext-rcode: NOERROR
;; PADDING: 408 B

;; QUESTION SECTION:
;; example.com.        		IN	A

;; ANSWER SECTION:
example.com.        	1735	IN	A	93.184.216.34

;; Received 468 B
;; Time 2018-04-09 13:45:10 CEST
;; From 1.1.1.1@853(TCP) in 19.9 ms

For Quad9 after inserting the forwarding in the GUI

thomas@TDI-ThinkPad-W520:~$ kdig -d @9.9.9.9 +tls-ca +tls-host=dns.quad9.net  example.com
;; DEBUG: Querying for owner(example.com.), class(1), type(1), server(9.9.9.9), port(853), protocol(TCP)
;; DEBUG: TLS, imported 148 system certificates
;; DEBUG: TLS, received certificate hierarchy:
;; DEBUG:  #1, CN=dns.quad9.net
;; DEBUG:      SHA-256 PIN: ZMZ1T16d9Qc5uvRpUn/mu6fh4+IdoJGOEKjANut91Io=
;; DEBUG:  #2, C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
;; DEBUG:      SHA-256 PIN: YLh1dUR9y6Kja30RrAn7JKnbQG/uEtLMkBgFF2Fuihg=
;; DEBUG: TLS, skipping certificate PIN check
;; DEBUG: TLS, The certificate is trusted. 
;; TLS session (TLS1.2)-(ECDHE-RSA-SECP256R1)-(AES-256-GCM)
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 28177
;; Flags: qr rd ra; QUERY: 1; ANSWER: 1; AUTHORITY: 0; ADDITIONAL: 1

;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 4096 B; ext-rcode: NOERROR

;; QUESTION SECTION:
;; example.com.        		IN	A

;; ANSWER SECTION:
example.com.        	81016	IN	A	93.184.216.34

;; Received 56 B
;; Time 2018-04-09 13:50:03 CEST
;; From 9.9.9.9@853(TCP) in 19.1 ms

Without adding the forwads for Quad9 in the GUI:

Quad9 did not work with DNS over TLS. After I modified the GUI it statted to work
Once the forwards did their job, you can remove

$ kdig -d @9.9.9.9 +tls-ca +tls-host=dns.quad9.net  example.com
;; DEBUG: Querying for owner(example.com.), class(1), type(1), server(9.9.9.9), port(853), protocol(TCP)
;; DEBUG: TLS, imported 148 system certificates
;; DEBUG: TLS, received certificate hierarchy:
;; DEBUG:  #1, CN=dns.quad9.net
;; DEBUG:      SHA-256 PIN: ZMZ1T16d9Qc5uvRpUn/mu6fh4+IdoJGOEKjANut91Io=
;; DEBUG:  #2, C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
;; DEBUG:      SHA-256 PIN: YLh1dUR9y6Kja30RrAn7JKnbQG/uEtLMkBgFF2Fuihg=
;; DEBUG: TLS, skipping certificate PIN check
;; DEBUG: TLS, The certificate is trusted. 
;; TLS session (TLS1.2)-(ECDHE-RSA-SECP256R1)-(AES-256-GCM)
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 31083
;; Flags: qr rd ra; QUERY: 1; ANSWER: 1; AUTHORITY: 0; ADDITIONAL: 1

;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 4096 B; ext-rcode: NOERROR

;; QUESTION SECTION:
;; example.com.        		IN	A

;; ANSWER SECTION:
example.com.        	80741	IN	A	93.184.216.34

;; Received 56 B
;; Time 2018-04-09 13:54:38 CEST
;; From 9.9.9.9@53(TCP) in 19.1 ms

Is it posible to get a tool within Pfsense which will check if DNS over TLS is working?

behemyth

So for this to work with 2.4.4, do you have to enable both checkboxes for "DNS Query Forwarding?"

There are two boxes in there, one to Enable Forwarding Mode, and one for using SSL/TLS for quaries, which states that Enable Forwarding Mode must be enabled before quaries over SSL/TLS will be sent.

Just making sure this is set up right.

gsmornot

@behemyth:

So for this to work with 2.4.4, do you have to enable both checkboxes for "DNS Query Forwarding?"

There are two boxes in there, one to Enable Forwarding Mode, and one for using SSL/TLS for quaries, which states that Enable Forwarding Mode must be enabled before quaries over SSL/TLS will be sent.

Just making sure this is set up right.

If you want to use TLS you will need to check both boxes. The forwarding addresses go on the general page and need to support TLS.

pfcode

@gsmornot:

@behemyth:

So for this to work with 2.4.4, do you have to enable both checkboxes for "DNS Query Forwarding?"

There are two boxes in there, one to Enable Forwarding Mode, and one for using SSL/TLS for quaries, which states that Enable Forwarding Mode must be enabled before quaries over SSL/TLS will be sent.

Just making sure this is set up right.

If you want to use TLS you will need to check both boxes. The forwarding addresses go on the general page and need to support TLS.

Does this setting work with pfBlockerNG DNSBL functions?

gsmornot

@pfcode:

@gsmornot:

@behemyth:

So for this to work with 2.4.4, do you have to enable both checkboxes for "DNS Query Forwarding?"

There are two boxes in there, one to Enable Forwarding Mode, and one for using SSL/TLS for quaries, which states that Enable Forwarding Mode must be enabled before quaries over SSL/TLS will be sent.

Just making sure this is set up right.

If you want to use TLS you will need to check both boxes. The forwarding addresses go on the general page and need to support TLS.

Does this setting work with pfBlockerNG DNSBL functions?

Yes because it hits the resolver first before the forward.

dwasifar

Is it normal for this to be substantially slower than ordinary DNS?

A host query from a machine on the local subnet takes quite a lot longer when put through pfSense's DNS over TLS.

For instance, these are the kind of times I would ordinarily get:

dwasifar@Oberon ~ $ host -v simple.com 192.168.1.2   #Debian with dnsmasq
Trying "simple.com"
Using domain server:
Name: 192.168.1.2
Address: 192.168.1.2#53
Aliases: 

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 285
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;simple.com.			IN	A

;; ANSWER SECTION:
simple.com.		300	IN	A	151.101.0.205
simple.com.		300	IN	A	151.101.64.205
simple.com.		300	IN	A	151.101.128.205
simple.com.		300	IN	A	151.101.192.205

Received 92 bytes from 192.168.1.2#53 in 35 ms
Trying "simple.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55050
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;simple.com.			IN	AAAA

;; AUTHORITY SECTION:
simple.com.		900	IN	SOA	ns-616.awsdns-13.net. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400

Received 112 bytes from 192.168.1.2#53 in 40 ms
Trying "simple.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44224
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;simple.com.			IN	MX

;; ANSWER SECTION:
simple.com.		3600	IN	MX	10 aspmx.l.google.com.
simple.com.		3600	IN	MX	20 alt1.aspmx.l.google.com.
simple.com.		3600	IN	MX	30 alt2.aspmx.l.google.com.
simple.com.		3600	IN	MX	50 aspmx2.googlemail.com.
simple.com.		3600	IN	MX	50 aspmx3.googlemail.com.

Received 158 bytes from 192.168.1.2#53 in 34 ms

Versus this with DNS over TLS:

dwasifar@Oberon ~ $ host -v simple.com 192.168.1.3   #pfSense with DNS over TLS
Trying "simple.com"
Using domain server:
Name: 192.168.1.3
Address: 192.168.1.3#53
Aliases: 

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58942
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;simple.com.			IN	A

;; ANSWER SECTION:
simple.com.		300	IN	A	151.101.128.205
simple.com.		300	IN	A	151.101.192.205
simple.com.		300	IN	A	151.101.64.205
simple.com.		300	IN	A	151.101.0.205

Received 92 bytes from 192.168.1.3#53 in 380 ms
Trying "simple.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49702
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;simple.com.			IN	AAAA

;; AUTHORITY SECTION:
simple.com.		900	IN	SOA	ns-616.awsdns-13.net. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400

Received 109 bytes from 192.168.1.3#53 in 134 ms
Trying "simple.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65371
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;simple.com.			IN	MX

;; ANSWER SECTION:
simple.com.		3535	IN	MX	10 aspmx.l.google.com.
simple.com.		3535	IN	MX	20 alt1.aspmx.l.google.com.
simple.com.		3535	IN	MX	30 alt2.aspmx.l.google.com.
simple.com.		3535	IN	MX	50 aspmx2.googlemail.com.
simple.com.		3535	IN	MX	50 aspmx3.googlemail.com.

Received 158 bytes from 192.168.1.3#53 in 123 ms

Both of these servers resolve upstream to Cloudflare.

I like the idea of DNS over TLS, but not if it's five to ten times slower. Is this consistent with everyone else's experience, or should I be looking for a configuration error?

behemyth

I haven't seen the massive spike in time that you are seeing - that's weird. Are you using both the ipv6 ipv4 servers at cloudflare? You also have to keep in mind that the DNS entry is going to be cached, so if it is slower, then it'll only happen the first time.

iqjet

I don't see any remarkable delays. I use only cloudflare DNS. When using cloudflare and Quad9 together,
then I noticed delays.

Harvy66

Could this be due to not having a ready connection and the round trips required to establish one vs UDP DNS just being a request and a response? More of an issue for a cold cache. I also had a 34ms response time for the first call to resolve simple.com, the second was 0ms.

dwasifar

@Harvy66:

Could this be due to not having a ready connection and the round trips required to establish one vs UDP DNS just being a request and a response? More of an issue for a cold cache. I also had a 34ms response time for the first call to resolve simple.com, the second was 0ms.

No, it's definitely not a ready connection problem. The Debian server is going out through the same connection, and in fact its connection is THROUGH the pfSense box.

I'm aware that it caches. The times I showed were first query to that destination from each box.

Doing the query a second time comes back instantly. The Debian box caches too, and it also comes back instantly if the query is repeated. But the times on the initial pfSense query are just unacceptably slow.

Harvy66

@dwasifar:

@Harvy66:

Could this be due to not having a ready connection and the round trips required to establish one vs UDP DNS just being a request and a response? More of an issue for a cold cache. I also had a 34ms response time for the first call to resolve simple.com, the second was 0ms.

No, it's definitely not a ready connection problem. The Debian server is going out through the same connection, and in fact its connection is THROUGH the pfSense box.

I'm aware that it caches. The times I showed were first query to that destination from each box.

Doing the query a second time comes back instantly. The Debian box caches too, and it also comes back instantly if the query is repeated. But the times on the initial pfSense query are just unacceptably slow.

No, it's definitely not a ready connection problem. The Debian server is going out through the same connection, and in fact its connection is THROUGH the pfSense box.

What? I'm not sure what kind of connection you're talking about, but I mean a ready TCP connection that is already established. TCP connections have a 3 way handshake. With a 34ms RTT, that makes it 51ms just to create a connection. Another 68ms to establish a new TLS session, then you can make your request. You can shave off 34ms when resuming a TLS session.

That would bring TCP+TLS-resume down to 85ms. Then you make your query, which is an additional 34ms just for the round trip, or 119ms. This is very close to 123ms.

Again, my question is can you capture packets and see what's going on when you make a request?

yellowbrick

I tried the unbound TLS option as well. The current implementation does not appear to re-use TLS connections (you can check in the firewall states with each query). It adds a perceptible delay to fresh queries.

However, I have since moved to using a raspberry pi with pihole on the network. The pihole is configured to forward to a local clouflared / argo-tunnel dns proxy. The cloudflared in turn uses DNS over HTTPS (DoH), as opposed to DNS over TLS, to forward requests to 1.1.1.1. Also it definitely keeps the https/TCP connection around for a while, and the additional latency is much less perceptible.

Would be worth trying the unbound TLS option again at some point when it can be configured to re-use connections/set a keep-alive.

Harvy66

I guess that would make sense. The simplest implementation of TLS would be to just create a new connection per request. Connection pooling adds a fair bit of complexity. If using an HTTPS library, you get that for free.