[SOLVED] Public upstream DNS Resolvers for EXTERNAL FQDN: CloudFlare, Quad9, ...

Sergei_Shablovsky · Oct 8, 2021, 2:31 AM

What possible issue may be with using CloudFlare & Quad9 Public DNS Resolvers for EXTERNAL FQDN ?

Upside for CloudFlare (in most of our clients in EU and US area) are the ping are short:

20-35 ms for Quad9 9.9.9.9 / 9.9.9.10

1  ISP_A.net (AAA.BBB.CCC.DDD)  0.479 ms  0.452 ms  0.466 ms
2  ISP_A (AAA.BBB.CCC.DDD)  13.628 ms  14.099 ms  13.510 ms
3  ipv4.de-cix.fra.de.as42.pch.net (80.81.194.42)  39.066 ms  32.901 ms  32.790 ms
4  dns9.quad9.net (9.9.9.9)  35.117 ms  35.067 ms  35.040 ms

18-25 ms - for Google 8.8.8.8 / 8.8.4.4

1  ISP_A.net (AAA.BBB.CCC.DDD)  0.521 ms  0.843 ms  0.446 ms
2  74.125.32.161 (74.125.32.161)  10.646 ms  7.997 ms  7.897 ms
3  74.125.32.160 (74.125.32.160)  7.329 ms  7.222 ms  7.215 ms
4  108.170.248.138 (108.170.248.138)  7.489 ms  7.402 ms  7.996 ms
5  142.251.67.218 (142.251.67.218)  21.332 ms  21.362 ms  21.335 ms
6  74.125.242.225 (74.125.242.225)  21.900 ms  21.905 ms  21.732 ms
7  142.251.65.221 (142.251.65.221)  20.601 ms  20.631 ms  20.631 ms
8  dns.google (8.8.8.8)  20.589 ms  20.564 ms  20.589 ms

6,8-7,4 ms for CloudFlare 1.1.1.1 / 1.0.0.1

1  ISP_A.net (AAA.BBB.CCC.DDD)  0.544 ms  0.457 ms  0.474 ms
2  193.41.62.137 (193.41.62.137)  8.099 ms  7.890 ms  7.786 ms
3  193.41.62.138 (193.41.62.138)  7.751 ms  7.731 ms  7.721 ms
4  one.one.one.one (1.1.1.1)  7.149 ms  7.164 ms  7.150 ms

Downside for Quad9 are the ping are 5 x longest than CloudFlare, but according the public releases, Quad9 make so-called pre-filtering for "bad places" (so You not need to put this work on Your pfSense). (See this article A Deeper Dive Into Public DNS Resolver Quad9 if You need small explanation what is Quad9).

Gertjan · Oct 8, 2021, 8:02 AM

@sergei_shablovsky

If the "the need for speed" is the only criteria : just take the fastest.

Still, I'll add another ballast in the comparison :
Fast answers, ok, but do you need DNS answers you can trust ? The big public resolvers do all DNSSEC, but won't communicate their evaluation to you : you have to trust them.

And now for the new one, the one that everybody will understand right away (this was less likely a couple of days ago) : what about a distributed solution like : use one of the 13 global main DNS resolvers ? and then one of the thousand(s) TLD resolvers, and then the domain resolvers (2 or more) ? Use the build in - pfSense - resolver & don't forward.
This way, if some 'guy' manages to get in the wrong settings into the 'head' of your public DNS resolver, your entire connection (everything, not just facebook) won't brake

MoonKnight · Oct 8, 2021, 8:11 AM

@gertjan
Hi, So I should untick the DNS Query Forwarding.? I use Cloudflare DNS under the General Setup.

Gertjan · Oct 8, 2021, 8:39 AM

@ciscox said in Public DNS Resolvers: CloudFlare, Quad9, ...:

So I should untick the DNS Query Forwarding.? I use Cloudflare DNS under the General Setup.

No way !
You should use whatever you want to use. It's a "pro against cons" (or your you flip a coin) and you choose.
I want you to use what you understand. Nothing else matters.

The debate about resolving against forwarding is eternal.

pfSense, when installed, uses resolving. For two reasons :
It works out of the box. Always. Everywhere.
They, Netgate, can not decide for you that your DNS traffic has to go to some second party - a private company - to make DNS work.

Take note : these 'two reasons' are the ones I made up. The real reason(s) might be different.

Sergei_Shablovsky · Oct 10, 2021, 9:47 AM

@gertjan said in Public DNS Resolvers for EXTERNAL FQDN: CloudFlare, Quad9, ...:

@ciscox said in Public DNS Resolvers: CloudFlare, Quad9, ...:

So I should untick the DNS Query Forwarding.? I use Cloudflare DNS under the General Setup.

No way !
You should use whatever you want to use. It's a "pro against cons" (or your you flip a coin) and you choose.
I want you to use what you understand. Nothing else matters.

The debate about resolving against forwarding is eternal.

pfSense, when installed, uses resolving. For two reasons :
It works out of the box. Always. Everywhere.
They, Netgate, can not decide for you that your DNS traffic has to go to some second party - a private company - to make DNS work.

Take note : these 'two reasons' are the ones I made up. The real reason(s) might be different.

I mean EXTERNAL DNS service.

ler762 · Oct 11, 2021, 8:44 AM

@sergei_shablovsky

What possible issue ...

they have a list of every site your pfsense users visit.
Is that an issue for you?
do they do DNSSEC? in other words, can you trust their answers?
are you forwarding to a DoH or DoT server?
if no, your ISP and whoever else is in the path also gets to record every DNS name lookup pfsense forwards.. along with anyone in the path getting to play MITM

Sergei_Shablovsky · Oct 11, 2021, 8:44 AM

@ler762 said in Adding extra repo for easy install 3rd party tool (like smokeping, zsh, BpyTop, LibreNMS...):

@sergei_shablovsky

What possible issue ...

they have a list of every site your pfsense users visit.
Is that an issue for you?

EACH public DNS service doing this, because of nature.
Like ALMOST OF ALL VPN providers sing cross-country laws.
And all of them selling users data.
So, no any illusions about privacy.

The difference are that common well-known public DNS providers need to care about their reputation and

have a fast and stable physical network structure;
selling less data;

do they do DNSSEC? in other words, can you trust their answers?

All of them doing DNSSEC.
We have no choice: or You have a VERY fast DNS reply from well-known and reputable DNS service, or - pay for using own server (dedicated or shared, or VM) and have increasing delays, jitter and need to monitoring server and reliable alerting...

are you forwarding to a DoH or DoT server?
if no, your ISP and whoever else is in the path also gets to record every DNS name lookup pfsense forwards.. along with anyone in the path getting to play MITM

Yes. Using DoT/DoH are common standard nowadays.

Sergei_Shablovsky · Oct 11, 2021, 9:44 AM

UPDATE:

Recently I remind that CloudFlare have public DNSs servers for against malware content filtering and against malware and adult content, here the article in blog Set up 1.1.1.1 for Families.

So the hands-up for CloudFlare, because have the same filtering capabilities as Quad9, but 5x TIMES FASTER
(in reality approx. 3-5 times faster depend on Your country)

bwoodcock · Oct 12, 2021, 2:37 PM

@sergei_shablovsky

I wouldn't say Cloudflare has the same filtering capabilities as Quad9... Plenty of independent lab tests have shown that Quad9 is about 97% effective, while Cloudflare is about 55% effective. That's a significant difference. I would speculate that Cloudflare will never DNS block their own malware customers, which puts them at a significant structural disadvantage, and explains a significant portion of their blocking deficiency.

The latency is just a question of how efficiently your ISP is choosing routes to anycast instances, and can be fixed. Quad9 and Cloudflare have similar anycast footprints, albeit Quad9 is stronger in Africa and Cloudflare is stronger in Latin America. If one or the other is slower, that can be fixed by sending traceroutes to your ISP.

Besides, there's a growing body of work showing that anything under 200ms or so is undetectable to end users anyway, so differences of less than 30ms, that you're citing, aren't even worth the time to fix. There are bigger fish to fry.

johnpoz · Oct 12, 2021, 2:45 PM

@bwoodcock said in [SOLVED] Public DNS Resolvers for EXTERNAL FQDN: CloudFlare, Quad9, ...:

so differences of less than 30ms, that you're citing, aren't even worth the time to fix. There are bigger fish to fry.

If only more people would understand this concept ;)

bwoodcock · Oct 12, 2021, 2:48 PM

@johnpoz

Alec Muffett has been publishing really interesting work on this recently:

https://blog.apnic.net/2021/09/28/dohot-better-security-privacy-and-integrity-via-load-balanced-dns-over-https-over-tor/

Sergei_Shablovsky · Nov 19, 2021, 9:03 AM

@bwoodcock said in [SOLVED] Public DNS Resolvers for EXTERNAL FQDN: CloudFlare, Quad9, ...:

I wouldn't say Cloudflare has the same filtering capabilities as Quad9... Plenty of independent lab tests have shown that Quad9 is about 97% effective, while Cloudflare is about 55% effective. That's a significant difference. I would speculate that Cloudflare will never DNS block their own malware customers, which puts them at a significant structural disadvantage, and explains a significant portion of their blocking deficiency.

Thank You for information.

In this topic I try to point on DSN resolving time (because establishing TCP connection - the biggest time part of whole request-answer procedure for end user) much more that on malware filtering capabilities of common DNS services.

The latency is just a question of how efficiently your ISP is choosing routes to anycast instances, and can be fixed. Quad9 and Cloudflare have similar anycast footprints, albeit Quad9 is stronger in Africa and Cloudflare is stronger in Latin America. If one or the other is slower, that can be fixed by sending traceroutes to your ISP.

As SysAdmins most of us not able to push on ISP, so the choose of right quick external DNS is only one way to impact on whole time “request-answer” in establishing connection.

Besides, there's a growing body of work showing that anything under 200ms or so is undetectable to end users anyway, so differences of less than 30ms, that you're citing, aren't even worth the time to fix. There are bigger fish to fry.

Sum of small fractions give You a big numbers. ;)

Anyway, the end users who using TikTok/Instagram only, and end users in financial/health/gov organization - are very different end users.

And part of our obligations as SysAdmins are take as much as possible from existed equipment/applience. Isn’t ?

Sergei_Shablovsky · Nov 20, 2021, 3:24 PM

@bwoodcock said in [SOLVED] Public upstream DNS Resolvers for EXTERNAL FQDN: CloudFlare, Quad9, ...:

@johnpoz

Alec Muffett has been publishing really interesting work on this recently:

https://blog.apnic.net/2021/09/28/dohot-better-security-privacy-and-integrity-via-load-balanced-dns-over-https-over-tor/

Recently I have a time to read more about exactly this project ”load-balanced secured DNS over HTTPS over Tor”. Let’s to put my 5th cents here.

To understanding exactly the perspectives of an technology we need to understand what is motivation for the dev team or person behind this technology. So the author say

My work for the past few years has largely consisted of disrupting people's prejudices about Tor and its performance and usability, where that document says "Tor significantly increases the latency of DNS responses", I am coming from the perspective of "can we make it 'good enough for most people'?" In truth any extra "hop" is going to add latency to my DNS resolutions, and I am willing to trade a little latency for some extra privacy.

So, for Mr. Alec adding extra latency for end user is not a problem.
And I personally agree with him in case when this “technology hamburger” used personally by peoples who place the security on the first place (for example independent journalists, activists in non-democracy country like russia/Iraq/North Korea, independent security and investigation services) and the same time have no special software or hardware.

But this “technology hamburger” dramatically impact on whole ability to processing traffic if we try to using it in core Firewall like pfSense in most cases are.

And let’s to look on this “technology hamburger” from end user point of view: most of users need this level of anonymity right on a devices which they working every day and every hour, mostly notebooks, smartphones and iPads. And installing and configuring of this “technology hamburger” would be too much complicated for them.
Even end user configure all of this on a own home router,- this working only at home. But the end users of this profession work mostly “on the road”.

So I make conclusion, this technology interesting more as R&D, but no as for using in pfSense world.

Sergei_Shablovsky · Nov 22, 2021, 1:12 PM

I just live this here for anyone who interested in “measurements of internet speed”:

Fundamentals of Internet Measurement: A Tutorial
Nevil Brownlee, CAIDA (Cooperative Association for Internet Data Analysis)
Chris Loosley, CMG (Computer Measurement Group)

Many Internet users need to understand how to measure Internet traffic and performance. The primary focus of this tutorial is the global Internet, and ways of measuring, analyzing, and reporting the services provided to a user's network via the Internet. Some sections apply to measuring any network that uses the TCP/IP protocol suite, including a private network, or intranet.

First published in the
CMG Journal of Computer Resource Management, Issue 102, Spring 2001

johnpoz · Nov 22, 2021, 1:21 PM

@sergei_shablovsky none of which has anything to do with your dns taking 30ms to resolve or 60ms or even 200 ms..

Resolve how you want, point to what you want for your dns - but sorry a difference in a 20 or 30 ms, or again even 200ms for resolving of some fqdn isn't going to be even noticeable to a user..

Trying to find the fastest NS to use is pretty pointless endeavor. You are always going to see fluctuations in time to resolve something. Sure some public dns might resolve some fqdn in 10 ms when its 10ms away from you. But then again resolving something else it does not have cached might take 300ms..

Also ping or traceroute times to such NS is not always indicative to time to resolve..

Sergei_Shablovsky · Dec 2, 2021, 12:51 PM

@johnpoz said in [SOLVED] Public upstream DNS Resolvers for EXTERNAL FQDN: CloudFlare, Quad9, ...:

@sergei_shablovsky none of which has anything to do with your dns taking 30ms to resolve or 60ms or even 200 ms..

Resolve how you want, point to what you want for your dns - but sorry a difference in a 20 or 30 ms, or again even 200ms for resolving of some fqdn isn't going to be even noticeable to a user..

Trying to find the fastest NS to use is pretty pointless endeavor. You are always going to see fluctuations in time to resolve something. Sure some public dns might resolve some fqdn in 10 ms when its 10ms away from you. But then again resolving something else it does not have cached might take 300ms..

Also ping or traceroute times to such NS is not always indicative to time to resolve..

Thank You for reply! Sounds reasonably;)