HEADS UP: Be aware of Trusted Recursive Resolver (TRR) in Firefox

jimp

Firefox has recently added a feature they call Trusted Recursive Resolver (TRR). It uses DNS over HTTPS to resolve DNS queries directly in the browser rather than using the native OS resolver. Though they have not currently enabled it by default, they would like to do so in the near future.

While this may offer some additional privacy benefits in certain situations, with regard to pfSense it means that your DNS policies like DNSSEC, host and domain overrides, caching, pfBlocker DNSBL, and so on will be ignored by clients using TRR.

Is this a good thing? Well, that depends. For some, sure. For others, no. There are technical reasons not to want this behavior, as mentioned above and in more detail below. There are also privacy benefits as well for many users. Mozilla has partnered with Cloudflare so that means TRR DNS queries are sent there and not to the intended server. Some people already use Cloudflare, or they don't care where the queries go, so that's a wash or a net gain. If you do not trust Cloudflare or do not want to put all your eggs in the Cloudflare basket, that's not so good.

In particular this could potentially interfere with things like Captive Portal, but TRR does have a built-in fallback if TRR fails to resolve a host. It may add some additional time to Captive Portal initial requests but how much remains to be seen. I haven't tested this yet, but I don't expect it to be a significant factor.

Another potential source of trouble is mixing this with a proxy, depending on whether or not you splice/bump/MITM SSL. I don't have a setup handy to try this, but it could easily go either way. May be fine, may break. Even if the DNS over HTTPS queries work through the proxy, the fact that it's using a different DNS server than the proxy could cause failures.

So what can be done about it? That remains to be seen as well. This is a user preference in Firefox but it not currently exposed as a user option outside of about:config. From about:config a user can set network.trr.mode to 5 to completely disable TRR. This may also be possible on larger managed deployments on a wider scale, but that depends on the OS and network infrastructure.

For BYOD/Guest networks there isn't much that can be done by a network admin. Mozilla is not using 1.1.1.1 for this, it is using https://cloudflare-dns.com/dns-query and the DNS query for that must be bootstrapped using the native OS resolver. So if queries for that domain fail it may cause TRR to fail and fall back to your configured DNS server. This could potentially be accomplished with DNS host overrides.

For those interested in testing the behavior, in Firefox, open about:config and set network.trr.mode to 2 which will prefer TRR but fall back to regular DNS. The current values are:

0: Off by default
1: Firefox will choose based on which is faster
2: TRR preferred, fall back to DNS on failure
3: TRR only, no DNS fallback
5: TRR completely disabled

Canary Domain

You can use the canary domain to nudge clients so they do not use DoH when they are attempting to do so automatically.

In the DNS Resolver advanced settings, add the following configuration:

server:
local-zone: "use-application-dns.net" always_nxdomain

johnpoz

Yeah I am not a fan of this at all.. If they want to allow the users to enable this - sure ok.. But it should never be on out of the box without user interaction to enable it if you ask me.

I already have network.trr.mode set to 5... I see no use of this at all

chpalmer

@johnpoz said in HEADS UP: Be aware of Trusted Recursive Resolver (TRR) in Firefox:

network.trr.mode

Default seems to be 0 for everyone I know.. Is 5 better?

jimp

0 may turn into 2 based on what Mozilla wants to do, since 0=default, 5=explicitly disabled.

chpalmer

Thanks 5 it is!

yon 0

@chpalmer said in HEADS UP: Be aware of Trusted Recursive Resolver (TRR) in Firefox:

默认似乎都是0 .. 5更好吗？

i am using u simpledns server :)

my ISP blocked the Cloudflare , so i want to have to change to close

johnpoz

Huh?? yon your post is gibberish.. what server you want to use has zero to do with what TRR is..

yon 0

@johnpoz
I mean, I can only use my own DNS or VPN.If a website is blocked, TRR may not solve this problem.

johnpoz

TRR is more problems than anything ti could possible solve.. Especially if they turn it on without explicit users acknowledgment.. Problem is even the user agrees to some pop up, vast majority of them not even going to understand what they are agree too.. Typical users - and then wondering why their local resolving of xyz.com broke.

This should require users actually having to do something to enable it, like edit about:config entry and on purpose turn it on.

And you sure and hell could use cloudflare through your vpn... So you still could use trr if you wanted to, even if your isp blocks where its going. I still don't why anyone would want to use this.. Sorry I don't want to send all my dns queries to 1 provider.. I don't want to use you for dns - I will do it myself thank you very much.

johnpoz

Got question about ios version of firefox.. How can you ensure this is never used? about:config is not available in ios version of firefox.

jimp

That's a tough one to answer.

I did see a post on Reddit earlier this week from someone who claimed association with Mozilla that said they no longer plan on ever making this the default, only available as a GUI option. I'm not holding my breath waiting on that to be verified, but it is at least a bit of hope that we won't have to jump through hoops.

johnpoz

Thanks for the info - lets hope they don't try and enable this on the sly in the background ;) Doing such a thing for sure would force me to rethink my browser choice..

bcruze

this is why i use firefox ESR.

the version that would get this automagically would be the standard version of FF

johnpoz

So it looks like mozilla is on course for enabling this by default... UGGHHH!
https://blog.mozilla.org/futurereleases/2019/09/06/whats-next-in-making-dns-over-https-the-default/

And upon checking seems somehow somewhere in some past update my setting of network.trr.mode to 5 got reset to 0..

I am really curious once they start turning this on, if turning it off in the options will be enough.. I do not want this, stay away from my dns.. They should use what the OS uses, if you want to offer doh or dot - great, it should be OPT IN ONLY!!! Turning this on be default is BS plain and simple..

Seems they are using the canary domain use-application-dns.net so if your dns resolves NX their doh should not be used.

you can set that in unbound like this
local-zone: "use-application-dns.net" static

Which I have already done on my network.

I believe you can also set
security.enterprise_roots.enabled

To true, to stop it...

I will be keeping an eye on this.. Stay away from my dns mozilla...

Looks like pihole already has commit to disable it via the canary.. I would hope pfblocker will be doing something similar
https://github.com/pi-hole/pi-hole/pull/2915

I think I will reach out to bbcan177 and ask..

chpalmer

@johnpoz said in HEADS UP: Be aware of Trusted Recursive Resolver (TRR) in Firefox:

And upon checking seems somehow somewhere in some past update my setting of network.trr.mode to 5 got reset to 0..

Just checked and so did mine. Back to Waterfox for me..

kiokoman

good, time to uninstall and change to another browser

Pippin

I'm on latest FF (Linux Mint 18.3), my setting 5 did not get changed.

tman222

I have a few Linux systems that use the ESR version of Firefox (60.x). On those systems the default setting is 0 and setting it to 5 appears to break DNS altogether. So I'm guessing 5 is not a supported option on the ESR version. Checking out a more recent version of Firefox on a couple Mac's did allow me to set the option to 5 and everything still works.

Some more info about disabling DoH from Mozilla:

https://support.mozilla.org/en-US/kb/configuring-networks-disable-dns-over-https

Glad that Pi-hole is already had a PR on this.

@johnpoz - can you explain what "local-zone: "use-application-dns.net" static" does exactly and how in NXDOMAIN being returned? Thanks in advance.

johnpoz

So per their disable article you linked to, if that domain returns a NX vs actual IP then firefox is not suppose to use doh..

So if you set that in your options box.
server:
local-zone: "use-application-dns.net" static

When a query is done for that - it will return NX.

$ dig @192.168.9.253 use-application-dns.net

; <<>> DiG 9.14.4 <<>> @192.168.9.253 use-application-dns.net
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 23041
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;use-application-dns.net.       IN      A

;; Query time: 0 msec
;; SERVER: 192.168.9.253#53(192.168.9.253)
;; WHEN: Sun Sep 08 14:22:35 Central Daylight Time 2019
;; MSG SIZE  rcvd: 52

Notice the
status: NXDOMAIN

I have sent a PM to bbcan177, but haven't heard anything back yet... But yeah would be nice if pfsense works out something - say if pfblocker is being used, unbound will auto return the NX for that, or something..

Or maybe pfsense could put in a check box to have that return NX..

I really don't now what Mozilla is thinking here.. That is great you want to offer this for your clients - but it really should be OPT IN ONLY!!! Run a PR campaign to get your users to set it.. but making it default is WRONG..

I have set the network.trr to 5 again, will keep an eye on that, and have set
security.enterprise_roots.enabled
To true as well.. so will be watching that as well, when the next update comes out.. Running 69.0 currently of firefox.

tman222

@johnpoz said in HEADS UP: Be aware of Trusted Recursive Resolver (TRR) in Firefox:

local-zone: "use-application-dns.net" static

Thanks @johnpoz - I appreciate the follow up an explanation. I was actually more curious what that configuration line did exactly and why you chose it specifically to solve this issue?

Is the idea just to create a local DNS Zone with this domain, but since no actual (local) records exist for it, it will simple return NXDOMAIN?

Thanks again!

johnpoz

Exactly, when you set the local zone as static, and no local entries - NX gets returned. So when NX gets returned when the browser looks for that, it is suppose to not enable DOH because local filtering is in place and the user has opted to use that vs doh.

edit: I can foresee future posts about why pfblocker is not working because browser is using doh vs asking the local dns which is using pfblocker.

tman222

Thanks @johnpoz - when entering into Unbound's option box, for multiple configuration options, is the correct syntax:

server:
config line 1
config line 2

OR

server:
config line 1

server:
config line 2

Apologies, I can never quite remember. Thanks again for your help.

kiokoman

bind9 + rpz
cloudflare-dns.com IN CNAME .

johnpoz

You only need 1 server: at the top

For example here are all my entries.
server:
private-domain: "plex.direct"
local-zone: "use-application-dns.net" static
so-reuseport: no
#log-queries: yes
#log-replies: yes
#private-address: ::/0 # filters out all AAAA !

I leave the log entries in there and the private address thing to stop AAAA because I enable them on the fly sometimes for testing.. I just leave them # out normally when I don't need them. I think I can remove that so-reuseport: no since I think that was changed in an update a while back.. Not sure on that one. But yeah you only need the one server: entry.

pfSenseTest

@johnpoz said in HEADS UP: Be aware of Trusted Recursive Resolver (TRR) in Firefox:

I think I can remove that so-reuseport: no since I think that was changed in an update a while back.. Not sure on that one.

I was wondering about this myself and haven't been able to confirm if it is still needed or not.

johnpoz

@johnpoz said in HEADS UP: Be aware of Trusted Recursive Resolver (TRR) in Firefox:

so-reuseport: no

I just commented it out and restarted unbound, and there are still multiple threads.. So its no longer needed..
https://forum.netgate.com/post/809158

JimP goes over in that post

JeGr

The whole use case Mozilla and the DoH folks that promote it that much are telling is falling apart IMHO. If a simple NXDOMAIN for the use-application-dns.net Zone is "blocking" the use of DoH in Firefox alltogether, how in the hell is the DoH implementation in Firefox well-thought in the first place? They are pointing fingers at MITM or Rogue ISPs etc. to protect your data and privacy against. OK, fair point, but how in the hell is that protecting if you

completely screw up a fine configured local DNS resolver setup with internal domains by using a DoH server from an external source (and handing your DNS data to them!?)
can override your privacy-DoH-firefox-thingy by said rogue ISP to simply hand out a nxdomain for the use-application-dns.net zone?
begin to segregate DNS into "per application use"? Firefox using DoH with Cloudflare, System using local resolver with internal domains, next Browser/Application uses whatever the f*** they want? How is that transparent and protecting anything about your privacy if you don't have a clue anymore what your system does? It's something like the whole Linux-systemd-everything. NTP? DNS resolving? Yay let's put it into systemd, too! What!?

Narf

kiokoman

simple. privacy is a businness after all. it will not be strange if that work was sponsorized by cloudflare and the like. they are only selling all your data to them and telling you that this way you are more protected
i hate when company make fun of customers. if they left it as an option instead of imposing it as a default it would have been something else

johnpoz

Now it seems google/chrome wants to join in on the data mining fun.. WTF!!!
https://www.zdnet.com/article/google-to-run-dns-over-https-doh-experiment-in-chrome/

dot is easy to block since uses its own port... But guess its time to start compiling list of all known doh servers and blocking them.. Gawd Damn it!!!

JeGr

@johnpoz said in HEADS UP: Be aware of Trusted Recursive Resolver (TRR) in Firefox:

But guess its time to start compiling list of all known doh servers and blocking them.. Gawd Damn it!!!

They are also contradicting their own argumentation. Mozilla for example arguments over governmental or rogue ISPs grabbing your data. At the same time they 1) send all your data to a single provider instead of using resolvers to dsitribute the DNS calls to a wide range of servers and 2) use use-application-dns.net as their canary to determine wether or not to disable automagic DoH. So we roll out some application specific black magic BS that breaks your local setup but hand out the free pass for their own attackers to disable it. WTF? How is that thought-out? First thing a rogue ISP or anyone else would be to hand out NXDOMAIN to use-application-dns.net calls to disable that. And if anyone does that... yeah.

Pushing OS developers to include DoT and DoH in their base OS and as next step push all DNS servers to wide-range adaption of both protocols would be the right way. Then you could just tell your local resolver to use either DoH/DoT for resolving (and do or don't fallback to normal udp/53 DNS). Their tactic? No just send it all to Google/Cloudflare etc. Right...

kiokoman

the Internet Services Providers Association (ISPAUK), a trade association for internet service providers in the UK, decided to nominate Mozilla for its award of 2019 Internet Villain, next to Donald Trump and the EU's Article 13 Copyright Directive.

https://www.zdnet.com/article/mozilla-no-plans-to-enable-dns-over-https-by-default-in-the-uk/

https://www.ispreview.co.uk/index.php/2019/07/ispa-pulls-uk-internet-villain-category-over-mozilla-doh-fallout.html

ISPAUK have a point here

An application switching to DoH should ensure that this switch does not undermine choices that have been previously made by the user. For example, if parents have decided to filter an internet connection in their home via network or local level DNS controls, these choices should not simply be ignored by the application.

If DoH doesn’t work or is slow, a customer’s internet access will be affected. The customer will contact their ISP, not the DoH provider, but the ISP won’t be able to fix things for them. As a minimum, any application switching to DoH should ensure that the selected resolver should provide a 24/7 user call centre reachable via low-cost/local rate telephony and an online support capability. Support for fault-diagnosis and resolution between ISP, resolver and users should also be provided.

johnpoz

@kiokoman said in HEADS UP: Be aware of Trusted Recursive Resolver (TRR) in Firefox:

An application switching to DoH should ensure that this switch does not undermine choices that have been previously made by the user

All of which just SCREAMS this can not be opt-out, but MUST be OPT-IN to switch...Where the users has to specifically do something to actually use it, vs just using the OS already set dns..

edit:
I think the thought process has gone something like this.
Q: How can we mine more data from our users without logging and sending the info where they go.. "telemetry" they don't seem to like that.

A: Lets have them use us for dns, so that we know everywhere they they go.

Q: How can we get them to do that?
A: We will tell them its for their own security, from the bad isp they are paying every month for internet.

Q: Well they are not switching over fast enough, and we don't want to actually spend any money on PR to have them think this is best option for their "security/privacy"

A: Lets just switch them over, and state that is for their own good if they complain, because they are just too stupid anyway.

bcruze

https://apple.news/AREysIdXnROWCIANTguz06A

Not sure if the link will work, they are trying a personal vpn now..

johnpoz

They can offer whatever they want - just don't freaking make something like that default... If the user has to turn it on, and make an effort to use it..

johnpoz

So I found this listing of known doh servers
https://github.com/curl/curl/wiki/DNS-over-HTTPS#publicly-available-servers

I have a pm out of @BBcan177 asking if he plans on easy way of blocking these in pfblocker, if not will be working my own listing of them to use in firewall rules and host overrides to block queries for them.

Will prob have dns queries for their fqdn point to a specific IP that I block and log, and any hits to that rule get attention and any devices doing it will if not easy to disable be off the network..

JeGr

I'd also use

local-zone: "use-application-dns.net" static

in the DNS Resolver "custom options" box so to answer with an NXDOMAIN to the canary domain Mozilla uses. Won't help if they switch their approach or simply use a service manually entered but at least the "automagic switch" should be off then.

johnpoz

Yeah been in place for some time already ;)

Notice I posted how to do it like 8 days ago ;) Look back over the thread..

BBcan177

@johnpoz said in HEADS UP: Be aware of Trusted Recursive Resolver (TRR) in Firefox:

I have a pm out of @BBcan177 asking if he plans on easy way of blocking these in pfblocker, if not will be working my own listing of them to use in firewall rules and host overrides to block queries for them.

Some more details here for now until I add this officially to pfBlockerNG:

https://www.reddit.com/r/pfBlockerNG/comments/d3p1gf/doh_server_blocklist/

johnpoz

Thanks - but you the have the
use-application-dns.net

In there - that would not return an NX, so it wouldn't work per my understanding of how the firefox canary is suppose to work.. It has to get back a NX to not enable it from my understanding.

I threw this together real quick this morning from list linked to earlier.

local-data: "dns.adguard.com. 120 IN A 172.19.19.19"
local-data: "dns-family.adguard.com. 120 IN A 172.19.19.19"
local-data: "dns.google. 120 IN A 172.19.19.19"
local-data: "cloudflare-dns.com. 120 IN A 172.19.19.19"
local-data: "dns.quad9.net. 120 IN A 172.19.19.19"
local-data: "dns9.quad9.net. 120 IN A 172.19.19.19"
local-data: "dns10.quad9.net. 120 IN A 172.19.19.19"
local-data: "doh.opendns.com. 120 IN A 172.19.19.19"
local-data: "doh.cleanbrowsing.org. 120 IN A 172.19.19.19"
local-data: "dns.nextdns.io. 120 IN A 172.19.19.19"
local-data: "dns.dnsoverhttps.net. 120 IN A 172.19.19.19"
local-data: "doh.crypto.sx. 120 IN A 172.19.19.19"
local-data: "doh.powerdns.org. 120 IN A 172.19.19.19"
local-data: "doh-ch.blahdns.com. 120 IN A 172.19.19.19"
local-data: "doh-jp.blahdns.com. 120 IN A 172.19.19.19"
local-data: "doh-de.blahdns.com. 120 IN A 172.19.19.19"
local-data: "dns.dns-over-https.com. 120 IN A 172.19.19.19"
local-data: "doh.securedns.eu. 120 IN A 172.19.19.19"
local-data: "dns.rubyfish.cn. 120 IN A 172.19.19.19"
local-data: "doh-2.seby.io. 120 IN A 172.19.19.19"
local-data: "doh.seby.ie. 120 IN A 172.19.19.19"
local-data: "commons.host. 120 IN A 172.19.19.19"
local-data: "doh.dnswarden.com. 120 IN A 172.19.19.19"
local-data: "dns-nyc.aaflalo.me. 120 IN A 172.19.19.19"
local-data: "dns.aaflalo.me. 120 IN A 172.19.19.19"
local-data: "doh.appliedprivary.net. 120 IN A 172.19.19.19"
local-data: "doh.captnemo.in. 120 IN A 172.19.19.19"
local-data: "doh.tiar.app. 120 IN A 172.19.19.19"
local-data: "doh.dns.sb. 120 IN A 172.19.19.19"
local-data: "rdns.faelix.net. 120 IN A 172.19.19.19"
local-data: "doh.li. 120 IN A 172.19.19.19"
local-data: "doh.armadillodns.net. 120 IN A 172.19.19.19"
local-data: "doh.netweaver.uk. 120 IN A 172.19.19.19"
local-data: "jp.tiar.app. 120 IN A 172.19.19.19"
local-data: "doh.42l.fr. 120 IN A 172.19.19.19"

Here is sample query for one of those.

;; QUESTION SECTION:
;doh.dns.sb.                    IN      A

;; ANSWER SECTION:
doh.dns.sb.             120     IN      A       172.19.19.19

Simple add that to your custom box in unbound, then put a block log rule in place that logs anything that tries to go to my out of thin air IP that I do not use locally 172.19.19.19

This way I can see if anything is trying to resolve this by just looking if any hits in the firewall rules, and then look into the logs to which IP tried and investigate what its trying to do exactly.

Keep in mind, this could cause you some grief if any of those fqdn are serving up stuff other than doh that you might want to get to.. Some are on other ports other then 443, so I just made the block rule any..

kiokoman

bind9

$TTL 60
@            IN    SOA  localhost. root.localhost.  (
                      2019091601
                      3H  ; refresh
                      1H  ; retry
                      1W  ; expiry
                      1H) ; minimum
              IN    NS    localhost.

; Make TRR unavailable
dns.adguard.com 120 IN CNAME  .
dns-family.adguard.com 120 IN CNAME  .
dns.google 120 IN CNAME  .
cloudflare-dns.com 120 IN CNAME  .
dns.quad9.net 120 IN CNAME  .
dns9.quad9.net 120 IN CNAME  .
dns10.quad9.net 120 IN CNAME  .
doh.opendns.com 120 IN CNAME  .
doh.cleanbrowsing.org 120 IN CNAME  .
dns.nextdns.io 120 IN CNAME  .
dns.dnsoverhttps.net 120 IN CNAME  .
doh.crypto.sx 120 IN CNAME  .
doh.powerdns.org 120 IN CNAME  .
doh-ch.blahdns.com 120 IN CNAME  .
doh-jp.blahdns.com 120 IN CNAME  .
doh-de.blahdns.com 120 IN CNAME  .
dns.dns-over-https.com 120 IN CNAME  .
doh.securedns.eu 120 IN CNAME  .
dns.rubyfish.cn 120 IN CNAME  .
doh-2.seby.io 120 IN CNAME  .
doh.seby.ie 120 IN CNAME  .
commons.host 120 IN CNAME  .
doh.dnswarden.com 120 IN CNAME  .
dns-nyc.aaflalo.me 120 IN CNAME  .
dns.aaflalo.me 120 IN CNAME  .
doh.appliedprivary.net 120 IN CNAME  .
doh.captnemo.in 120 IN CNAME  .
doh.tiar.app 120 IN CNAME  .
doh.dns.sb 120 IN CNAME  .
rdns.faelix.net 120 IN CNAME  .
doh.li 120 IN CNAME  .
doh.armadillodns.net 120 IN CNAME  .
doh.netweaver.uk 120 IN CNAME  .
jp.tiar.app 120 IN CNAME  .
doh.42l.fr 120 IN CNAME  .

i can change the dot to some "out of thin air ip" if i want to log, reporting NX for the moment

laboratorio@server:/$ dig @127.0.0.1 doh.crypto.sx

; <<>> DiG 9.11.3-1ubuntu1.9-Ubuntu <<>> @127.0.0.1 doh.crypto.sx
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 10547
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 2

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: dda6e2c23949f6cda855e6485d7fa7fddeb8459af20d8f8b (good)
;; QUESTION SECTION:
;doh.crypto.sx.                 IN      A

;; ADDITIONAL SECTION:
overrides.              60      IN      SOA     localhost. root.localhost. 2019091601 10800 3600 604800 3600

;; Query time: 4 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Sep 16 17:19:25 CEST 2019
;; MSG SIZE  rcvd: 129