Big DNS Problems *Illustrated*

johnpoz

@skalyx said in Big DNS Problems *Illustrated*:

I guess they did not provide good info.

Yup falls in line with stephenw10 comment and my own - there a lot of people that don't actually understand dns at even a basic level... But they think they can write up guides on how to do XYZ.. Mindless of makes any actual sense or not..

Cloudflare resolver is already doing dnssec.. You have to do ZERO to have it do dnssec if you forward your queries to them..

Simple test will show you that..

Here 1.1.1.1 does dnssec, so if ask it for test fqdn dnssec-failed.org, it fails...

$ dig @1.1.1.1 dnssec-failed.org

; <<>> DiG 9.14.1 <<>> @1.1.1.1 dnssec-failed.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 11978
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;dnssec-failed.org.             IN      A

;; Query time: 52 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Mon May 27 09:14:39 Central Daylight Time 2019
;; MSG SIZE  rcvd: 35

If you ask opendns - it returns IP that it shouldn't since that domain fails dnssec tests

$ dig @208.67.222.123 dnssec-failed.org

; <<>> DiG 9.14.1 <<>> @208.67.222.123 dnssec-failed.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3941
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;dnssec-failed.org.             IN      A

;; ANSWER SECTION:
dnssec-failed.org.      7200    IN      A       69.252.80.75

;; Query time: 67 msec
;; SERVER: 208.67.222.123#53(208.67.222.123)
;; WHEN: Mon May 27 09:14:31 Central Daylight Time 2019
;; MSG SIZE  rcvd: 62

skalyx

@johnpoz Thanks! I love learning stuff. I understand it better. The "dig" tool is definitely something I will have to learn about.
Unfortunately, we cannot believe everything on the internet!
It makes a lot of sense that we do not need to do DNSSEC with unbound if we forward our requests to a server that already makes DNSSEC queries for us.

stephenw10

If you are not using forwarding mode then whatever you have set in System > General is not used. So right now you are not using Cloudflare's DNS.

There is a check box for DNS over TLS in the GUI now which you have checked. That doesn't do anything until forwarding mode is enabled but if you do then it will use the servers defined in General Setup and will try to reach them over TLS. No need to add any custom options. Disable DNSSec if you do that as discussed.

Nice test @johnpoz.

Steve

skalyx

@stephenw10

Thanks Steve. I will deactivate DNSSEC support and let forwarding mode. I hope it will solve my issues. I will update whenever possible or when I get the same problems again !

johnpoz

If the reason for your problem is unbound restarting - forwarding mode is not going to fix anything.

doing DOH or DOT depending on what you what to call it, ie dns over tls is only going to bring other possible issues to the table that make it harder to troubleshoot.. Is your issue the tls tunnel? Is it the tls setup time? Is the added overhead of tcp on what is normally udp, etc.

I would use unbound out of the box, let it resolve - let it do dnssec.. And when/if you run into resolving problems or delays then troubleshoot those.. First thing to check for sure is if unbound was restarting when you were trying to do a query.

Things you can do to help with problematic resolving is turn on prefetch, turn on serve 0 TTL... This allows client to get the last known good record that is cached even if the ttl has expired.. While your resolving will then update in the background after the client has been given its answer to its question - if cached, etc.

At the core dns is pretty basic.. Your client asks for some fqdn host.domain.tld, if unbound has it in its cache then bam within couple of ms client will have an answer to its question. If not cached, then unbound will either ask something upstream (forwarder)... Who will in turn return the answer if cached, or will have to forward or resolve itself to find the answer..

At some point in the chain be it local or upstream to find some fqdn a resolver will be involved.. I prefer to just have the resolver local to me, so if there are any issues I can easy via the log or even sniff and watch the queries to see exactly where the problem is..

When you hand off your query to some ns upstream.. Its like asking a black box for an answer.. It either answers or it doesn't have will never know the "why" to either of those questions ;) Nor do you actually know for a fact that what you get handed back for host.domain.tld is actually valid.. But when you resolve locally, you know for sure since you will have gotten it direct from the authoritative ns of that domain directly.

skalyx

@johnpoz Thanks, John!
I see. Yes, enabling DOH will for sure make the troubleshooting harder as you stated.

However, I tried to activate DNSSEC Support and deactivate Forwarding mode as stated in my previous post where I posted my config screenshot. It just does not work even after rebooting. No DNS resolving... It only works when I activate forwarding mode.

I will, nevertheless, try to activate turn on prefetch and serve 0 TTL.

On another hand, my current configuration with no DNSSEC Support and forwarding mode just created me problems. It states: DNS_PROBE_FINISHED_BAD_CONFIG. After 2minutes, the page reload and everything gets back to normal. Then, same problem after 10minutes. I really do not understand...

Having a local resolver is for sure better but is not something doable for me because I do not have enough knowledge (yet).

EDIT: I already have the following activated:

Prefetch DNS Key Support: DNSKEYs are fetched earlier in the validation process when a Delegation signer is encountered
Serve Expired: Serve cache records even with TTL of 0

I just, however, activated "Prefetch Support: Message cache elements are prefetched before they expire to help keep the cache up to date" and deactivated "Harden DNSSEC Data: DNSSEC data is required for trust-anchored zones" as DNSSEC Support must be activated for this.

johnpoz

@skalyx said in Big DNS Problems *Illustrated*:

Then, same problem after 10minutes. I really do not understand...

And in the log for unbound - did it restart, did unbound stop?

resolving should work out of the box!!!

Your on a 4G connection... Does your ISP only allow access to its own DNS? And shitty ass connections to anything else?

Make sure you turn OFF and to be honest just remove pfblocker for any sort of testing.

Working sometimes and not working others points to unbound restarting if you ask me.. Simple enough to see in the log..

skalyx

@johnpoz
Yes, my ISP allows other DNS from what they told me.

I just added to my post:
EDIT: I already have the following activated:
Prefetch DNS Key Support: DNSKEYs are fetched earlier in the validation process when a Delegation signer is encountered
Serve Expired: Serve cache records even with TTL of 0
I just, however, activated "Prefetch Support: Message cache elements are prefetched before they expire to help keep the cache up to date" and deactivated "Harden DNSSEC Data: DNSSEC data is required for trust-anchored zones" as DNSSEC Support must be activated for this.

I will look in the logs whenever I get this issue again.
Thanks for the help!

stephenw10

Did you also disable DNS over TLS? As far as I know that does nothing without forwarding enabled but it should be disabled for resolving anyway.

Steve

johnpoz

If unbound is actually running... Can you resolve say pfsense own name? When you do a nslookup on your machine does it come back with a name for the IP?

example

$ nslookup
Default Server:  sg4860.local.lan
Address:  192.168.9.253

If not then unbound is not running... Or you client its having an issue talking to pfsense for dns in the first place... Which also could be your problem.

skalyx

@stephenw10
My current configuration is:

No DNSSEC Support
Forwarding mode activated
Use SSL/TLS for outgoing DNS Queries to Forwarding Servers (DNS over TLS activated)

if I deactivate forwarding mode, nothing works. Maybe I should turn of DNS TLS when I deactivate forwarding mode?

johnpoz

Yeah you should turn off tls if resolving - since that option is not viable if resolving.

skalyx

@johnpoz
I get the hostname and address with nslookup so my unbound is working, isn't it?

I just activate DNSSEC Support, deactivated forwarding mode and DNS over TLS. Let us see how it goes!

skalyx

This post is deleted!

skalyx

I just activate DNSSEC Support, deactivated forwarding mode and DNS over TLS.

It now works after having disable Harden DNSSEC Data: DNSSEC data is required for trust-anchored zones.

However, the problem still persists. I noticed in my log that :
May 27 19:25:38 unbound 84243:0 notice: Restart of unbound 1.9.1.

Moreover, DNS resolving queries are slow. Much slower than usual. Please let me know if you have any idea.

Thanks,

johnpoz

And how often are you seeing unbound restart?

Resolving can be slower to start.. Sure it might be a few ms longer talking the authoritative ns vs just asking a forwarder what what it has cached.. But once is cached then its no different, and once you resolve to where the authoritative ns is for domainx, then you don't have to walk down the tree from roots again until the whole tree ttl expires.

Not sure how anyone could care about a few ms, when they are wanting to FORWARD over tls ;)

If unbound is restarting all the time your going to loose the advantage in caching be it you forward or resolve.

skalyx

@johnpoz

Haha yes I know, but it is abnormally slow.
Just found the cause. It restarts about every minute. Here is a sample of log:
May 27 21:25:24 unbound 84243:0 notice: Restart of unbound 1.9.1.
May 27 21:26:33 unbound 84243:0 notice: Restart of unbound 1.9.1.
May 27 21:27:17 unbound 84243:0 notice: Restart of unbound 1.9.1.
May 27 21:27:48 unbound 84243:0 notice: Restart of unbound 1.9.1.
May 27 21:28:18 unbound 84243:0 notice: Restart of unbound 1.9.1.
May 27 21:30:28 unbound 84243:0 notice: Restart of unbound 1.9.1.
.... same

Putting DNS resolver in forwarding fixes the issues. I need to see if it causes problem (it normally does) and check the logs...

johnpoz

restarting that often is going to be make it useless.

Do you have it set to register dhcp or vpn clients? You prob want to bump its log level up to try and figure out why its restarting like that.. Which makes it useless.. how can it resolve or even forward if its restarting every few seconds. No wonder you having pain with dns.

Gertjan

Checkout the DHCP server log.

Every time a new lease comes in, you have many devices, some with very short lease times, know that unbound is restarted.

Solution for this issue :

Uncheck DCHP Registration.
Where possible, leave all your devices on DHCP - but make static leases using their MAC addresses.

On my pfSense installations, unbound restarts one a day, or less - other events can trigger a restart also. This means it keeps up it's DNS cache, and that's what's counts.

skalyx

@johnpoz
Hmm. No VPN clients. Yes, I had register DHCP leases in the DNS resolver.
@Gertjan Thanks for your answer. I do not understand why it would restart nor how DHCP leases in a DNS resolver works but I just unchecked it according to your advice! I hope it will fix my issues.
I have about 80 clients at home (Google home, smartphones, etc.) where 10 of my clients such as my Synology NAS, smart switches, and access points are on static ips. Are you talking about this or about DHCP related to DNS?