Slow DNS after 22.05

Jax

@lohphat said in Slow DNS after 22.05:

force reload a page for the domain to resolve.

You sure your browser isn't doing doh? If you feel something isn't resolving - then troubleshoot it vs just thinking X is the problem. If something is taking long to resolve - why?

Some of us experiencing this problem have done a bit of testing, including from the command line. The resolution problem is not specific to browsing, it's an intermittent failure of resolution.

johnpoz

@jax said in Slow DNS after 22.05:

it's an intermittent failure of resolution.

This screams unbound restarting.. Is it?

[22.05-RELEASE][admin@sg4860.local.lan]/: unbound-control -c /var/unbound/unbound.conf status
version: 1.15.0
verbosity: 1
threads: 4
modules: 2 [ validator iterator ]
uptime: 457693 seconds
options: control(ssl)
unbound (pid 87400) is running...
[22.05-RELEASE][admin@sg4860.local.lan]/:

457k seconds up - whats that like 5 days? Only reason its prob not longer is testing something for some thread here, etc.

Jax

@johnpoz I don't know enough about these mechanisms to respond to your comment. I'd have to go hit the man pages and study more. Unless you can lay this out for me.

johnpoz

@jax look in your log - is unbound restarting, if it is - hard to resolve something if its not actually running or just started up couple of ms ago, etc.

You can validate how long its been up with that command..

Again to the root of the problem, if your having issues - why.. Unbound just doesn't say eh I don't feel like resolving that right now ;)

So either it is having a hard time finding what you asked for, or maybe it in the middle of restart why that specific query failed, etc.

Its a long running issue - register dhcp, unbound restarts - this can be quite often depending on the number of dhcp clients, the length of the lease, etc. When unbound restarts, cache is lost, etc

With pfblocker - the length of time for unbound to restart can be much longer than normal, etc.

If you ask unbound for www.domain.tld - and you don't get the answer you want - the question is why? There is almost always a logical explanation to why..

lohphat

@johnpoz For one thing the behavior is new since 22.01 to 22.05 and it's happening with common websites using different apps, not just FF.

I have DoH/DoT disabled in both 22.01 and 22.05 but the incidence of failed lookups since the upgrade is consistent over the last two days.

johnpoz

@lohphat said in Slow DNS after 22.05:

is new since 22.01 to 22.05

On 22.05 went from 22.01 - not seeing any such issue.. Have had zero issues resolving stuff.

If your having an issue - the logical thing to do is is troubleshoot why, not oh something wrong with version X vs Y.. Maybe there is nothing wrong with Y, but some other variable has been introduced. Like maybe whatever you doing now, unbound is taking longer to restart - maybe on your system it use to restart in like .3 seconds, and now its taking 30.. So before you never noticed, but now you do.

lohphat

@johnpoz As soon as I get I lookup error, I go and look at logs and find nothing out of the ordinary. I have DHCP registration disabled as I know it restarts unbound and with pfB-dev that's a non-starter.

I just can't find a smoking gun yet other than it's still happening and others are reporting it too.

Jax

In any case, things are currently working for me as follows:

DNS Resolver enabled
DNS Forwarder disabled
DNS Server Override checked "Allow"
DNS Servers set to 8.8.8.8 and 8.8.4.4

And not experiencing any slowness nor intermittent failures.

johnpoz

@jax so your pointing your clients to 8.8.8.8 ? Doesn't matter what dns you put into pfsense general tab, or allow from dhcp on wan, etc..

If unbound is running, dhcp would point to pfsense IP for dns (unless you told it something else in the dhcp server dns section) - unbound would resolve.. 8.8.8.8 would never come into play, other than pfsense itself looking up something if unbound stop working..

Clients would always use unbound, and never use 8.8.8.8, etc.. Those would only be for pfsense itself, not clients asking unbound..

You not having any issues - unbound working how its set to work out of the box (resolver).. So how is dns slow after 22.05??

Here is the thing, out of the box unbound is a resolver.. What settings you have in general have zero to do with unbound, and clients asking unbound on pfsense lan IP, etc.

Any servers you have in general are only for pfsense own use, if unbound is off or failing, etc. You know checking for an update, checking package list, etc. Zero to do with clients on your network asking unbound to lookup www.google.com

The only time servers you have listed in general could ever come into play for clients, is if you have unbound in forwarder mode.

Merlin55

Hey!

Since a few month I´m using a Netgate 3100 in my home network. I´ve had no issues with the Firewall itself or with the DNS Resolver on previous versions, except that the domain override hasn´t really worked.

With installing the Ver. 22.05, and no other changes were mady by the configuration, I also got problems to load webpages. It takes some seconds to react on the client, but then it will load normaly. My first thought: DNS problem!
I´ve been testing the on or other thing reading in this thread in the DNS Resolver and in the General Tab. Sometimes it feels better for the moment, but after a little time the behaviour is again like before. I got this on all devices in my network, so it doesn´t matter if it is a desktop computer (tested with Windows and Mac OSX an different webbrowser), or a mobile device (Smartphones and Tablets with iOS and Android). Without telling my wife I asked her if she figured out something about the internet, she also told me about "slow websites".

For testing I´ve installed a DNS Resolver on my Synology NAS and set it up to the clients by the DHCP Settings. The clients all got normal behaviour with a quick reaction by looking up for DNS and loading the content.

In my opinion the problem have to be something with the Firewall, the DNS Resolver and the Version 22.05, because it just came up on the same day of the update. Maybe it´s, like someone supposed in the thread earlier, really only on some Hardware versions like the ones with the ARM - Processor.

Greetings,

Markus

PS: I hope I was able to tell my thoughts rightly, because my native speech is german.

Gertjan

@lohphat said in Slow DNS after 22.05:

I'm getting a lot of these in the DNS Resolver log with pfBlockerNG-devel uninstalled, DNS forwarding and DNSSEC enabled:

Jul 9 12:43:02 	filterdns 	82159 	failed to resolve host steamusercontent.com will retry later again.
Jul 9 12:43:02 	filterdns 	82159 	failed to resolve host steamstatic.com will retry later again.
Jul 9 12:43:02 	filterdns 	82159 	failed to resolve host steamcontent.com will retry later again.

Why did you ask 'filterdns' to pre resolve these domains ?

Take one for yourself :

[22.05-RELEASE][admin@pfSense.mypfsense.net]/root: host steamcontent.com
steamcontent.com mail is handled by 10 us-smtp-inbound-2.mimecast.com.
steamcontent.com mail is handled by 10 us-smtp-inbound-1.mimecast.com.

You saw the word 'smtp' in the domain name ?
These domain names point to a host that is supposed to be a mail server. No web, no other service.
Are your firewall filtering using aliases that allow/forbid these hosts ?
Are you sending a lot of mail to these guys, using their mail facilities ?
smtp server nearly always have very static IP, as dynamic IP for a smtp server is a pure nightmare.

Also, these domain names imply that they are present on the domain name server "mimecast.com" : that's a casting DNS server so very accessible.
And still unbound has troubles resolving it ??
Your issue goes deeper as a 'filterdns' issue : it a DNS issue.
I advice you to stays away from any 'user' (== you) added DNS settings. Use the dafault Netgate DNS settings, and you'll be fine.

Keep in mind : 8.8.8.8 is only needed if you have to give
them your DNS requests ;)
Internet, DNS, etc, was working just fine even before 8.8.8.8 (and 1.1.1.1 etc etc) existed.

lohphat

This post is deleted!

lohphat

@gertjan

In drilling down to where the log errors are coming from, they are part of a firewall alias list used in a single f/w rule which is disabled.

My new question is why the alias being resolved used while the test rule I use the alias in, isn't being used at all.

I assume it's to support dynamic rules which may activate at any time. I removed the FQDN names from the aliases to remove them from generating log errors.

lohphat

It happened again. DNS forwarding off and DNS 'died' within 24 hours in a way which mostly affects CDN content (i.e. YouTube). Nothing interesting in the logs.

Enabling DNS forwarding apparently solves the problem. pfBlocker-devel doesn't seem to be part of the issue (yet).

johnpoz

@lohphat said in Slow DNS after 22.05:

DNS forwarding off and DNS 'died'

And what specific did not resolve? Did you do a dig from your client, did you do a dig +trace to see where the resolving was failing?

When you have a dns problem, you need to troubleshoot what exactly is failing.. Like your above entries - that just do not resolve, issue with the parent dns, etc.

lohphat

@johnpoz Can you recommend a log level setting other than Level 3 to catch the failures?

johnpoz

@lohphat log might not catch the failure.. Depending on what the failure is - getting back a nx isn't a failure, etc.

What specific fqdn did not resolve - what did you get back from unbound, was it refused was it nx, was it a timeout?

Saying dns didn't work is like telling the mechanic your car is broke. Without details - have no clue to what is wrong or could be wrong.

You can for sure up the log level - also you can have it log queries and replies with

log-queries: yes
log-replies: yes

In the option box, etc.

But some actual details what you specific asked for, what was the response? And then doing a dig +trace is prob going to tell you more of what is failing then in the log..

lohphat

@johnpoz said in Slow DNS after 22.05:

@lohphat log might not catch the failure.. Depending on what the failure is - getting back a nx isn't a failure, etc.

What specific fqdn did not resolve - what did you get back from unbound, was it refused was it nx, was it a timeout?

That's the problem, I don't have a specific FQDN but I have cases where YT thumbnails don't load in a browser on my PC or iPad or on my Roku on my TV -- other similar sites which use distributed content like linkedin.com also fail at the same time. Since the content is embedded, getting the FQDN is more of a challenge.

You can for sure up the log level - also you can have it log queries and replies with

log-queries: yes
log-replies: yes

In the option box, etc.

Thanks! Will try this.

But some actual details what you specific asked for, what was the response? And then doing a dig +trace is prob going to tell you more of what is failing then in the log..

Once I can do that I will.

lohphat

@johnpoz

Here we go!

One second duckduckgo.com is resolved and the SAME second it's not.

Note: Not in forwarding mode.

Jul 11 12:00:23 	unbound 	85415 	[85415:0] info: 192.168.0.114 improving.duckduckgo.com. A IN SERVFAIL 0.015555 0 42
Jul 11 12:00:23 	unbound 	85415 	[85415:1] info: 192.168.0.114 duckduckgo.com. A IN SERVFAIL 0.010721 0 32
Jul 11 12:00:23 	unbound 	85415 	[85415:0] info: 192.168.0.114 external-content.duckduckgo.com. A IN SERVFAIL 0.011008 0 49
Jul 11 12:00:23 	unbound 	85415 	[85415:1] info: 192.168.0.114 improving.duckduckgo.com. AAAA IN NOERROR 0.000000 1 121
Jul 11 12:00:23 	unbound 	85415 	[85415:0] info: 192.168.0.114 external-content.duckduckgo.com. AAAA IN NOERROR 0.000000 1 114
Jul 11 12:00:23 	unbound 	85415 	[85415:1] info: 192.168.0.114 improving.duckduckgo.com. AAAA IN
Jul 11 12:00:23 	unbound 	85415 	[85415:1] info: 192.168.0.114 duckduckgo.com. AAAA IN NOERROR 0.000000 1 97
Jul 11 12:00:23 	unbound 	85415 	[85415:0] info: 192.168.0.114 external-content.duckduckgo.com. AAAA IN
Jul 11 12:00:23 	unbound 	85415 	[85415:0] info: 192.168.0.114 improving.duckduckgo.com. A IN
Jul 11 12:00:23 	unbound 	85415 	[85415:1] info: 192.168.0.114 duckduckgo.com. AAAA IN

lohphat

@johnpoz

Even more frustrating:

Jul 11 12:06:13 	unbound 	85415 	[85415:1] info: 192.168.0.114 duckduckgo.com. AAAA IN SERVFAIL 0.002623 0 32
Jul 11 12:06:13 	unbound 	85415 	[85415:0] info: 192.168.0.114 duckduckgo.com. A IN SERVFAIL 0.002413 0 32
Jul 11 12:06:13 	unbound 	85415 	[85415:1] info: 192.168.0.114 duckduckgo.com. AAAA IN
Jul 11 12:06:13 	unbound 	85415 	[85415:0] info: 192.168.0.114 duckduckgo.com. A IN

But when I do a DNS query in pfsense it resolves while my client is getting can't resolve errors. Presumably due to negative caching.

Results
Result 	Record type
52.149.246.39	A
Timings
Name server 	Query time
127.0.0.1	4 msec
9.9.9.9	15 msec
149.112.112.112	13 msec
2620:fe::fe	13 msec
2620:fe::9	14 msec