Slow DNS after 22.05

Jax

This is really quite frustrating, I'm not getting anywhere debugging this slowness problem.

bmeeks

This is really quite frustrating, I'm not getting anywhere debugging this slowness problem.

The first step in troubleshooting is to isolate the problem. Since you've tried a number of things on pfSense itself, why not take pfSense's DNS completely out of the picture?

Do this -- in the SYSTEM > GENERAL SETUP page, down in the DNS Settings area, put 8.8.8.8 (the Google DNS server IP) in the DNS Servers box. Save that change.
Next, go to SERVICES > DHCP SERVER and in Servers in the DNS Servers box also put 8.8.8.8. This will tell the DHCP server to give your LAN clients the Google DNS server for name resolution.

Now pfSense is out of the picture unless you have created any DNS related firewall rules previously. See how things behave with this test setup. If things are good, then you can assume you are having issues with unbound on your box when using the default settings. Those default settings configure the DNS Resolver to "resolver mode" and hand out the address of the pfSense box as the DNS server for your DHCP clients.

If things are still poor, then pfSense it likely not at fault here (assuming you don't have a firewall rule in the way), and you need to look elsewhere for the problem.

If you have any DNS related firewall rules, make sure you are allowing both UDP and TCP for port 53 as some DNS lookups will need to use TCP.

vaidas

I am too having problems after 22.05 upgrade with dns resolves timing out completely
unbound logs does not show any problems.
config haven't changed from 22.01 where dns worked perfectly.
running bare metal
plugins: openvpn client export, nut service for ups, watchdog that's it.

Jax

@bmeeks I took your suggestion and this morning things seem to be working better.
We'll see how things go on later in the day, thanks for your help.

Jax

@bmeeks of course this very much suggests pfSense DNS is indeed the problem.

bmeeks

@jax said in Slow DNS after 22.05:

@bmeeks of course this very much suggests pfSense DNS is indeed the problem.

But it's not a widespread problem or the forum here would be overflowing with posts about it. There are only a few. Not saying there can't be a problem, but it's not affecting everyone it seems.

It's entirely possible your virtualization environment could be at fault here as well. There could be an issue with the latest pfSense (FreeBSD) version and Proxmox.

Jax

@bmeeks Thanks again for your help.

I'm on a Netgate device that I purchased with pfSense already installed so no virtualization issues that would be unique to my setup. I have made no software modifications. There are many variables here:

pfSense DNS
pfSense DHCP interacting with desktop operating system
pfSense DNS interacting with service provider premises devices

... and so forth.

In any case, the setup has been working for about 18 months for me, I personally made no changes and the problem seemed to emerge with pfSense 22.05. However, Correlation ≠ Causation as we all have been taught So I suppose I will have to continue to gather clues and see what I can figure out over time .

lohphat

@vaidas I've come to report the same thing.

I thought it was PfBlockerNG-devel but even with that off I'm seeing CDN content fail (e.g. YouTube). If I bypass unbound by using a client VPN, no problems.

Seeing a lot of timeout and "domain not found" and have to manually reload pages to get them to load.

No changes were made to the unbound settings between 22.01 and 22.05.

Jax

@lohphat Yes, I'm seeing the same thing, that the problem seems to be mostly with cdn site resolution.

bmeeks

@jax said in Slow DNS after 22.05:

@bmeeks Thanks again for your help.

I'm on a Netgate device that I purchased with pfSense already installed so no virtualization issues that would be unique to my setup.

Sorry, I confused your post with another at the top of this thread where the OP said they were running on Proxmox.

I see where you said you were running on an SG-2100. That is an ARM-based appliance (not Intel). Another poster in this thread has an SG-3100 in his signature. That is also an ARM-based appliance. Could be an issue with the latest unbound version and ARM hardware. I did notice that when my SG-5100 updated it pulled down a new unbound version as part of the upgrade. I've not seen any issue on my SG-5100, but it is Intel-based hardware.

There have, in the past, been some weird issues with software running on ARM hardware due in part to some quirkiness with the llvm compiler used.

Jax

@bmeeks Thanks again for clarifying ... @rcoleman-netgate do you have any further observations on this issue? It does seem to be Netgate-specific.

domnado

I had to put in a support ticket because I wasn't able to boot my device after trying to rollback to 22.01. While in my support ticket for getting the firmware to reflash I was told about an IP monitoring option "System->Routing->Gateways" edit ipv4 Gateway and to change "Monitor IP" to a public DNS server because the system pings this IP on a consistent basis and some ISPs treat this as a DoS attack and will temporarily block it, then the router will consider the WAN connection down. After using a Google, Cloudflare or OpenDNS server IP address in this field I have not had any issues with unbound on my SG 1100.

lohphat

Datapoint:

I had DNSSEC enabled in 22.01 and the setting carried over into 22.05 when I upgraded this morning.

After playing with different configs all day, turning off DNSSEC seems to have made things stable for me. I'll keep playing with it.

8jul2022 Update: Nope, it was better after startup but started misbehaving anyway after an hour.

9jul2022 Update: The stable config for me is to disable local DNS resolution and just forward it to the upstream DNS providers. DNSSEC is enabled. I've just added pfBlockerNG-devel into the mix and will see how things work over the day.

9jul2022 Update 2: Had to disable pfBlockerNG-devel due to inability to resolve domains. Just running on unbound only, no filtering.

lohphat

I'm getting a lot of these in the DNS Resolver log with pfBlockerNG-devel uninstalled, DNS forwarding and DNSSEC enabled:

Jul 9 12:43:02 	filterdns 	82159 	failed to resolve host steamusercontent.com will retry later again.
Jul 9 12:43:02 	filterdns 	82159 	failed to resolve host steamstatic.com will retry later again.
Jul 9 12:43:02 	filterdns 	82159 	failed to resolve host steamcontent.com will retry later again.

johnpoz

@lohphat said in Slow DNS after 22.05:

DNS forwarding and DNSSEC enabled:

That is never a good idea, if your going to forward where you forward either does dnssec or it doesn't you asking for it does nothing - other than problematic.

If you forward, uncheck use dnssec in the unbound settings.

example, 8.8.8.8 does dnssec be it you ask it to or not

$ dig @8.8.8.8 www.dnssec-failed.org

; <<>> DiG 9.16.30 <<>> @8.8.8.8 www.dnssec-failed.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 13556
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.dnssec-failed.org.         IN      A

;; Query time: 83 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sat Jul 09 13:05:11 Central Daylight Time 2022
;; MSG SIZE  rcvd: 50

see how that fails - while if I ask 4.2.2.2 it does not

$ dig @4.2.2.2 www.dnssec-failed.org

; <<>> DiG 9.16.30 <<>> @4.2.2.2 www.dnssec-failed.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 25984
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 8192
;; QUESTION SECTION:
;www.dnssec-failed.org.         IN      A

;; ANSWER SECTION:
www.dnssec-failed.org.  7200    IN      A       68.87.109.242
www.dnssec-failed.org.  7200    IN      A       69.252.193.191

;; Query time: 191 msec
;; SERVER: 4.2.2.2#53(4.2.2.2)
;; WHEN: Sat Jul 09 13:05:24 Central Daylight Time 2022
;; MSG SIZE  rcvd: 82

lohphat

@johnpoz Yes, I know, but I'm forwarding to the 9.9.9.9 group of servers and they claim to support DNSSEC.

https://quad9.net/support/faq/#dnssec

johnpoz

@lohphat said in Slow DNS after 22.05:

9.9.9.9 group of servers and they claim to support DNSSEC.

There is no need to ask for dnssec - they are doing it be it you ask them or not..

$ dig @9.9.9.9 www.dnssec-failed.org

; <<>> DiG 9.16.30 <<>> @9.9.9.9 www.dnssec-failed.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 1538
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
; EDE: 9 (DNSKEY Missing)
;; QUESTION SECTION:
;www.dnssec-failed.org.         IN      A

;; Query time: 72 msec
;; SERVER: 9.9.9.9#53(9.9.9.9)
;; WHEN: Sat Jul 09 13:10:14 Central Daylight Time 2022
;; MSG SIZE  rcvd: 56

lohphat

@johnpoz Ok, Done. The option needs a better info text as it is it seems to imply that it needs to be on to support any DNSSEC at all.

johnpoz

@lohphat I would agree with you, should really have a note under it - If your going to forward do not set this or something like that.

lohphat

Still getting filterdns errors with DNSSEC unchecked:

Jul 9 15:23:01 	filterdns 	82159 	failed to resolve host steamstatic.com will retry later again.
Jul 9 15:23:01 	filterdns 	82159 	failed to resolve host steamcontent.com will retry later again.
Jul 9 15:23:01 	filterdns 	82159 	failed to resolve host steamusercontent.com will retry later again.
Jul 9 15:18:01 	filterdns 	82159 	failed to resolve host steamstatic.com will retry later again.
Jul 9 15:18:01 	filterdns 	82159 	failed to resolve host steamcontent.com will retry later again.