DNS unresponsive to clients

markster

I have 4 separate network segments on my netgate 5100 running pfsense+ (23.01). My DNS Resolver is pretty standard. Listening interfaces include all my local LAN's plus localhost. Outgoing interfaces - ALL. DNSSEC enabled.

Once configured all is working fine. I also have DNS redirect rules to force my DNS to all local clients.

Problem starts when I reboot my netgate 5100. After reboot all local clients on all segments are unable to resolve and DNS queries. At this point when I access netgate and internally run lookup eg: cnn.com it works.

Lookig at /var/unboud/unbound.conf I can validate that access-control entries are fine. At this point I have to stop and start DNS or make a small change and then things start working again.

Has anyone had this happened ?

SteveITS

@markster Have you installed the System Patches package and the patches? There is at least one for "Fix DNS Resolver missing automatic ACLs for local IPv6 when bound to All," though you say that's not your issue.

Is DNS Resolver set to listen on All interfaces? IIRC if one is a bit slow and comes up after Unbound loads it isn't seen.

markster

@steveits
Good suggestion. I did applied all recommneded patches, rebooted and no go. Still have the same issue.

For Interfaces I have selected only local interfaces and localhost. For Outgoing I selected ALL.

I tried different scenarious with interfaces and it does not make a difference. Its strange.

I ssh to the box and run unbound-control -c /var/unbound/unbound.conf reload|stop|start but these did not help at all.

I feel there is something outside of DNS that is the cause.

SteveITS

@markster Try All for Network Interfaces just to see. No one can connect on WAN unless it's allowed by firewall rule.

I vaguely recall a thread saying something similar, in the last month or two maybe, but I have not seen this issue across our clients.

I assume "make a small change" involves applying it which will restart unbound.

@markster said in DNS unresponsive to clients:

I also have DNS redirect rules to force my DNS to all local clients.

Does DNS work if client PCs connect to pfSense directly ? (without using the redirect)

markster

@steveits
I tried your suggestion and set Interfaces to ALL.
Unbound would restart every 5 minutes. Something is up.

When speaking about DNS redirect what I meant was that I have these rules

Screen Shot 2023-05-04 at 6.05.10 PM.jpg

This should not matter.

FYI.
Before I upgraded to pfsense+ I was on pfsense CE and run own Unbound DNS in docker on my Synology NAS.

When I enable this config and use remote DNS (like before) it all works and it is a solid architecture.
However, I wanted to try running DNS Resolver on pfsense netgear box to see if it meants my needs.
For now it is a show stopper for me.

SteveITS

@markster Every 5 minutes exactly sounds like a recurring task.

What is logged at the time?

If it was random, enabling DHCP registration triggers a restart at every lease renewal which is increasingly frequent as a network grows.

markster

@steveits
My bad. It was not. Verified.
I will keep digging. Maybe clear all logs and do a reboot so I can see exactly what may be causing that. It’s strange.
After rebooting I can access Netgate by IP so it’s not like I loose a network connection. I can perform DNS lookup from GUI so that tells me that Unbound is working.

markster

@markster Update.. After setting both Inbound and Outbound Interfaces to ALL then rebooting it started to work.

I will perform more test to validate.

markster

@markster Update: After about 1 hour DNS Resolver log shows "restarted Unbound 1.17.1".

I honestly dont know what to make of it, but it has been a busy day looking into it. Not much of a benefit for me to run Unbound DNS on pfsense+ if it is so unpredictable.

I am going to stick back to my proven setup, having Unbound on a separate host in a docker. I had this setup for over a year now and never had any, any issues with DNS, RPZ or my local LAN segments. 100% availability.
Big plus is that I can compile new version when its available.

I was hoping that moving to pfsense+ and running Unbound (1.17.1) on pfsense would be better architecture, in case say my NAS goes off-line. But, I am not sold yet to move to this configuration given how Unbound is glued into pfsense. My NAS in 5 years has never go off-line and it looks like Netgate 5100 also be running for along time.

So as it is now, I am sadly giving up.

Gertjan

@markster said in DNS unresponsive to clients:

Update: After about 1 hour DNS Resolver log shows "restarted Unbound 1.17.1".
I honestly dont know what to make of it, but it has been a busy day looking into it. Not much of a benefit for me to run Unbound DNS on pfsense+ if it is so unpredictable.

Example :
When you chose to use pfBlockerng, and update your DNSBL feeds every hour, then, if there was an updated feed, unbound will get restarted.

If you select :

then for every new DHCP lease, or DHCP, lease renewal, unbound gets restarted.

You can 'predict' the resolver's restarting.
I'll show you my unbound (resolver) restarts.

markster

@gertjan None of this applies to my case.
I do not use pfBlockerng or checked DHCP Registration.

It is unexplained.

Gertjan

@markster

Just to be sure :

grep 'start' /var/log/resolver.log

Ho many per day ? hour ? or even minute ?

pfBlockerng and 'DHCP registration' are just two possible 'unbound re-starters'.
If 'any Interface' on pfSense, to which unbound is bound, goes down and/or up, unbound restarts also.
There are probably other reasons ... I don't know them all.

Example : recently, some one noticed that unbound restarted often.
At the end we discovered that he uses a direct LAN cable from his PC to the LAN interface, and his PC went to sleep every xx minutes, thus pulling the LAN NIC down.
The solution was : add a switch.

And then there is the issue that others have seen : forwarding to x.x.x.x (using IPv6 ?) makes unbound not happy - fail ....
I'm test-driving unbound right now, forwarding to 9.9.9.9 and the IPv6 equivalent, using TLS, and all goes well ....
I've been forwarding to the 1.1.1.1 family for weeks : I've nothing to declare.

markster

@gertjan Thank you for suggestions. You talking very good points.
There are just so many cases of Unbound restarting that it maybe difficult to point out. Like the case you mentioned about the PC is directly connected to pfsense box. This could be my case. However, I don’t see a reason why this would force Unbound restarts.

I moved on and restored my original configuration. These are just few advantages I see running DNS separate from the firewall.

No restarts (pfsense+ related)
efficient DNS cache
easy maintenance
compile and install new versions on demand

I implemented my own ads/malware blocking with RPZ - I use the same DNSBL feeds that pfblockerng is using. When one any DNSBL feed gets updated Unbound does not need to be restarted. Rather you can issue command "unbound-control auth_zone_reload file".

SteveITS

@markster said in DNS unresponsive to clients:

PC is directly connected to pfsense box. This could be my case. However, I don’t see a reason why this would force Unbound restarts.

If the PC hibernates or shuts down pfSense sees this as an interface up/down event and restarts multiple services, to see the new/unsee the old interface.

I'm not saying you're having a problem, or that your solution won't work, but it's not normal for unbound to restart without a reason. If unbound was crashing it would not restart on its own.

Shell Output - unbound-control -c /var/unbound/unbound.conf status

version: 1.17.1
...
uptime: 3166788 seconds

...so ~36 days which was when I made the last config change on that router.

johnpoz

@steveits said in DNS unresponsive to clients:

uptime: 3166788 seconds

nice - if I didn't always make changes to help users showing how to do something ;)

I'm only at 83 hours ;)

uptime: 299808 seconds

Gertjan

@johnpoz said in DNS unresponsive to clients:

if I didn't always make changes to help users showing how to do something ;)

Here.

Every 'dip' is a restart.

Now, I'm even forwarding to quad9.
Still waiting for it to fail.

markster

@gertjan I decided, as a last resort try to start from default setting. Reset the box to default and re-entered the rules. Took me a while but now all its working as designed/intended.

My Interfaces are set to include only local+localhost. Outbound is set to ALL.

Tested the configuration by rebooting netgate 5100
and also Disconnect/reconnect clients. Unbound did not restarted.

markster

@markster After running this config DNS started again to restart caused by dhcp renewals. I also noticed that for many cases Unbound get restarted were it does not need to be.
Cases like chaning a setting in General->DNS Resolution; or
chaning Advanced option in Unbound. These could just call unbound-control reload.

Someone on this forum has mentioned that many of these issues have been fixed in OPnsense. The issue with DHCP leases restarting DNS is for me a mind twister. It does not need to happen.

Unfortunalty it looks like it is a big issue to fix, I suspect that there are just some many dependencies that no one wants to touch it. Sad.

I am in the situation that I have a NAS box and I can run Unbound in a docker. This is my "solid" solution. I wanted to test Unbound on my pfsense 5100 but I will not spend more time trying to get things working (unpredictable), in effect this should be working perfectly out of the box. After all DNS is such a important part of any network. I cannot understand why Netgate and pfsense developers does not think that way.

RobbieTT

@gertjan said in DNS unresponsive to clients:

pfBlockerng and 'DHCP registration' are just two possible 'unbound re-starters'.
If 'any Interface' on pfSense, to which unbound is bound, goes down and/or up, unbound restarts also.

As an aside, DNS Resolver on my system is not restarted due to pfBlockerNG.

As for an interface going down, yep - got that T-shirt. I've been minded to change the 'All' bind to just my LAN interfaces to avoid a Resolver restart when my WAN link goes down / up again.

️

johnpoz

@robbiett said in DNS unresponsive to clients:

LAN interfaces to avoid a Resolver restart when my WAN link goes

I bind my outgoing to loopback.. That never goes down ;)

And why would you have it listen on your wan? You serving up dns off your wan interface? When it talks outbound, that traffic would be natted to your public IP anyway, etc.