Strange DNS Issue for all clients

pwood999

Hi All,
Getting DNS resolution failure periodically for all clients on my LAN. DHCP is configured to tell clients to use pfSense as the DNS. Dual WAN is all static IP's. Inbound & Outbound resolver interfaces are both set to "All".

Today all browsers & command line in both Macbook & windows failed to resolve sites such as Amazon, BBC, etc. although google.co.uk seemed to be working.

The pfSense DNS Lookup diagnostic worked ok.

Restarting the DNS Resolver seemed to fix the problem for clients. No restart of client network interfaces required.

Any ideas ?

bmeeks

If you have checked the box in the DHCP Server settings for it to update DNS with client hostnames when issuing/renewing a DHCP lease, then you will cause DNS problems in your network. That's because currently the DHCP server daemon on pfSense will restart the unbound daemon that underpins the DNS Resolver each time a DHCP lease updates. That restart means DNS lookups don't happen while unbound is restarting.

Also, if you are using pfBlockerNG-devel and its DNSBL feature, that further exacerbates the problem because large DNS blocklists take a long time to parse and load, so the DNS Resolver takes longer to start up.

Are you using DNSBL with Python mode enabled? If not, then check the DHCP setting I mentioned first. Uncheck that if checked. Also see if the DNS Resolver has stopped when you are having DNS issues. The system log in pfSense can be a help here.

pwood999

DHCP client hostname update was already disabled.

I have also now disabled registering static mappings in case that caused any problems.

I'm not using pfBlockerNG, etc.

Will monitor & see what happens.

bmeeks

You mentioned in the original post about restarting the DNS Resolver. When you did that, was it stopped and you started it? Or was it still running and you just restarted it?

If it was stopped, you need to check the pfSense system log for any clues as to why it may have stopped. If stopped, then of course no name resolution will happen.

pwood999

I believe it was running, but not sure. Restarting it definitely sorted client DNS, so maybe it had stopped,

The pfSense diagnostics was able to resolve even when clients could not, but not sure if that uses the resolver, or just sends queries out to external dns ?

It has been stable last night & all day today so far.

bmeeks

@pwood999 said in Strange DNS Issue for all clients:

I believe it was running, but not sure. Restarting it definitely sorted client DNS, so maybe it had stopped,

The pfSense diagnostics was able to resolve even when clients could not, but not sure if that uses the resolver, or just sends queries out to external dns ?

It has been stable last night & all day today so far.

The default setup out-of-the-box is for pfSense to use its own DNS Resolver (so the unbound daemon) for lookups. But many people fiddle with that by adding external DNS servers to the DNS Settings section of the SYSTEM > GENERAL SETTINGS tab. By default those DNS boxes should be empty, and so pfSense would use the local DNS Resolver for any name resolution it needs. But if you put another DNS server IP address there, then pfSense itself will use that server and not the local resolver. So that may be what is happening in your case. pfSense itself is using a different DNS server than your LAN clients which are likely using unbound (the DNS Resolver on pfSense).

That setup is not ideal. You generally want everybody in a network using the same DNS server so returned results are uniform.

You can configure the DNS Resolver to either resolve via the root servers or to forward to an external server. How do you have it configured?

And I will provide this suggestion, but bear in mind I have no idea of your skill level here so don't think I'm patronizing. Make sure that you fully understand the difference between resolving and forwarding when it comes to DNS. Many users post here with DNS issues they have inflicted upon themselves because they do not understand the very important distinction between resolving and forwarding and how to correctly configure each in pfSense. The best way to start out with pfSense is to DO NOTHING to any DNS settings. Leave them at the out-of-the-box default and it will just work. By default the DNS Resolver is enabled and it is set to resolve via the root servers. And pfSense itself is configured to use the local DNS Resolver.

pwood999

@bmeeks I did have my two WAN routers in System-General because I though that was required, but have removed them now.

Here's the resolver config:

<unbound>
<enable></enable>
<active_interface>all</active_interface>
<outgoing_interface>all</outgoing_interface>
<custom_options></custom_options>
<hideidentity></hideidentity>
<hideversion></hideversion>
<port></port>
<system_domain_local_zone_type>transparent</system_domain_local_zone_type>
<hosts>
<host>ESXi-Host</host>
<domain>wood</domain>
<ip>192.168.123.20</ip>
<descr></descr>
<aliases></aliases>
</hosts>
<hosts>
<host>HP_DL160_ILO</host>
<domain>wood</domain>
<ip>192.168.123.248</ip>
<descr><![CDATA[DL160 ILO Management Port]]></descr>
<aliases></aliases>
</hosts>
<hosts>
<host>HP_DL380_ILO</host>
<domain>wood</domain>
<ip>192.168.123.247</ip>
<descr></descr>
<aliases></aliases>
</hosts>
<sslcertref>56c89c8fa2c20</sslcertref>
<dnssec></dnssec>
<tlsport></tlsport>
<msgcachesize>4</msgcachesize>
<outgoing_num_tcp>10</outgoing_num_tcp>
<incoming_num_tcp>10</incoming_num_tcp>
<edns_buffer_size>auto</edns_buffer_size>
<num_queries_per_thread>512</num_queries_per_thread>
<jostle_timeout>200</jostle_timeout>
<cache_max_ttl>86400</cache_max_ttl>
<cache_min_ttl>0</cache_min_ttl>
<infra_host_ttl>900</infra_host_ttl>
<infra_cache_numhosts>10000</infra_cache_numhosts>
<unwanted_reply_threshold>disabled</unwanted_reply_threshold>
<log_verbosity>1</log_verbosity>
</unbound>

bmeeks

@pwood999 said in Strange DNS Issue for all clients:

@bmeeks I did have my two WAN routers in System-General because I though that was required, but have removed them now.

Here's the resolver config:

<unbound>
<enable></enable>
<active_interface>all</active_interface>
<outgoing_interface>all</outgoing_interface>
<custom_options></custom_options>
<hideidentity></hideidentity>
<hideversion></hideversion>
<port></port>
<system_domain_local_zone_type>transparent</system_domain_local_zone_type>
<hosts>
<host>ESXi-Host</host>
<domain>wood</domain>
<ip>192.168.123.20</ip>
<descr></descr>
<aliases></aliases>
</hosts>
<hosts>
<host>HP_DL160_ILO</host>
<domain>wood</domain>
<ip>192.168.123.248</ip>
<descr><![CDATA[DL160 ILO Management Port]]></descr>
<aliases></aliases>
</hosts>
<hosts>
<host>HP_DL380_ILO</host>
<domain>wood</domain>
<ip>192.168.123.247</ip>
<descr></descr>
<aliases></aliases>
</hosts>
<sslcertref>56c89c8fa2c20</sslcertref>
<dnssec></dnssec>
<tlsport></tlsport>
<msgcachesize>4</msgcachesize>
<outgoing_num_tcp>10</outgoing_num_tcp>
<incoming_num_tcp>10</incoming_num_tcp>
<edns_buffer_size>auto</edns_buffer_size>
<num_queries_per_thread>512</num_queries_per_thread>
<jostle_timeout>200</jostle_timeout>
<cache_max_ttl>86400</cache_max_ttl>
<cache_min_ttl>0</cache_min_ttl>
<infra_host_ttl>900</infra_host_ttl>
<infra_cache_numhosts>10000</infra_cache_numhosts>
<unwanted_reply_threshold>disabled</unwanted_reply_threshold>
<log_verbosity>1</log_verbosity>
</unbound>

Read my further edits to my post just above yours.

Did either of your WAN interfaces flap? The DNS Resolver does not take kindly to the physical interface it is listening on going down and coming back. It can cause the daemon to die. By default, the DNS Resolver will listen on all interfaces but it will not respond to any WAN queries because of the default ACL permissions.

pwood999

ok, thanks. Apart from those local host overrides, I think it's back to default settings now.

bmeeks

@pwood999 said in Strange DNS Issue for all clients:

ok, thanks. Apart from those local host overrides, I think it's back to default settings now.

If you have problems again, and you find the DNS Resolver service not running, first check the pfSense system log to see if there are any clues as to what happened.

Another thing you can try if the problem returns is to configure the Resolver to bind to only LAN interfaces (not the WANs). I assume you have dual WANs for some kind of failover redundancy, so if one of those physical links goes down, the DNS Resolver itself may crash. Not binding the Resolver to the WAN interfaces could help prevent that.

And you may already have this setup, but it's good practice to put some kind of small switch between each WAN interface and the ISP demarc equipment. That way the physical WAN link is likely to stay up even when communications across it fails. Having the physical link stay up will help keep from bouncing the unbound daemon.

pwood999

@bmeeks ok, I presume you mean to bind Network Interfaces to LAN, but set Outgoing to the two WAN interfaces like this ?

What about my OpenVPN server ? This is tied to localhost, but has inbound port forwards from both WAN modems.

bmeeks

@pwood999 said in Strange DNS Issue for all clients:

@bmeeks ok, I presume you mean to bind Network Interfaces to LAN, but set Outgoing to the two WAN interfaces like this ?

What about my OpenVPN server ? This is tied to localhost, but has inbound port forwards from both WAN modems.

Never mind what I said before. I don't believe what I was thinking is possible since the Resolver wants to be explicitly told which outgoing interfaces to use. I had forgotten about the Outgoing Network Interfaces section when I first replied.

In your case I would just leave those settings at the default which is "All".

Gertjan

@pwood999

Be careful with that setting "Network interfaces : LAN ( and only LAN ).

You saw the 'Localhost' at the bottom of the list ? You've omitted to select that one.
Result : pfSense itself, that is, code like the GUI, will consult what that IP. Not the LAN, not the WAN.
Local software, like on your phone and PC will consult always 127.0.0.1 or Localhost or ::1
On localhost, port 53, is running a local DNS 'collector' that will forward to an (that's what a typical Windows PC does) upstream router. As per instructiosn by the local DHCP server (pfSense) and the local DHCP client your your PC.
Type

ipconfig /all

to see your local DNS setting : it's most probably 192.168.1.1 == the LAN IP of pfSense and pSense is (should !) listen on that interface.

Actually, it's a no brainer : just keep "All" selected for both "Network Interfaces" and "Outgoing Network Interfaces". That's the default settings, created by Netgate after years of investigation.
It's not an issue that unbound is listing on some WAN type interface : nothing can come into the WAN anyway.

And before you say : but that is dangerous !!
Don't worry. It' looks scary because you think it's scary.

The Web server GUI nginx is also listing on All available interfaces ( but don't tell any body, as you might wake up the thin foil hat association )

[22.01-RELEASE][root@pfSense.mypfsense.net]/root: sockstat -4l | grep 'nginx'
......
root     nginx      98071 5  tcp4   *:443                 *:*
root     nginx      98071 6  tcp6   *:443                 *:*
root     nginx      98071 7  tcp4   *:80                  *:*
root     nginx      98071 8  tcp6   *:80                  *:*
root     nginx      97984 5  tcp4   *:443                 *:*
root     nginx      97984 6  tcp6   *:443                 *:*
root     nginx      97984 7  tcp4   *:80                  *:*
root     nginx      97984 8  tcp6   *:80                  *:*
root     nginx      97740 5  tcp4   *:443                 *:*
root     nginx      97740 6  tcp6   *:443                 *:*
root     nginx      97740 7  tcp4   *:80                  *:*
root     nginx      97740 8  tcp6   *:80                  *:*

which means : listen on every interface for every 443 and 80 using TCP.

Every interface == "All".