Lost all client DNS

IanHK

I had trouble yesterday setting up a 2nd LAN interface but went to bed with everything apparently working after a lot of help on this forum spotting a dumb mistake.

This morning I noticed my phone wifi is saying ”internet might be unavailable" and although it does seem to work, PCs on both wired LAN and wifi LAN2 cannot connect to internet.

The ipconfig is 100% as expected, with correct IP, subnet mask, DNS server and Gateway.

In pfSense diagnostics, DNS Lookup is working, but nslookup on PC shows:
DNS request timed out
Server: unknown
Address: 192.168.2.1
DNS request timed out.
Timeout was 2 seconds.

Before last night, I had a NAT port forward on LAN pushing all DNS on 53/853 to 127.0.0.1 which was working OK for weeks. Last night I replicated these for the LAN2 interface.

DNS revolver has been restarted with ”DNS query forwarding” both ticked and unticked. The ”network interfaces” and outgoing network interfaces are set to “All“

Any idea what might be blocking the DNS on the clients or what I can upload to clarify?

Thanks

IanHK

I backed out all the rules and started again.

No port forward

works.

Port forward 53 to 127.0.0.1

still works, and auto creates firewall rule allowing 53 traffic to 127.0.0.1

Added back the firewall rule to block 53 going anywhere, placing it under the auto-tune.

does NOT work.

Turn off the rule in [3] and it works again.

I understood port fwd rules were applied before LAN firewall rules so I dont understand why rule 3 is blocking requests ?

It should be redundant if the port forward works, but this was the first ”cool" thing I tried in pfSense so just confused about what's going on.

IanHK

Even more confused the more I look at this.

At step 2 it works.

Then if I just add another port fwd rule for the LAN2 interface, the LAN DNS stops working again?!

Surely I can port fwd DNS requests on 2 different interfaces without them conflicting. Is there something in the DNS resolver that needs to be taken into account here?

Grimson

Show screenshots of your NAT and firewall rules if you want help.

IanHK

Hi Grimson, thank you for looking at this for me:

The original setup was just simple WAN/LAN where, as a first step, I was trying to lockdown the DNS.

I had a port-forward rule to send all DNS queries to the pfSense DNS resolver.
This auto-generated the firewall rule to allow this connection on LAN.
It all seemed to worked, no issues.

I then moved on to using a LAN_2 interface so I could separate the servers (NAS) from the WiFi clients - looking for an easy way to stop "guest" users VLANs on the access point. All sorts of self-inflicted caused by an overlooked typo, but the basic DHCP was sorted out (by others!).

Thinking it was all sorted, I duplicated my DNS Lockdown onto LAN_2 - but then everything fell apart. I can replicate the basic issue by simply adding the second port forward for the LAN_2 interface. As soon as I do this, the DNS on the original LAN stops working.

Screenshots of "Before" on a LAN workstation

And after adding the LAN_2 port forward - but still using the same LAN workstation for the nslookup:

Grimson

@IanHK
You are still missing the rules from LAN_2.

IanHK

Grimson

Some more information is required:

DNS on LAN is still working with the port forwards in place? Or does it stop working on all interfaces?
IP addresses of your LANs.
Show all rules, not just a snipped.
Try "nslookup google.com 8.8.8.8" to actually trigger the port forward and see if that works.

Just for clarity, I do forward DNS requests from multiple LANs to unbound without an issue.

IanHK

OK, thanks -

No, DNS on LAN (192.168.1.x) breaks when the port forward for LAN2 is added - that's what I find so bizarre. The DHCP assigned details look right:

LAN is 192.168.1.0/24 LAN2 is 192.168.2.0/25
That was all the rules except for the LAN2 where a block ALL dropped off the bottom, sorry:

IanHK

with just the LAN port forward:

nslookup google.com 8.8.8.8 - fails
nslookup google.com - works

Adding the 2nd port forward for LAN2:

both fail

Grimson

@IanHK said in Lost all client DNS:

No, DNS on LAN (192.168.1.x) breaks when the port forward for LAN2 is added - that's what I find so bizarre.

Strange, did you copy the port forward rule from LAN to LAN_2 or did you create it manually?

LAN is 192.168.1.0/24 LAN2 is 192.168.2.0/25

Just out of interest, why /25 for LAN_2?

That was all the rules except for the LAN2 where a block ALL dropped off the bottom, sorry:

Some things here (though likely not related to the above issue). "WAN net" is not the Internet, so your LAN_2 will not be able to make outside connections. Additionally it is only allowed to talk to TCP port 443 of pfSense, if devices on LAN_2 are supposed to talk to other services, on pfSense (like DNS) you will also need to allow these or create a rule that allows full access to "LAN_2 address".

Also describe your network setup, are you using VLANs? What kind of AP and switch are you using?

@IanHK said in Lost all client DNS:

with just the LAN port forward:

nslookup google.com 8.8.8.8 - fails

That should match the port forward and still work. Something is royally messed up in your config.

IanHK

Thanks Grimson - I abandoned LAN_2 after the problems yesterday and today so getting that actually working properly is a job for another day(s) I fear.

For this first step - where basic LAN DNS routing seems to be susceptible to things it shouldn't care about at all, any pointers on where to look - it just seems so "default" I cant imagine where to look.

IanHK

LAN2 /25 is a result of trying to force a change in the settings yesterday when the issue was DHCP handing out a weird subnet mask - that was down to my typo.

I think I did copy the port forward and edit rather than create from new, but it put the auto-rule in the LAN_2 ruleset so it appeared to know what it was doing. I can try re-creating form scratch to see if anything different happens.

IanHK

So, I have deleted all the rules from LAN_2 and the PC is direct connected to the LAN interface on the pfSense box.

pfSense has been rebooted.

There is just the single DNS port 53 forward rule for LAN.

Even with this bare bones setup, the nslookup google.com 8.8.8.8 fails - which, I assume, means the port forward rule is not re-directing this external DNS request to 127.0.0.1 or it is but 127.0.0.1 doesn't recognize itself as a DNS server.

For the port forward itself, I have seen comments about choosing "Pure NAT" or "NAT+Proxy" Is this a possible issue.

The setting in System>General>disable dns forwarder is UNCHECKED
The setting in System>General>DNS server override is UNCHECKED
The setting in DNS resolver>DNS query forwarding is CHECKED
The setting in System>Advanced>FW&NAT>Nat Reflection mode is PURE NAT
The specific port forward rule nat reflection setting is DISABLED

For 127.0.0.1 not seeing itself as a DNS server

As [1] above, UNCHECKED is supposed to force the use of 127.0.0.1 as first DNS server....
...where the resolver is set to listen on localhost. DNS Resolver Network interfaces has "ALL" selected & outgoing interfaces is just "WAN".
There are some old posts about127.0.0.1 not being in a hosts file but that seems to be an old version issue.

Does any of this ring any bells at all ?

Thanks

Grimson

@IanHK
I would advice you to take a config backup, reset pfSense to defaults, then start adding back one thing after the other and check that nothing breaks. Make sure you only change settings that really need to be changed and where you know exactly what they do. If you find the point where it breaks come back.

@IanHK said in Lost all client DNS:

For the port forward itself, I have seen comments about choosing "Pure NAT" or "NAT+Proxy" Is this a possible issue.

The setting in System>Advanced>FW&NAT>Nat Reflection mode is PURE NAT

Do not use NAT reflection.

For port forwards on WAN use split DNS to reach them locally: https://docs.netgate.com/pfsense/en/latest/nat/accessing-port-forwards-from-local-networks.html#method-2-split-dns

For port forwards on LAN it could cause issues, but it's a crutch anyway so turn it off.

For 127.0.0.1 not seeing itself as a DNS server

Can you do a "nslookup google.com 127.0.0.1" from the pfSense console? Any errors in the DNS Resolver logs (Status -> System Logs -> DNS Resolver)?

Also I read the following from you in your previous topic:

The only non-standard setting may be the Firewall NAT Outbound is set to Manual Outbound NAT as recommended/required for pfBlockerNG setup.

Manual Outbound NAT is not needed or recommended for pfBlockerNG, it actually has no relation to it. But badly configured outbound NAT rules can cause all kinds of issues. So revert those changes, if you need an additional NAT rule for your VPN connection switch to Hybrid Outbound NAT and just add the mapping you require there.

Again, don't mess with settings where you don't understand what they are doing. And don't trust random "guides" or youtube videos, they often contain a lot of crap. Use the pfSense documentation : https://docs.netgate.com/pfsense/en/latest/, book: https://docs.netgate.com/pfsense/en/latest/book/ and the official pfSense hangouts: https://www.youtube.com/channel/UC3Cq2kjCWM8odzoIzftS04A/videos.

IanHK

when i nslookup google.com 8.8.8.8 the Status>System>DNS Resolver logs show:

Apr 5 21:01:19 unbound 71465:0 info: error sending query to auth server 9.9.9.9 port 853
Apr 5 21:01:19 unbound 71465:0 info: error sending query to auth server 9.9.9.9 port 853
Apr 5 21:01:19 unbound 71465:0 info: error sending query to auth server 9.9.9.9 port 853
Apr 5 21:01:19 unbound 71465:0 info: error sending query to auth server 9.9.9.9 port 853
Apr 5 21:01:19 unbound 71465:0 info: response for goog. DS IN
Apr 5 21:01:19 unbound 71465:0 info: reply from <.> 149.112.112.112#853
Apr 5 21:01:19 unbound 71465:0 info: query response was ANSWER
Apr 5 21:01:19 unbound 71465:0 info: validated DS goog. DS IN
Apr 5 21:01:19 unbound 71465:0 info: resolving goog. DNSKEY IN

where 9.9.9.9 & 149.112.112.112 are the DNS servers defined in System General Settings.

IanHK

Shell Output - nslookup google.com 127.0.0.1

Server: 127.0.0.1
Address: 127.0.0.1#53

Non-authoritative answer:
Name: google.com
Address: 74.125.130.102
Name: google.com
Address: 74.125.130.139
Name: google.com
Address: 74.125.130.100
Name: google.com
Address: 74.125.130.138
Name: google.com
Address: 74.125.130.101
Name: google.com
Address: 74.125.130.113
Name: google.com
Address: 2404:6800:4003:c02::8b

IanHK

Just to close this out:

The main problem was that:

From my location, one DNS Service 9.9.9.9 works about 5% of the time from my office, and never from my home. Oddly, their alternate 149.112.112.112 is very reliable, but slow.
Getting ahead of myself earlier, I already had SNORT running, and that was also blocking 1.1.1.1

So, rather than getting some redundancy by using 2 service providers, I was getting close to nothing.

I also had issues with the DNS resolver settings but finally got this sorted out, with everything running over 853 which was the objective there.

The only outstanding question I have is:

On main LAN, a PC (192.168.1.x) gets DNS server default IP of 192.168.1.1 via DHCP and an NSLOOKUP reports the server as "firewall.localdomain" as expected.
Replicating the setup on LAN 2, the PC (192.168.2.x) gets DNS server 192.168.2.1 via DHCP defaults but the NSLOOKUP reports the server as "unknown" - but everything seems to work - Admin WebGUI; servers on LAN; and Internet.
Changing the DHCP Server settings to hand out 192.168.1.1 as the DNS server for LAN2 resolves the issue and PCs on LAN2 show the DNS server name.

I thought the Interface IPs x.x.1.1 & x.x.2.1 as I have them, would behave the same - especially as the default action is to use these for each interface.

Is there an extra step needed for LAN2 to properly identify the DNS server?

Thanks