Completely confused by DNS failure (dnsmasq)

NickJH · Mar 7, 2024, 4:45 PM

Hi, newbe alert!

I am trying to test pfSense prior to deploying it as my gateway so it is currently run on my LAN and giving me strange DNS failures. Long term I will probably run a recursive DNS server, but for the moment I am running pfSense as a forwarder. My setup is something like:

Wan -- Firewall (172.17.2.254/24) ----- Sia2 (172.17.2.51)
                                    |
                                    |-- 172.17.2.5 -- pfSense -- 192.168.1.1 ----- PC Windows 11 (192.168.1.4)
                                            WAN                      LAN

pfSense's upstream resolver is configured as the Firewall. The firewall itself has host file entries for machines like Sia2 and all other machines on the Firewall LAN resolve Sia2 correctly. The PC behind pfSense fails to resolve and I can't understand it. In the PC I get:

C:\Users\Nick>nslookup sia2
Server:  pfsense.howitts.co.uk
Address:  192.168.1.1

*** No internal type for both IPv4 and IPv6 Addresses (A+AAAA) records available for sia2

I have tried packet sniffing. on the pfSense LAN I get:

[2.7.2-RELEASE][admin@pfSense.howitts.co.uk]/root: tcpdump -nni bge1 'port 53 and host 192.168.1.4'
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on bge1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:59:39.889613 IP 192.168.1.4.63610 > 192.168.1.1.53: 1+ PTR? 1.1.168.192.in-addr.arpa. (42)
15:59:39.889894 IP 192.168.1.1.53 > 192.168.1.4.63610: 1* 1/0/0 PTR pfsense.howitts.co.uk. (77)
15:59:39.894160 IP 192.168.1.4.63611 > 192.168.1.1.53: 2+ A? sia2.howitts.co.uk. (36)
15:59:39.895289 IP 192.168.1.1.53 > 192.168.1.4.63611: 2* 0/0/0 (36)
15:59:39.897595 IP 192.168.1.4.63612 > 192.168.1.1.53: 3+ AAAA? sia2.howitts.co.uk. (36)
15:59:39.898479 IP 192.168.1.1.53 > 192.168.1.4.63612: 3* 0/0/0 (36)
15:59:39.900778 IP 192.168.1.4.63613 > 192.168.1.1.53: 4+ A? sia2.co.uk. (28)
15:59:39.901035 IP 192.168.1.1.53 > 192.168.1.4.63613: 4 NXDomain 0/0/0 (28)
15:59:39.902729 IP 192.168.1.4.63614 > 192.168.1.1.53: 5+ AAAA? sia2.co.uk. (28)
15:59:39.902961 IP 192.168.1.1.53 > 192.168.1.4.63614: 5 NXDomain 0/0/0 (28)

And on the firewall LAN interface I get:

[root@gateway ~]# tcpdump -nni any 'port 53 and host 172.17.2.5'
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
15:59:39.899492 green0 In  IP 172.17.2.5.29507 > 172.17.2.254.53: 13494+ A? sia2.howitts.co.uk. (36)
15:59:39.899835 green0 Out IP 172.17.2.254.53 > 172.17.2.5.29507: 13494* 1/0/0 A 172.17.2.51 (52)
15:59:39.902902 green0 In  IP 172.17.2.5.53730 > 172.17.2.254.53: 52797+ AAAA? sia2.howitts.co.uk. (36)
15:59:39.903150 green0 Out IP 172.17.2.254.53 > 172.17.2.5.53730: 52797* 0/0/0 (36)

So I can see the Firewall is getting the request and returning the correct A record (I do not use IPv6), so I have to assume pfSense is receiving it. Bit in the pfSense LAN interface I see:

[2.7.2-RELEASE][admin@pfSense.howitts.co.uk]/root: tcpdump -nni bge1 'port 53 and host 192.168.1.4'
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on bge1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:59:39.889613 IP 192.168.1.4.63610 > 192.168.1.1.53: 1+ PTR? 1.1.168.192.in-addr.arpa. (42)
15:59:39.889894 IP 192.168.1.1.53 > 192.168.1.4.63610: 1* 1/0/0 PTR pfsense.howitts.co.uk. (77)
15:59:39.894160 IP 192.168.1.4.63611 > 192.168.1.1.53: 2+ A? sia2.howitts.co.uk. (36)
15:59:39.895289 IP 192.168.1.1.53 > 192.168.1.4.63611: 2* 0/0/0 (36)
15:59:39.897595 IP 192.168.1.4.63612 > 192.168.1.1.53: 3+ AAAA? sia2.howitts.co.uk. (36)
15:59:39.898479 IP 192.168.1.1.53 > 192.168.1.4.63612: 3* 0/0/0 (36)
15:59:39.900778 IP 192.168.1.4.63613 > 192.168.1.1.53: 4+ A? sia2.co.uk. (28)
15:59:39.901035 IP 192.168.1.1.53 > 192.168.1.4.63613: 4 NXDomain 0/0/0 (28)
15:59:39.902729 IP 192.168.1.4.63614 > 192.168.1.1.53: 5+ AAAA? sia2.co.uk. (28)
15:59:39.902961 IP 192.168.1.1.53 > 192.168.1.4.63614: 5 NXDomain 0/0/0 (28)

I also see the packets lookup and response packets at the pfSense WAN interface.

So something is going wrong but what? If it helps, both the Firewall and pfSense have been configured with the same domain. External DNS lookups from the PC work fine.

[edit]
I edited the post as I don't use IPv6. Originally I incorrectly said I don't use IPv4.
[/edit]

SteveITS · Mar 7, 2024, 7:26 PM

@NickJH if pfSense is not NATting then the upstream router needs a static route to know where to send replies back to 192.168.1.0/24.

NickJH · Mar 7, 2024, 8:21 PM

pfSense is natting. I didn't include the dump of the packets at the pfSense external interface but they match the packets coming back from the Firewall.

Also external resolution works in the PC.

SteveITS · Mar 7, 2024, 8:38 PM

@NickJH Then it should "just work." DNS Forwarder is really old though...you can set DNS Resolver to forward via a checkbox, though uncheck the DNSSEC checkbox if you do.

NickJH · Mar 7, 2024, 9:21 PM

That is the problem. It doesn't work and I have no idea why. It correctly gets a response back from the Firewall when querying external DNS.

I can't use a recursive resolver unless I replicate all the hosts that are in the Firewall for my split DNS. That would be a pain. For that reason I switched to a forwarder temporarily.

Alternatively, can I bulk load hosts into pfSense?

I am now just trying to switch the resolver to forwarding as you suggest.

johnpoz · Mar 7, 2024, 9:31 PM

@NickJH said in Completely confused by DNS failure (dnsmasq):

response packets at the pfSense WAN interface.

But what response - what should it respond with for sia2.howitts.co.uk?

I don't see A response.. A response would look like this.

    192.168.9.100.49907 > 192.168.9.253.53: 64979+ [1au] A? nas.home.arpa. (54)
    192.168.9.253.53 > 192.168.9.100.49907: 64979* 1/0/1 nas.home.arpa. A 192.168.9.10 (58)

This is not a valid response
15:59:39.895289 IP 192.168.1.1.53 > 192.168.1.4.63611: 2* 0/0/0 (36)

SteveITS · Mar 7, 2024, 9:31 PM

@NickJH The "Enable Forwarding Mode" forwards anything that isn't a host or domain override.

@NickJH said in Completely confused by DNS failure (dnsmasq):

can I bulk load hosts into pfSense?

I don't think so but you can put it in the config file and then restore just the DNS Resolver config.

@johnpoz "The [upstream] firewall itself has host file entries for machines like Sia2 "

johnpoz · Mar 7, 2024, 9:35 PM

@SteveITS said in Completely confused by DNS failure (dnsmasq):

@johnpoz "The [upstream] firewall itself has host file entries for machines like Sia2 "

It didn't send them, unless you edited the response.

edit: oh I see you did some more posts... Pretty sure sure dnsmasq also does rebind protection.. When you forward a rfc1918 response is not going to be returned to the client.. Unless you have turned off rebind or have setup a domain to be private and allowed to return rfc1918

https://docs.netgate.com/pfsense/en/latest/services/dns/rebinding.html

SteveITS · Mar 7, 2024, 9:35 PM

@johnpoz said in Completely confused by DNS failure (dnsmasq):

When you forward a rfc1918 response is not going to be returned to the client

Ah yes there it is.

NickJH · Mar 7, 2024, 9:55 PM

Yes, it could be rebind protection as I expect it to return a private IP (172.17.2.51). It is late here so I'll have a look in the morning.

NickJH · Mar 8, 2024, 1:06 PM

I can confirm it was rebind protection causing it. I have disabled it and am using the the DNS Resolver successfully in forwarder mode. Thanks.

johnpoz · Mar 8, 2024, 1:37 PM

@NickJH dig you disable it globally? I would suggest just setting your domain your forwarding for and want rfc1918 vs turning it completely off.

NickJH · Mar 8, 2024, 2:01 PM

@johnpoz Yes I did it globally. I don't know how to do it by domain only, but it does not matter as pfSense is on my LAN for testing/learning. When I deploy it properly, I'll be turning it back on. It is just that it was interfering with my testing.

SteveITS · Mar 8, 2024, 2:27 PM

@NickJH for reference it’s on that doc page:

“To exclude a domain from DNS rebinding protection, use the Custom Options box in the DNS resolver settings. Enter one domain per line in the following format, preceded by the server: line.

server:
private-domain: "example.com"
private-domain: "dnsbl.example"
“

I just forget about this “feature” because it’s rarely needed, but we had to discover/use it ourselves 10 years ago.

johnpoz · Mar 8, 2024, 2:48 PM

@NickJH how to do that was right in the link I posted..

For both unbound and dnsmasq

I take it you didn't read past the "This behavior is controlled by the DNS Rebind Check option under System > Advanced, Admin Access tab." part ;)

NickJH · Mar 8, 2024, 3:15 PM

@johnpoz All I needed was a quick and dirty fix because it is not going to be the production set up. I did the fix late yesterday but it was about 10pm and if the quick and dirty was going to fix it, it was good enough for me. I only tested it this morning.

cb831 · Feb 7, 2025, 5:51 PM

@johnpoz Thanks for this I have been through the same debugging process as @NickJH. The documentation for DNS forwarder says "To exclude a domain from DNS rebinding protection, use the DNS forwarder Advanced Settings box as follows:" - It should be "...Custom Settings".

SteveITS · Feb 7, 2025, 6:13 PM

@cb831 There's a "Give Feedback" link at the top of each doc page. It probably got renamed at some point.

johnpoz · Feb 7, 2025, 6:17 PM

@SteveITS yeah I would highly doubt there has been much work on the forwarder (dnsmasq) in quite some time to be honest. I am surprised that anyone would still be using it to be honest.. I mean it can do some things unbound can't like forward to multiple NS as the same time, etc.

But if you can't figure out that the custom options box is what they were talking about - not sure what to tell you ;)

Now if there was 2 boxes, one labeled advanced, and the other custom - and putting it in advanced didn't work because they called out the wrong box - yeah that could be problematic.. But there is only one possible place such commands could be put into that gui form.