Out of the box install, DNS broken (DNSSEC?)
-
Hello there! I just finished my APU2E0 build and flashed pfsense. All went well. Booted up, followed the setup wizard (left everything in default as far as I know) but no DNS resolution on clients. Followed the troubleshooting steps in https://docs.netgate.com/pfsense/en/latest/troubleshooting/connectivity.html down to the very last step (client cannot ping host by host name). Setup is simple for now: fiber -> ISP modem -> cat6 -> APU2E0 (pfsense) -> cat6 -> laptop.
The only single item that seems to help is if I disable DNSSEC. Then the client can resolve websites and everything is great. (I could totally run it this way...but...)
I have googled for a couple of days now learning all about DNS. Everything I've read says this should work out of the box with default settings. Eventually I would like to add a pihole which uses pfsense as the resolver with DNSSEC, so would like to get this working. Plus I'm interested in why it doesn't work. :)
Any pointers on what to try? I haven't really changed anything from defaults so it's hard to know where to begin...
-
And do you have a specific domain as an example..
There are not many domains that are dnssec enabled to be honest.. But if your saying dnssec is causing you issues with ALL domains.. You should pick 1 to troubleshoot with that you know is actually dnssec enabled..
google.com is not dnssec enabled for example.. Are you saying you can not resolve google.com? But if you turn off dnssec it works?
https://dnsviz.net/d/google.com/dnssec/Your resolving - you did not change unbound to forward right?
Here is example of domain that is
https://dnsviz.net/d/dnsviz.net/dnssec/Can you get the dnskeys via dig?
$ dig DNSKEY dnsviz.net ; <<>> DiG 9.16.9 <<>> DNSKEY dnsviz.net ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22407 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;dnsviz.net. IN DNSKEY ;; ANSWER SECTION: dnsviz.net. 84645 IN DNSKEY 256 3 8 AwEAAdhUrGVtXReHSI/C5Hfjz/PgQFq/QO5aAsBcmbCMkDFPkv4m/w9u S1t0h+XMiCVHo04TrxjOHn4VyaNMQ4mVQ srk6enGLs8YnnEPRdmmR8Bx JT4BNPYW3iIhBeMIjeo/L3njn6+xxdeub9gpgpBykh5kNEb6WwjAq6lN DBG4u2AeiA9iVXNyCV0a9pS0iwpHhRfb3k7fn2SGt6vdSqYwHmLKGV1x 7FjzmelF EzErzsYvo6JO6w4GXM6g2N3CfzO8EvAbqW0T/RKnLR3b40z4 ASVCKppSXaxJoCid2PjFJXR3THd/c/3inhx/BYbDC0fKMML7AMgl1LOE 1PYhY5uFKik= dnsviz.net. 84645 IN DNSKEY 257 3 8 AwEAAdlHDyfbmz04mAe5p9SXamkiWQPc9o7hvsmWq2nTppsHx48eVtez Mfq1EkPMfzFX1yKhahWPTbko7QtbSvqMz /+PM+vD41FbkN+wc2SONCFY BpJcaeP4iYT7kfJ/uH9BrkFs7ieIz32bLlNiEU0r1+WFRW0+njoNaF7F jKoNUFOI5S4WIg+lk2gHviSFGm0qp6+QrHWjFfPKzTqPe1QvM7efAjMI l3Tbi2BM UQ1En8FFODoa1Pbgxa0BoXpBWKbiuVzfO9lTRcCA/gs2KbUl 7MU9TZMVC5/Qq2ycaEv9cBNHDJrsJCjXk+qwVLsVXn8gS5ToP3QnbvLW q+cevLZZMj8= ;; Query time: 4 msec ;; SERVER: 192.168.3.10#53(192.168.3.10) ;; WHEN: Mon Dec 14 09:43:50 Central Standard Time 2020 ;; MSG SIZE rcvd: 591 -
Hi @johnpoz, I have read many of your excellent posts over the past couple of days going back years trying to find the answer on my own. Thanks for replying.
It has been any domain, www.google.com, www.tested.com, www.nytimes.com. I cannot resolve google.com until I turn off DNSSEC, then all websites resolve.
(Thanks to your posts) I believe I know the difference between resolve and forward. Unbound is resolving. The only ip address in the dns section of the dashboard is 127.0.0.1
This is on a windows laptop.
In order to run dig, I had to get my older linux laptop and connected in the same way as the windows machine. With settings unchanged, the above domains were resolving (!!!), so something looks to be different between clients...I may even try a different windows machine to see if it is that one client that is misbehaving...although...
When I ran "dig DNSKEY dnsviz.net" I got a SERVFAIL error on the status and no answer section.
What was interesting is going to https://en.internet.nl/connection/ and checking the DNSSEC validation section, I saw my ISP under the DNS provider. Is that right? I guess I would have expected to see something else if I am supposed to be querying a root server and resolving locally?
-
Well from your dig output your pointing to some local cache on the box your running.. Where does that point?
You know you can run dig on windows right? Just install tools only part of the bind package.
https://www.isc.org/download/
I would direct say to something specific like 8.8.8.8 example
; <<>> DiG 9.16.9 <<>> DNSKEY @8.8.8.8 dnsviz.net ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33855 ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;dnsviz.net. IN DNSKEY ;; ANSWER SECTION: dnsviz.net. 21599 IN DNSKEY 256 3 8 AwEAAdhUrGVtXReHSI/C5Hfjz/PgQFq/QO5aAsBcmbCMkDFPkv4m/w9u S1t0h+XMiCVHo04TrxjOHn4VyaNMQ4mVQsrk6enGLs8YnnEPRdmmR8Bx JT4BNPYW3iIhBeMIjeo/L3njn6+xxdeub9gpgpBykh5kNEb6WwjAq6lN DBG4u2AeiA9iVXNyCV0a9pS0iwpHhRfb3k7fn2SGt6vdSqYwHmLKGV1x 7FjzmelFEzErzsYvo6JO6w4GXM6g2N3CfzO8EvAbqW0T/RKnLR3b40z4 ASVCKppSXaxJoCid2PjFJXR3THd/c/3inhx/BYbDC0fKMML7AMgl1LOE 1PYhY5uFKik= dnsviz.net. 21599 IN DNSKEY 257 3 8 AwEAAdlHDyfbmz04mAe5p9SXamkiWQPc9o7hvsmWq2nTppsHx48eVtez Mfq1EkPMfzFX1yKhahWPTbko7QtbSvqMz/+PM+vD41FbkN+wc2SONCFY BpJcaeP4iYT7kfJ/uH9BrkFs7ieIz32bLlNiEU0r1+WFRW0+njoNaF7F jKoNUFOI5S4WIg+lk2gHviSFGm0qp6+QrHWjFfPKzTqPe1QvM7efAjMI l3Tbi2BMUQ1En8FFODoa1Pbgxa0BoXpBWKbiuVzfO9lTRcCA/gs2KbUl 7MU9TZMVC5/Qq2ycaEv9cBNHDJrsJCjXk+qwVLsVXn8gS5ToP3QnbvLW q+cevLZZMj8= ;; Query time: 192 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Mon Dec 14 13:52:51 Central Standard Time 2020 ;; MSG SIZE rcvd: 591Notice where it says my server in that query was 8.8.8.8
Out the box, pfsense should be resolving.. But its quite possible that your ISP is intercepting your dns traffic? Simple test of dns interception would be to just use dns leak test site... If your resolving.. the onlything being shown should be your IP..
There are other ways to test.. Doing a dns query to some public IP you know for sure doesn't provide dns and getting back an answer for sure would mean your dns is being intercept ;)
Doing a directed to query to authoritative NS, and getting back something that is non authoritative.. Ie if you query directly the ns for any domain, and its missing the aa in the dig results.. Then you were most likely intercepted..
But simple test is just any of the dnsleak sites - if you are resolving which is what pfsense should do out the box.. This should be YOUR Public IP address.. if anything else then something is going on with your dns..
-
Hi @johnpoz. Thanks for the information and additional commands to run.
I think trying the linux laptop added some confusion (to me!) I don't know why it points to 127.0.0.1 instead of 192.168.1.1 (pfsense). It looks like the linux laptop has a second OpenDNS fallback, which is why it was resolving websites - but when I query the router itself with
dig DNSKEY @192.168.1.1 dnsviz.netit gives a consistent output to the windows laptop (SERVFAIL). I will just try and diagnose the windows laptop from now on, which is what I used for the commands below.
Querying an authoritative NS seems to work (it would be most unlikely that my dns is being intercepted, but you never know...):
Getting a DNSKEY from 8.8.8.8 seems to also work:
It is when I attempt to use pfsense that it fails, it shows the correct server ip address (192.168.1.1)
I of course cannot get to dnsleaktest.com if it not resolving, but if I turn OFF DNSSEC, everything "works" as it should and I see only my public IP address and one server:
So I am back to everything works if DNSSEC is off, and everything is broken when DNSSEC is on.
Do you have any other diagnostic commands up your sleeve to try? :)
-
Check your system clock.
The #1 way I have seen that behavior is when the time is way off out of the box.
-
Looks like you are on to something!
I changed the time servers to 0.pool.ntp.org, 1.pool.ntp.org, 2.pool.ntp.org and 3.pool.ntp.org (and rebooted just for good measure) but it still seems to be having problems resolving the hostnames. Is this a chicken and egg situation? Is it trying to use the pfsense resolver to resolve the ntp hostnames, but can't because of the wrong time?
-
Your time is WAY off - says its March 4th ??
Put in an IP or 3 vs trying to resolve.. Or turn off dnssec until your time is correct
time-a-g.nist.gov 129.6.15.28 NIST, Gaithersburg, Maryland All services available time-b-g.nist.gov 129.6.15.29 NIST, Gaithersburg, Maryland All services available time-c-g.nist.gov 129.6.15.30 NIST, Gaithersburg, Maryland All services available time-d-g.nist.gov 129.6.15.27 NIST, Gaithersburg, Maryland All services available time-d-g.nist.gov 2610:20:6f15:15::27 NIST, Gaithersburg, Maryland All services via IPV6 time-e-g.nist.gov 129.6.15.26 NIST, Gaithersburg, Maryland All services available time-e-g.nist.gov 2610:20:6f15:15::26 NIST, Gaithersburg, Maryland All services via IPv6 time-a-wwv.nist.gov 132.163.97.1 WWV, Fort Collins, Colorado All services available time-b-wwv.nist.gov 132.163.97.2 WWV, Fort Collins, Colorado All services available time-c-wwv.nist.gov 132.163.97.3 WWV, Fort Collins, Colorado All services available time-d-wwv.nist.gov 132.163.97.4 WWV, Fort Collins, Colorado All services available time-d-wwv.nist.gov 2610:20:6f97:97::4 WWV, Fort Collins, Colorado All services via IPv6 time-e-wwv.nist.gov 132.163.97.6 WWV, Fort Collins, Colorado All services available time-e-wwv.nist.gov 2610:20:6f97:97::6 WWV, Fort Collins, Colorado new server, services via IPV6 time-a-b.nist.gov 132.163.96.1 NIST, Boulder, Colorado All services available time-b-b.nist.gov 132.163.96.2 NIST, Boulder, Colorado All services available time-c-b.nist.gov 132.163.96.3 NIST, Boulder, Colorado All services available time-d-b.nist.gov 132.163.96.4 NIST, Boulder, Colorado All services available time-d-b.nist.gov 2610:20:6f96:96::4 NIST, Boulder, Colorado All services available time-e-b.nist.gov 132.163.96.6 NIST, Boulder Colorado All services available time-e-b.nist.gov 2610:20:6f96:96::6 NIST, Boulder, Colorado All services available time.nist.gov global address for all servers Multiple locations All services available utcnist.colorado.edu 128.138.140.44 University of Colorado, Boulder All services available utcnist2.colorado.edu 128.138.141.172 University of Colorado, Boulder All services availableAn intelligent man is sometimes forced to be drunk to spend time with his fools
If you get confused: Listen to the Music Play
Please don't Chat/PM me for help, unless mod related
SG-4860 24.11 | Lab VMs 2.7.2, 24.11 -
Yep, that would do it.
Disable DNSSEC, restart NTP, wait a bit and see if the clock is correct, then enable DNSSEC again.
You could also kill ntpd and run something like
ntpdate x.x.x.xwherex.x.x.xis the IP address of a known good NTP server. Once it skews the clock back to current you can restart unbound and ntpd. -
Is is not a time issue, but you proving that resolving (on pfSense !) doesn't work :
So 'NTP' can't do it's work, which is ........ set the clock straight.
Apply the golden rule : when you install a system - what ever the system is : check the clock.
Never take the state of the on battery for granted ; Batteries dies. They are designed to do so.This :
is your"'case closed" ;)
( except if the screen shot was taken on March, 4 .... )In the BIOS, set the clock, and do a clean reboot. Go back in the BIOS right away, and check the clock. Change batteries if needed.
As soon as the (initial - hardware) time is ok, pfSense => Resolver => DNSSEC will behave correctly.
And NTP can do it's job.edit : jimp nailed it.
No "help me" PM's please. Use the forum, the community will thank you.
Edit : and where are the logs ?? -
@gertjan said in Out of the box install, DNS broken (DNSSEC?):
Is is not a time issue, but you proving that resolving (on pfSense !) doesn't work.
It is a time issue. DNSSEC fails because it can't validate DNSSEC due to the time being so far off.
In the BIOS, set the clock, and do a clean reboot. Go back in the BIOS right away, and check the clock. Change batteries if needed.
As soon as the (initial - hardware) time is ok, pfSense => Resolver => DNSSEC will behave correctly.
And NTP can do it's job.You could also set the clock to a close value by hand from a command prompt on the firewall:
date [[[[[cc]yy]mm]dd]HH]MM[.ss]]so for 2020, Dec 15th at 09:38 and 30 seconds:
date 202012150938.30Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!
Need help fast? Netgate Global Support!
Do not Chat/PM for help!
-
@gertjan said in Out of the box install, DNS broken (DNSSEC?):
DNSSEC will behave correctly. (sorry :-))
this is the solution...
hhmm considering these:
https://dnssec.vs.uni-due.de/I don't even understand what their problem is to others with the clock... easy...
so in this world everything is determined by this, just think of (stock market, banks, IT) -
@jimp said in Out of the box install, DNS broken (DNSSEC?):
Is is not a time issue,
Conflict between my fingers and the head.
I guess I wanted to start my phrase with "If it is not the time ....." and got de synced while capturing.
Wanted to show with the first image - the NTP server (pool) host name not resolving, and the second image, the date/hour completely wrong, that something is wrong. -
@gertjan said in Out of the box install, DNS broken (DNSSEC?):
Conflict between my fingers and the head.
I don't deny it's right
+++edit:
let me tell you what is the difference, but decide above what will be below
-
Fixed! After all that it was simply the system time that was out (really out). I stopped the NTP service, disabled DNSSEC, updated the time with the command line "ntpdate <random server>", restarted the NTP service, turned back on DNSSEC and everything is working as it should. So happy!
I'm glad I stuck with it, and thank you so much for the help. I learned a lot about DNS along the way, and some new commands to make sure it is all working correctly. Thanks for the help, much appreciated.
-
@madcatinc Glad you got it sorted. That was as great catch by @jimp on the time being an issue with dnssec..
I normally just assume that time would be correct ;) I mean who doesn't make sure their time is correct? ;)
Would of prob taken quite a bit longer to find your issue if jim hadn't chimed in..
