DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times

RickyBaker · Apr 19, 2024, 3:05 PM

@johnpoz said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

@RickyBaker unbound is a resolver.. Ie if you ask it for www.somedomain.tld it will ask the root servers, hey what is the NS (nameserver) for .tld, then ask hey NSers of .tld what is the ns for somedomain.tld, then hey ns for somedomain.tld what is the IP address of www.somedomain.tld

thank you! that's a very good explanation. so it's kind of the structural umbrella above the DNS server...and it seems like it includes DNS servers? so it's really overkill to include google/opendns ones?

@johnpoz said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

As to register dhcp, no that restarts unbound every single time there is a dhcp event, client gets an IP, client renews an IP, etc.. Look in your log is unbound restarting - look for next time you have an issue - was unbound restarting, or restarted exactly when you had your dns problem?

ok great, this is a good troubleshooting action plan. I did remove the 4 extra DNS servers from General Setup and still had a "No internet" event this morning that lasted 2-3 minutes. I subsequently restarted my Unifi Switches, just cause...will dig into the log now

@johnpoz said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

If you register static dhcp, then this only restarts unbound when you add a new record for static dhcp.. But with register dhcp every single dhcp event will restart unbound.

yeah this seems like an excessive amt of work, i'll disable for now and see what breaks. What type of use-case is the Register DHCP option typically used for? What benefit am I giving up by disabling it?

@johnpoz said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

If you only have a few clients, and a decent length lease time so there are not many events then you may never notice it.

I have a TON of clients and a TON of static DHCP addresses. I use default lease time, never considered changing that but maybe I should..

@johnpoz said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

Your browser throwing up an error about dns, could be maybe dns was having issues resolving what you asked for... You could do an above example +trace on pfsense to see if maybe your just having a problem talking to a NS to find what your looking for..

another good suggestion. I've actually never heard of dig before. I'll have to research how to use it but your example should help, what type of shell is that being executed in?

SteveITS · Apr 19, 2024, 3:26 PM

@RickyBaker Dig is a program similar to nslookup, distributed with BIND, on Windows as part of the "tools" install choice, and could be installed by itself. However BIND on Windows is EOL so not sure if you can get it from isc.org anymore.

It is much more common on Linux/BSD.

use-case is the Register DHCP option

To access PCs by full hostname, however, often PCs can discover other PCs on the network.

johnpoz · Apr 19, 2024, 4:22 PM

@SteveITS you can still get dig for windows... Its just a bit behind what is current version.. And yeah they are not going to provide it past the 9.16 branch.

I use it all the time, but you can install on wsl on windows so there is that which is on the 9.18 branch and underdevelopment

user@i9-win:~$ dig

; <<>> DiG 9.18.26-1+ubuntu22.04.1+deb.sury.org+1-Ubuntu <<>>

Latest for windows is 9.16.50

$ dig

; <<>> DiG 9.16.50 <<>>

side note, how to say you don't know anything about dns without stating you don't know anything about dns..

"I've actually never heard of dig before." hahahah

RickyBaker · Apr 22, 2024, 7:51 PM

@johnpoz said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

side note, how to say you don't know anything about dns without stating you don't know anything about dns..

"I've actually never heard of dig before." hahahah

o i never claimed to know anything about DNS. or much of anything for that matter. I'll be honest, a bit out of my depth with dig. Downloaded the zip from isc.org and installed it but i'm really not sure how to execute it.

I'm likewise a bit out of my depth with the logs. I've navigated to the DHCP subsection of the systemlogs and see the restarts you were referring to from before I disabled "Register DHCP" but nothing alarming just after it. But i have no idea what to be looking for. Can I upload the log for more expert eyes to see if anything jumps out?

johnpoz · Apr 22, 2024, 8:06 PM

@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

but i'm really not sure how to execute it.

type dig ;) at a command prompt

$ dig -h
Usage:  dig [@global-server] [domain] [q-type] [q-class] {q-opt}
            {global-d-opt} host [@local-server] {local-d-opt}
            [ host [@local-server] {local-d-opt} [...]]
Where:  domain    is in the Domain Name System
        q-class  is one of (in,hs,ch,...) [default: in]
        q-type   is one of (a,any,mx,ns,soa,hinfo,axfr,txt,...) [default:a]
                 (Use ixfr=version for type ixfr)
        q-opt    is one of:
                 -4                  (use IPv4 query transport only)
                 -6                  (use IPv6 query transport only)
                 -b address[#port]   (bind to source address/port)
                 -c class            (specify query class)
                 -f filename         (batch mode)
                 -k keyfile          (specify tsig key file)
                 -m                  (enable memory usage debugging)
                 -p port             (specify port number)
                 -q name             (specify query name)
                 -r                  (do not read ~/.digrc)
                 -t type             (specify query type)
                 -u                  (display times in usec instead of msec)
                 -x dot-notation     (shortcut for reverse lookups)
                 -y [hmac:]name:key  (specify named base64 tsig key)
        d-opt    is of the form +keyword[=value], where keyword is:
                 +[no]aaflag         (Set AA flag in query (+[no]aaflag))
                 +[no]aaonly         (Set AA flag in query (+[no]aaflag))
                 +[no]additional     (Control display of additional section)
                 +[no]adflag         (Set AD flag in query (default on))
                 +[no]all            (Set or clear all display flags)
                 +[no]answer         (Control display of answer section)
                 +[no]authority      (Control display of authority section)
                 +[no]badcookie      (Retry BADCOOKIE responses)
                 +[no]besteffort     (Try to parse even illegal messages)
                 +bufsize[=###]      (Set EDNS0 Max UDP packet size)
                 +[no]cdflag         (Set checking disabled flag in query)
                 +[no]class          (Control display of class in records)
                 +[no]cmd            (Control display of command line -
                                      global option)
                 +[no]comments       (Control display of packet header
                                      and section name comments)
                 +[no]cookie         (Add a COOKIE option to the request)
                 +[no]crypto         (Control display of cryptographic
                                      fields in records)
                 +[no]defname        (Use search list (+[no]search))
                 +[no]dnssec         (Request DNSSEC records)
                 +domain=###         (Set default domainname)
                 +[no]dscp[=###]     (Set the DSCP value to ### [0..63])
                 +[no]edns[=###]     (Set EDNS version) [0]
                 +ednsflags=###      (Set EDNS flag bits)
                 +[no]ednsnegotiation (Set EDNS version negotiation)
                 +ednsopt=###[:value] (Send specified EDNS option)
                 +noednsopt          (Clear list of +ednsopt options)
                 +[no]expandaaaa     (Expand AAAA records)
                 +[no]expire         (Request time to expire)
                 +[no]fail           (Don't try next server on SERVFAIL)
                 +[no]header-only    (Send query without a question section)
                 +[no]identify       (ID responders in short answers)
                 +[no]ignore         (Don't revert to TCP for TC responses.)
                 +[no]keepalive      (Request EDNS TCP keepalive)
                 +[no]keepopen       (Keep the TCP socket open between queries)
                 +[no]mapped         (Allow mapped IPv4 over IPv6)
                 +[no]multiline      (Print records in an expanded format)
                 +ndots=###          (Set search NDOTS value)
                 +[no]nsid           (Request Name Server ID)
                 +[no]nssearch       (Search all authoritative nameservers)
                 +[no]onesoa         (AXFR prints only one soa record)
                 +[no]opcode=###     (Set the opcode of the request)
                 +padding=###        (Set padding block size [0])
                 +[no]qr             (Print question before sending)
                 +[no]question       (Control display of question section)
                 +[no]raflag         (Set RA flag in query (+[no]raflag))
                 +[no]rdflag         (Recursive mode (+[no]recurse))
                 +[no]recurse        (Recursive mode (+[no]rdflag))
                 +retry=###          (Set number of UDP retries) [2]
                 +[no]rrcomments     (Control display of per-record comments)
                 +[no]search         (Set whether to use searchlist)
                 +[no]short          (Display nothing except short
                                      form of answers - global option)
                 +[no]showsearch     (Search with intermediate results)
                 +[no]split=##       (Split hex/base64 fields into chunks)
                 +[no]stats          (Control display of statistics)
                 +subnet=addr        (Set edns-client-subnet option)
                 +[no]tcflag         (Set TC flag in query (+[no]tcflag))
                 +[no]tcp            (TCP mode (+[no]vc))
                 +timeout=###        (Set query timeout) [5]
                 +[no]trace          (Trace delegation down from root [+dnssec])
                 +tries=###          (Set number of UDP attempts) [3]
                 +[no]ttlid          (Control display of ttls in records)
                 +[no]ttlunits       (Display TTLs in human-readable units)
                 +[no]unexpected     (Print replies from unexpected sources
                                      default=off)
                 +[no]unknownformat  (Print RDATA in RFC 3597 "unknown" format)
                 +[no]vc             (TCP mode (+[no]tcp))
                 +[no]yaml           (Present the results as YAML)
                 +[no]zflag          (Set Z flag in query)
        global d-opts and servers (before host name) affect all queries.
        local d-opts and servers (after host name) affect only that lookup.
        -h                           (print help and exit)
        -v                           (print version and exit)

Sure you can upload stuff.. There are some file size limits and such

RickyBaker · Apr 23, 2024, 5:26 PM

@johnpoz said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

type dig ;) at a command prompt

lol yeah obviously not in the windows cmd prompt i was trying it in. I thought installing the package from isc.org would install the command into the PATH but i guess not. I will try on a linux machine.

as for the logs, took me a while but here's some logs
dhcp: https://pastebin.com/3RuMUSc2
resolver: https://pastebin.com/3GFsYCE7 (not sure why this stopped updating on saturday, probably from one of the changes i instituted)
system: https://pastebin.com/1c6PwRu2

let me know if any others would be useful. fwiw my wife experienced the error around 8:24am today (4.23.24

RickyBaker · Apr 23, 2024, 5:43 PM

@johnpoz said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

type dig ;) at a command prompt

well i'll be...

🔒 Log in to view

tried to paste as code like you but kept identifying as spam. Running the same command while ssh'ed into the pfsense box though:

🔒 Log in to view

SteveITS · Apr 23, 2024, 5:56 PM

@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

resolver: https://pastebin.com/3GFsYCE7

Unbound restarted 32 times in the ~1.5 days of log you posted. Looks like your DHCP is set for a 2 hour lease (meaning 1 hour renewal). You could try extending that to say 8 or 12 hours.

I don't see whether you posted you tried with DNSSEC disabled. Are you actually forwarding or not?

RickyBaker · Apr 23, 2024, 7:30 PM

@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

Looks like your DHCP is set for a 2 hour lease (meaning 1 hour renewal). You could try extending that to say 8 or 12 hours.

thanks, i'll look for that option and change it to 8

@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

I don't see whether you posted you tried with DNSSEC disabled. Are you actually forwarding or not?

🔒 Log in to view
is this option not showing up unticked in the logs? or is this screenshot suffice to say it's disabled.
I don't think I'm forwarding?
🔒 Log in to view

SteveITS · Apr 23, 2024, 7:43 PM

@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

or is this screenshot suffice to say it's disabled.

That's fine, I just didn't see a response above. Before 23.01 forwarding + DNSSEC didn't seem to be a problem but after 23.01 it often is.

There's a checkbox in the resolver settings to forward but if DNSSEC is disabled that point is irrelevant.

RickyBaker · Apr 24, 2024, 1:05 PM

@SteveITS Happened again this morning. Though just for my wife. She got DNS_PROBE_FINISHED_NXDOMAIN error in chrome but said the amazon link worked (so weird) but I did not experience any issues by the time i got a browser opened. This is the log for around that time:

🔒 Log in to view

I did notice that attack from 180.101.88.225 with a Level 10 a LOT in the logs, is it possible my firewall has misidentified some of my devices as attackers? But then quickly resolves that's wrong before making the same mistake again soon?

Gertjan · Apr 24, 2024, 1:08 PM

@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

180.101.88.225 with a Level 10 a LOT in the logs, is it possible my firewall has misidentified some of my devices as attackers?

misidentified ?
It's 180.101.88.225. No doubt about it.
It that IP coming from your LAN ? Disconnect it, have it cleaned. Do have a talk with the owner.
Is the IP coming from the Internet ? Empty the WAN firewall rule list, and you're good. I would fire the pfSense admin

edit : no LAN IP, whois told me "180.101.88.225" is Chinese allocated.
Rip out your WAN cable now. We'll talk later ^^

SteveITS · Apr 24, 2024, 1:13 PM

@RickyBaker the “now monitoring attacks” is logged whenever a log file rotates so is normal.

The logged attacks though do indicate you have port 22 open on WAN, and check for other ports too, as that’s a good way to get hacked.

RickyBaker · Apr 24, 2024, 2:18 PM

@SteveITS so close port 22 immediately?

SteveITS · Apr 24, 2024, 2:22 PM

@RickyBaker I'd close all ports on WAN that are not needed. By default WAN has no rules so all incoming traffic from the Internet is blocked.

RickyBaker · Apr 24, 2024, 2:25 PM

@SteveITS I certainly did not intentionally leave any ports open...Am firing up my VPN now...These logs are saying this IP is TRYING to access my network, not accessing it though right?

Gertjan · Apr 24, 2024, 2:26 PM

@RickyBaker
Not "closing".
Don't use any firewall rules that allow SSH access (port 22) or actually any access on the WAN interface.
Exactly like the way you found it, when installing pfSense.
No rules on WAN = safe.
Opening port 22 = "China" (the entire planet in reality) is lining up for you to 'try'.

There is an exception (as always) :
If you activate a VPN server (on pfSense), this will, by default, allow UDP traffic on port 1194 on the WAN interface.
If you need to access resources from the outside = WAN, use a VPN, or comparable.

RickyBaker · Apr 24, 2024, 2:27 PM

@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:

There is an exception (as always) :
If you activate a VPN server (on pfSense), this will, by default, allow UDP traffic on port 1194 on the WAN interface.
If you need to access resources from the outside = WAN, use a VPN, or comparable.

I do! there's always the concern i messed up during set up, but that was the intention. checking now

RickyBaker · Apr 24, 2024, 2:31 PM

@Gertjan 🔒 Log in to view

f#$k this looks wide open. i dunno how that happened it says it comes from OpenVPN wizard. Is this wrong? Should Destination port be changed from asterix to 1194?

RickyBaker · Apr 24, 2024, 2:35 PM

@RickyBaker 🔒 Log in to view
also, DEF didn't put in this rule, that ip address is my Synology box, think it added this rule via UPNP?