DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times
-
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Was the pfSense switched of during April 28, 09h23 and April 29, 14h00 ?
umm not at 2pm on Monday April 29th but I do believe that I reset the pfsense from the GUI on Sunday Apr 28 in the morning. I didn't think this was this instance but I know that I tried to reboot from the GUI before and it just wouldn't reboot (waited 10 minutes or so) so i pushed the power button (I know I'm not supposed to, but i wasn't sure what else to do). I can say pretty confidently that it wasn't, at least purposely, turned off at 2pm on Monday. That time seems awfully specific as well (i.e. 14:00:01) like some kind of schedule?
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Disable level 3+ resolver (unbound) logging as soon as possible.
Yes i turned on debugging to try to troubleshoot it, i understand to change it back asap, but I need to identify this problem first....thank you for pointing that out though...
-
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Up until April 28, 09h23 ..... and then it stops - nothing anymore.
Some shut the device down ? (power switch ? that's very bad)looking at your paste though...it def wasn't down from sunday at 9 to monday at 2pm...? It was down for the amt of time it takes to reboot. that is perplexing?
-
@RickyBaker I think i misunderstood, apologies. I had another weird internet event last night at 17:18 in the evening and when i went to go paste the logs I discovered what you were alluding too. the DNS Resolver log seems to have stopped updating yesterday at 14;00. what gives? I didn't discover til this morning the "restart log" button so i tried to change the log level to 2 as a bootleg way to "restart" it. Well the DNS NX DOMAIN event happened again on mutliple devices between 6:09 and 6:15 but I couldn't get to a computer til 6:42 and the DNS Resolver log set to 2000 entries didn't go past 6:42. So my question is which log level is appropriate to troubleshoot this? Any other logs I should change the logging level on? This issueis becoming very problematic.
I've also added about 6 IP address to the blacklist of various LANs, waiting to see what, if anything, breaks. All the mac addresses were "no vendor" results on a mac address lookup, anything to look into that?
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
All the mac addresses were "no vendor" results on a mac address lookup
If I were to guess - those would be mobile devices, apple or android - they love to use made up mac address - you know for your privacy ;) You can turn it off on the device.. So it uses its actual mac
-
As probably already said above (I didn't check) : you don't want unbound to get restarted every xx seconds (minutes).
So : uncheck this one :
From now on, you should see very few :
Maybe once a day ?
And remember : under pfBlockerng control, unbound can also get restarted.
To see unbound (DNS) activity, I use this :
tail -f /var/unbound/var/log/pfblockerng/dns_reply.logas I have pfBlocker already running.
You can set unbound logging back to "Level 1 basic operations".What you also can try is : use the unbound settings as pre initialized by Netgate.
De activate forwarding.
Ditch 8.8.8.8 8 etc.
You'll be using the default resolving.This is what I'm using :
and is rock solid for close to a decade.
Don't worry about 8.8.8.8 etc, they will get over it ;)No "help me" PM's please. Use the forum, the community will thank you.
Edit : and where are the logs ?? -
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Don't worry about 8.8.8.8 etc, they will get over it ;)
hahaha - made me laugh.. Oh man they are going to wonder why Ricky stopped asking for dns..
-
First, thanks for all the screenshots and suggestions, really appreciate the time, i'm going nuts.
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
So : uncheck this one :
Since @johnpoz first message i've had this unenabled in the DNS Resolver settings menu:
are you still seeing this constant restarting in the current log? Is there somewhere else i can disable or does a similar setting live elsewhere in the menu?@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
And remember : under pfBlockerng control, unbound can also get restarted.
I still don't think i have pfblockerng installed. I believe its an installed package that would show up in the Firewall drop down:
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
To see unbound (DNS) activity, I use this :
I don't seem to have a var folder under /var/unbound/ Is this an issue with being on 2.7.0 and not 2.7.2? i would love to try to update but I'm running into a roadblock there as well:
After posting this I will go back to this thread and see if there's any other suggestions to try besides what I already have: https://forum.netgate.com/topic/184670/issue-with-going-from-2-7-0-to-2-7-2/9
edit: there didn't appear to be anything more I hadn't tried besides installing 2.7.2 from scratch and restoring the backup, which i'm not currently prepared to do@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
This is what I'm using :
Differences I see are:-
I have All selected under Outgoing Network Interface, I've changed that to LocalHost to match yours but i don't really understand this setting
-
I have System Domain Local Zone Type set to Transparent and not static like you. I don't understand this option and can switch it if the above doesn't fix anything
-
I disabled DNSSEC per @johnpoz suggestion in the first post
-
I don't have Python Module enabled (should I?)
-
i have a different SSL cert but I assume that's just a personal one you created
I also don't understand what the domain overrides at the end are, should I trash them?
Again thanks for everything. anything to attempt is appreciated
-
-
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Don't worry about 8.8.8.8 etc, they will get over it ;)
sorry missed this part:
I removed those 4 other dns after that first post as well, are they also still showing up in the logs? -
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
I have All selected under Outgoing Network Interface, I've changed that to LocalHost to match yours but i don't really understand this settingAll is default (I guess). The local DNS guru explained me somewhere that Localhost was 'better'.
I have System Domain Local Zone Type set to Transparent and not static like you. I don't understand this option and can switch it if the above doesn't fix anythingI can read this (as an explanation). You probably have to translate. I've chosen 'static' for my use, but there is really no important differences with transparent.
I disabled DNSSEC per @johnpoz suggestion in the first postDNSSEC can be sued without issues if you do not Forward. You don't, so you an use it.
Disabling it won't hurt, though.I don't have Python Module enabled (should I?)You can. DNSSEC, if activated, uses Python. And pfBlockerng - but you don't use pfBlockerng .
It's activated by default.i have a different SSL cert but I assume that's just a personal one you createdPick one you've listed. Certs are not used by default.
I also don't understand what the domain overrides at the end are, should I trash them?
These are added by the pfSense admin = you. Not Netgate.
But I know why these (your) domains are listed : it will disable registration checking of some known adobe (photoshop, to name it) software ^^
If you don't have the cracked Photoshop installed on one of your PCs, you can remove them all. -
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
You can. DNSSEC, if activated, uses Python. And pfBlockerng - but you don't use pfBlockerng .
It's activated by default.any suggestions on what options to select for the Python Module after enabling? If I understood you correctly you were suggesting that it's necessary IF i enable DNSSEC support? But that DNSSEC isn't necessary?
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Pick one you've listed. Certs are not used by default.
I picked my name, which was a user I created for OpenVPN i think. I assumed you were implying that picking "Webconfigurator Default" was suboptimal/wrong?
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
any suggestions
The resolver can do DNSSEC for you. It's not mandatory.
But If you know what DNSSEC is, you will activate DNSSEC. It will not secure every DNS request, as most domain names are not signed yet. But when they are, why not securing the request ?@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
I picked my name
TLS can be used by unbound to secure the TLS channels, like the control chancel, commonly on 853. Any cert will do.
-
The cert means nothing - unless you want unbound to be a dot server to your clients. Forget about the cert that is listed. It matters when you want to answer dot queries, not when you want to make them.
-
I've enabled DNSSEC again since it didn't really help my issue having it disabled. Is there another log I should increase the logging on? What level should I have the dns resolver on? I'm still experiencing the issue, yesterday for about 20 minutes (longest yet) I couldn't' open a webpage on my phone, but I was concurrently streaming the Knicks game on Youtube TV.
I've also added all those "no vendor" Mac address's I couldn't explain with randomly assigned DHCP leases to the whitelist block list and they've yet to come back and I've yet to discover anything broken. Just an update.
Any advice on getting it updated to 2.7.2 without doing a full clean install and restore?
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
I've enabled DNSSEC again since it didn't really help my issue having it disabled
if you are forwarding dnssec should be disabled.. While it might not seem like no issues with a few queries, but can tell you its going to be problematic at some point.. even quad9 faq for when forwarding says to disable it. It is pointless if you forward, where you forward does dnssec or they don't you telling unbound to do it isn't going to do anything other than cause you issues at some point.
-
@johnpoz I'm no longer forwarding per your first post in this thread.
-
@RickyBaker Long thread and haven't paid close attention, could tell if you had switched back to forwarding or not. Yeah if your resolving then dnssec is good to have enabled.
-
@johnpoz great thanks for circling back
-
https://pastebin.com/SFR8BXb0
Woke up from a nap and experienced one of the longest internet outages of this whole saga. It was out at 3:14 when I tried to open venmo and was out for over 20 minutes before it came back. the above is the DNS resolver log but I think i have the log level dialed too high cause 2000 entries didn't even go back 2 minutes. I've changed it back to Log Level 1 but could someone check it out and see if there's any clues in there (or what log level I should have it at)? Or is there another log that I should also be monitoring? Is it possible the problem is purely something with the wifi and Ubiquiti?
-
Same thoughts here : a high level of log details actually the details your looking for, as there is only for 2 minutes worth of info.
If you have some disk space left, you can make the log files bigger.
If needed, you can make the log retention a bit smaller - I've "7", you can make it 5 or 4.
You can also make this, one
a bit bigger.
The actual goal is :
As soon as you find a situation where a device has no access anymore, you have to check :
Does the access without using DNS works ? For example, ping 8.8.8.8 from that device.
Also double check : does the device has a valid IP, gateway and dns set at that moment ?
Example :ipconfig /alland check the duration of the lease, the gateway, the DNS (both should point to the IP of pfSense.
Check on the device if "DNS" works :
C:\Users\Gauche>nslookup www.google.com Serveur : pfSense.bhf.tld Address: 2a01:cb19:907:bedf:92ec:77ff:fe29:392c Réponse ne faisant pas autorité : Nom : www.google.com Addresses: 2a00:1450:4007:81a::2004 142.250.201.164Take note : for me, both IPv6 and IPv4 works.
Then (also) check on pfSense if resolving works :
dig @127.0.0.1 www.google.com +shortand then
dig @192.168.1.1 www.google.com +shortwhere 192.168.1.1 is your LAN interface.
Check if unbound is up and running :
[24.03-RELEASE][root@pfSense.bhf.tld]/root: ps ax | grep 'unbound' 74113 - Ss 4:32.60 /usr/local/sbin/unbound -c /var/unbound/unbound.conf .... ....and
[24.03-RELEASE][root@pfSense.bhf.tld]/root: sockstat | grep 'unbound' unbound unbound 74113 3 udp6 *:53 *:* unbound unbound 74113 4 tcp6 *:53 *:* unbound unbound 74113 5 udp4 *:53 *:* unbound unbound 74113 6 tcp4 *:53 *:* unbound unbound 74113 9 tcp4 127.0.0.1:953 *:* ... ... ... ...With the unbound log details set to "1", it will still contains the number of restarts (a controlled stop and then a start :
grep "stopped" /var/log/resolver.log ..... <30>1 2024-05-06T00:15:24.852356+02:00 pfSense.bhf.tld unbound 12814 - - [12814:0] info: service stopped (unbound 1.19.3).Btw : the actual unbound version is 1.19.3 as I'm using 24.03.
pfSense 2.8.0 will be coming out soon.
Not that the version really matters (imho) as I was using 1.17.x also a long time, and don't recall having any issues.@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Is it possible the problem is purely something with the wifi and Ubiquiti?
For me, an AP should be what it should do :; being a radio to wire signal converter.
True, an AP can do a lot more, and really braking the connection for you.
When testing connectivity issues, add APs and other gadgets later on, when you know the wired connection works well.
The same thing goes for L3 'smart, VLAN based' switches : only use the when the bare bone network works well. -
@RickyBaker did you actually do some dns queries while you were having the issue, to both unbound and say external dns?
You should log queries and replies as well if your wanting to troubleshoot dns not working.
int your options box in unbound
server: log-queries: yes log-replies: yeswhat is the response, timeout talking to unbound, servfail, nx?
fire up your fav dns tool, nslookup, dig, doggo, host, etc. and actual validate what is failing.. If you look some fqdn do you get a response. if so what is the response, did it work, did unbound return servfail, or nx domain ?
Does unbound answer local resources, like pfsense fqdn? Does something that is cached work, only new queries fail. You can view what is in your cache
[23.09.1-RELEASE][admin@sg4860.home.arpa]/root: unbound-control -c /var/unbound/unbound.conf dump_cache | grep forum.netgate.com forum.netgate.com. 2452 IN A 208.123.73.71 msg forum.netgate.com. IN A 32896 1 2452 3 1 1 3 6 forum.netgate.com. IN A 0 [23.09.1-RELEASE][admin@sg4860.home.arpa]/root:If that fails, then do a query directed to some external NS like quad9 or google - do those work?
