DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
per @johnpoz suggestion i have unchecked "Register DHCP", should I re-enable for testing purposes?
Certainly not ;) Keep it of.
Your DHCP log image above show about 10 DHCP request/renewals in let then (42-26)=16 minutes.
That means 10 unbound restart in 16 minutes ...
Every restart takes ... 30 seconds ? So during this 16 minutes your DNS is 'out' for 5 minutes.
That's not good at all.And before you start to think : isn't that totally flawed ?
Yes, it is. But help is coming - see here what cmcdonald said this morning.
( some of us are waiting for this to happen ... ten years )@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
f your DNS outage wa around 6:26-6:40 and you have DHCP set to register leases in DNS, unbound would have restarted a bunch of times there.
Exact.
As I said above.
Or, his unbound doesn't restart that often. Not 10 x in 16 minutes ^^@RickyBaker : I saw you use 10.10.10.x as a LAN network
You don't use pfBlockerng, right ? -
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
So is unbound no longer restarting? But still the errors? I do not have another idea. Perhaps, on the DNS Resolver advanced page raise Log Level temporarily and see if that provides any info.
i mean, there was no indication to me other than the log that it was restarting. so I guess it's not? I will raise the log level of the DNS Resolver....cause it happened again this morning. Text from my wife 8:26am:
system.log:
DHCP log:
Nothing new in the DNS Resolver log@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
You don't use pfBlockerng, right ?
I don't (intend to) but during this thread it's been clear things I did years ago have left breadcrumbs of settings I didn't intend. Where would I check?
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
That's not good at all.
Not to get too in the weeds, but what is Register DHCP used for if it's that unwieldy?
-
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
raise Log Level temporarily and see if that provides any info.
I went to do this in the advanced settings and when i saved (I've never changed anything in Advanced Settings of dns resolver to my knowledge) I got this error:
So i disabled that, but maybe that was causing issues? -
@RickyBaker the sshguard log entries are irrelevant by themselves, but it showing every 3 minutes means you have a large amount of logging going on somewhere, and a log is rotating every 3 minutes.
The DHCP log looks like it is assigning the same address multiple times (10.10.10.177)? Are you using Kea or ISC? If Kea change back to ISC since Kea is still in preview mode. If ISC there was a bug in the initial release of 23.09 but IIRC that was fixed in a slipstream a few days later and then fixed in 23.09.1.
re: pfBlocker, it is in the Firewall menu, or would be an installed package.
-
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
You don't use pfBlockerng, right ?
would this mean no? could UDPBroadcastRelay cause issues? -
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
The DHCP log looks like it is assigning the same address multiple times (10.10.10.177)? Are you using Kea or ISC? If Kea change back to ISC since Kea is still in preview mode. If ISC there was a bug in the initial release of 23.09 but IIRC that was fixed in a slipstream a few days later and then fixed in 23.09.1.
so this was one of the 4 devices without a static ip that I was trying to identify yesterday. It was idle so I deleted it but when i typed that addrees into a mac address lookup, the manufacturer couldn't be located. I deleted it yesterday and it reappeared by the time I got home (but is not in the DHCP). So perhaps this is the issue? Should I block it via a firewall rule and see what breaks (or if anything is fixed)?
On a somewhat related note, I checked the leases for the 10.10.10.177 device and saw that it was NOT there but there WAS a DHCP lease for a non-descript android. When I typed that address into mac address lookup i discovered it was the Peloton. But i have a statically assigned IP for the peloton which is, from what i can tell, entered correctly. Is there any other reason a device wouldn't grab a statically assigned IP that it def has grabbed in the past and instead get a randomly assigned one?
and more mechanically as I'm troubleshooting all this, is there a quick and dirty way to simply rescind a randomly assigned DHCP lease inside the pfsense gui?
OOOO sorry, to answer your qeustions: I don't know what either KEA or ISC are, so i'll be googling that now....
-
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
re: pfBlocker, it is in the Firewall menu, or would be an installed package.
So... no right?
I've been googling Kea and ISC and i found that the option to switch is System->Advanced->Networking but I can't seem to find anything about it in there. I'm on pfSense 2.7.0 if that helps...
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
pfSense 2.7.0
Kea wasn't in 2.7.0. You are two versions behind though.
https://docs.netgate.com/pfsense/en/latest/releases/2-7-1.html#kea-dhcp-server-feature-preview-now-availablethis was the fix in 2.7.2:
https://docs.netgate.com/pfsense/en/latest/releases/2-7-2.html#dhcp-ipv4
However I think that started in 2.7.1.And no you don't seem to have pfBlocker installed.
-
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
you have a large amount of logging going on somewhere, and a log is rotating every 3 minutes.
any suggestions for tracing this?
-
@SteveITS to be clear, you aren't necessarily recommending I update, right?
-
any suggestions for tracing this?
Take a look at the various log files in the pfSense GUI and see if any have high activity. Or "ls -l /var/log" and see if that shows any logs with close-together timestamps.
It could be benign, for instance some people leave the dashboard open all day and pfSense logs all the web requests to update that.
to be clear, you aren't necessarily recommending I update, right?
2.7.2 is better than 2.7.1, is all. Is there a reason you're not updating? There were patches (via System Patches package) just released for 2.7.2 (and 23.09.1).
-
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Is there a reason you're not updating?
cause everything was working great and I didn't want anything to break lololol
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
"ls -l /var/log"
this just returned a list of the logs...did i do it wrong?
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Take a look at the various log files in the pfSense GUI
i flipped through every log and submenu log in the gui and nothing even closely matched up with the regular 3 minute interval of the sshguard "Exiting on signal" and "Now Monitoring Attacks"
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
returned a list of the logs
Right but if you can't see timestamps indicating they are rotating every few minutes, it's not any of those logs.
In System Logs/Settings is Log Rotation Size (Bytes) set low?
Ultimately the logs are likely not related to your symptom.
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
to be clear, you aren't necessarily recommending I update, right?
Boils down to the question : what do 'we' remember about 2.7.0 (years ago ?)
Maybe you and we are looking for an issue that was resolved long time, but we don't remember. The forum can tell you of course. For me, I'm just human, and I focus on the current version, and use the Form search button for the ancient issues.Also, keep in mind : ok to use old version but when deciding to do so you become basically your own tech supporter because of what I've outlined above.
I get it, when we started to talk about 'kea' you didn't understand what we were talking about ...
Btw : you should only install and update pfSense packages (always build against the latest pfSense version) with an up to date pfSense version.
-
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
In System Logs/Settings is Log Rotation Size (Bytes) set low?
I don't believe I've ever changed these settings:
Is this low? -
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
so you become basically your own tech supporter
Happy to update if it helps troubleshooting. Why does this say i'm up to date but also say I'm on 2.7.0 and 2.7.2 is the latest stable release?
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Why does this say i'm up to date but also say I'm on 2.7.0 and 2.7.2 is the latest stable release?
https://docs.netgate.com/pfsense/en/latest/releases/2-7-1.html#troubleshooting
Your log size field is grayed out so is the default.
-
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Your log size field is grayed out so is the default.
is that good/what you'd want to see?
-
@SteveITS I messaged my wife to ask her if any internet events had happened today and said, literally this second. I was connected to the VPN and working on the pfsense AS i texted her. I immediately refreshed the DNS Resolver log and pasted them here:
https://pastebin.com/jDipsG94
nothing interesting in the General or DHCP logs that i could tell. After pasting I raced to open a webpage to see if I was having issues. I typed 2 random words into google and opened the first link and it opened fine. I'm so perplexed.
In the meantime, since I'm so stumped. I'm working on updating to 2.7.2. I found this post @Gertjan referenced at some point. the command line suggestions early on the post seem to have gotten me in the right direction cause I'm now seeing this instead of "up to date", but clicking on update within the GUI or option 13 while ssh'ed into the pfsense both result in failure. I'm now realizing there's a bit more to the thread so I'm gonna see if there was anything further I missed but just want to document my current efforts. If anyone has any idea what this failure means, i'd love to know, thanks!
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
https://pastebin.com/jDipsG94
One thing :
The DNS log was being bombarded (you use the debug mode 3 or higher, that's ok but be aware that that creates a lot of log activity, and log files can get rotated fast as they tend to get filled up fast.
Up until April 28, 09h23 ..... and then it stops - nothing anymore.
Some shut the device down ? (power switch ? that's very bad)Then at April 29, 14h00, unbound starts, but the first part of start log sequence is missing.
Was the pfSense switched of during April 28, 09h23 and April 29, 14h00 ?
Keep an eye on free disk space.
Disable level 3+ resolver (unbound) logging as soon as possible.
