DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times
-
@SteveITS thanks so much for your help with this. Assuming for the best that the attacker was not able to gain access, what is my next steps? Could this have caused my weird DNS/DHCP problems or is this just a "happy" coincidence that y'all helped me find this vulnerability.
Currently tracking down the 4 DHCP leases that aren't statically assigned. FIgured out 2 of them, need to google Part II research for the 3rd and the 4th was idle so i just booted it
-
@RickyBaker I think it's just a coincidence but a fortunate one.
Is there anything notable in the DNS Resolver log when the outage happens?
-
The final device has this mac address manufacturer. The most common device seems to be a Dreo fan (which I don't have). Googling isn't ringing any bells and pointing to the IP address doesn't give me a splash page or anything. Any advice on blocking just this mac address so I can see what breaks?
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
Is there anything notable in the DNS Resolver log when the outage happens?
Surprisingly sparse....
-
@RickyBaker the DHCP log seems like all the same as well:
-
@RickyBaker if your DNS outage wa around 6:26-6:40 and you have DHCP set to register leases in DNS, unbound would have restarted a bunch of times there.
re: MAC, it has to be something on the network. You could find its IP on the Status/DHCP leases page and create a rule on LAN to block (or allow, and/or log) it.
-
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
if your DNS outage wa around 6:26-6:40 and you have DHCP set to register leases in DNS, unbound would have restarted a bunch of times there.
per @johnpoz suggestion i have unchecked "Register DHCP", should I re-enable for testing purposes?
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
re: MAC, it has to be something on the network. You could find its IP on the Status/DHCP leases page and create a rule on LAN to block (or allow, and/or log) it.
good suggestion. I THINK i found it. My wife recently purchased a fancy humidifier that, for some reason, has internet connectivity. So i will confirm when i'm home but that's most likely it...So no errant devices that aren't accounted for aside from the stale lease i booted.
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
i have unchecked "Register DHCP", should I re-enable for testing purposes
No, having it on is unlikely to help here. It's hard to keep track of multiple threads over a few days...
So is unbound no longer restarting? But still the errors? I do not have another idea. Perhaps, on the DNS Resolver advanced page raise Log Level temporarily and see if that provides any info.
-
@johnpoz said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
This is going to restart unbound..
i thought this was fixed last year, no?
-
@michmoor said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
thought this was fixed last year, no?
nope still open
https://redmine.pfsense.org/issues/5413 -
@michmoor said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
i thought this was fixed last year, no?
update: https://forum.netgate.com/topic/187506/kea-dhcp-feature-roadmap/6
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
per @johnpoz suggestion i have unchecked "Register DHCP", should I re-enable for testing purposes?
Certainly not ;) Keep it of.
Your DHCP log image above show about 10 DHCP request/renewals in let then (42-26)=16 minutes.
That means 10 unbound restart in 16 minutes ...
Every restart takes ... 30 seconds ? So during this 16 minutes your DNS is 'out' for 5 minutes.
That's not good at all.And before you start to think : isn't that totally flawed ?
Yes, it is. But help is coming - see here what cmcdonald said this morning.
( some of us are waiting for this to happen ... ten years )@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
f your DNS outage wa around 6:26-6:40 and you have DHCP set to register leases in DNS, unbound would have restarted a bunch of times there.
Exact.
As I said above.
Or, his unbound doesn't restart that often. Not 10 x in 16 minutes ^^@RickyBaker : I saw you use 10.10.10.x as a LAN network
You don't use pfBlockerng, right ? -
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
So is unbound no longer restarting? But still the errors? I do not have another idea. Perhaps, on the DNS Resolver advanced page raise Log Level temporarily and see if that provides any info.
i mean, there was no indication to me other than the log that it was restarting. so I guess it's not? I will raise the log level of the DNS Resolver....cause it happened again this morning. Text from my wife 8:26am:
system.log:
DHCP log:
Nothing new in the DNS Resolver log@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
You don't use pfBlockerng, right ?
I don't (intend to) but during this thread it's been clear things I did years ago have left breadcrumbs of settings I didn't intend. Where would I check?
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
That's not good at all.
Not to get too in the weeds, but what is Register DHCP used for if it's that unwieldy?
-
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
raise Log Level temporarily and see if that provides any info.
I went to do this in the advanced settings and when i saved (I've never changed anything in Advanced Settings of dns resolver to my knowledge) I got this error:
So i disabled that, but maybe that was causing issues? -
@RickyBaker the sshguard log entries are irrelevant by themselves, but it showing every 3 minutes means you have a large amount of logging going on somewhere, and a log is rotating every 3 minutes.
The DHCP log looks like it is assigning the same address multiple times (10.10.10.177)? Are you using Kea or ISC? If Kea change back to ISC since Kea is still in preview mode. If ISC there was a bug in the initial release of 23.09 but IIRC that was fixed in a slipstream a few days later and then fixed in 23.09.1.
re: pfBlocker, it is in the Firewall menu, or would be an installed package.
-
@Gertjan said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
You don't use pfBlockerng, right ?
would this mean no? could UDPBroadcastRelay cause issues? -
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
The DHCP log looks like it is assigning the same address multiple times (10.10.10.177)? Are you using Kea or ISC? If Kea change back to ISC since Kea is still in preview mode. If ISC there was a bug in the initial release of 23.09 but IIRC that was fixed in a slipstream a few days later and then fixed in 23.09.1.
so this was one of the 4 devices without a static ip that I was trying to identify yesterday. It was idle so I deleted it but when i typed that addrees into a mac address lookup, the manufacturer couldn't be located. I deleted it yesterday and it reappeared by the time I got home (but is not in the DHCP). So perhaps this is the issue? Should I block it via a firewall rule and see what breaks (or if anything is fixed)?
On a somewhat related note, I checked the leases for the 10.10.10.177 device and saw that it was NOT there but there WAS a DHCP lease for a non-descript android. When I typed that address into mac address lookup i discovered it was the Peloton. But i have a statically assigned IP for the peloton which is, from what i can tell, entered correctly. Is there any other reason a device wouldn't grab a statically assigned IP that it def has grabbed in the past and instead get a randomly assigned one?
and more mechanically as I'm troubleshooting all this, is there a quick and dirty way to simply rescind a randomly assigned DHCP lease inside the pfsense gui?
OOOO sorry, to answer your qeustions: I don't know what either KEA or ISC are, so i'll be googling that now....
-
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
re: pfBlocker, it is in the Firewall menu, or would be an installed package.
So... no right?
I've been googling Kea and ISC and i found that the option to switch is System->Advanced->Networking but I can't seem to find anything about it in there. I'm on pfSense 2.7.0 if that helps...
-
@RickyBaker said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
pfSense 2.7.0
Kea wasn't in 2.7.0. You are two versions behind though.
https://docs.netgate.com/pfsense/en/latest/releases/2-7-1.html#kea-dhcp-server-feature-preview-now-availablethis was the fix in 2.7.2:
https://docs.netgate.com/pfsense/en/latest/releases/2-7-2.html#dhcp-ipv4
However I think that started in 2.7.1.And no you don't seem to have pfBlocker installed.
-
@SteveITS said in DNS_PROBE_FINISHED_NXDOMAIN sporadically for anywhere from 30secs to 10min. works flawlessly at all other times:
you have a large amount of logging going on somewhere, and a log is rotating every 3 minutes.
any suggestions for tracing this?
-
@SteveITS to be clear, you aren't necessarily recommending I update, right?
