Logging DNS queries
-
@johnpoz I have already disabled DNSSEC after yesterday's replies, and I have had IPv6 disabled since the beginning.
I'm just slightly annoyed there doesn't seem to be a way to know what exactly is actually being sent to the ISP DNS servers with local queries showing up in dnstop output.
I'm wondering why doesn't pfSense have a component that does this kind of monitoring. -
@Octopuss said in Logging DNS queries:
IF I start watching the WAN interface, then I do see the FIRST DNS requests on specific addresses made from the phone. Subsequent refreshing doesn't do anything.
The first part is weird.Ah ok.
Like this :@Gertjan said in Logging DNS queries:
If you 'dnstop' on a LAN, you would see even more DNS traffic. The traffic that was already in the resolver cache with a not expired TTL would get answered directly without the need of a more time consuming resolve process.
The resolver will resolve ones, and hand over the DNS request answer to the device/phone.
Subsequent DNS request for the same host name will be server from the revolver's cache, and nothing will show up on the WAN.
Only when the TTL became zero, so the resolved host name gets removed from the cache, a new resolve is needed if it is asked again and this will generate WAN DNS traffic.. -
@Octopuss said in Logging DNS queries:
I'm wondering why doesn't pfSense have a component that does this kind of monitoring.
because in 30 years in this industry this is the first I’ve heard of an ISP complaining about the volume of DNS requests.
You could set pfSense to resolve itself (the default) with Google or other DNS set for pfSense , or forward somewhere else, and then your ISP shouldn’t see (m)any requests, if nothing points to them.
-
@johnpoz said in Logging DNS queries:
I have been running a min ttl of 3600 seconds for years and years and have never ran into any issues.. So if you wanted to do that its a simple setting in unbound, in advanced
minttl.jpg
This setting is 0 by default, but the amount of requests still increases despite setting it to 3600. What's even more weird, for one website it doesn't increase at all no matter what I set and for other it increases if I refresh after longer than 5 seconds (meaning it doesn't increase if it was under 5s since last refresh).
I think this is way over my head in terms of being able to understand WTF is going on. I don't even understand the basics of networking.
Nevermind, I was watching the LAN interface.
Now it doesn't increase for the WAN interface either no matter whether I use -Q or -R parameters for dnstop (or both at once).
I give up, I am digging in pointless shit.I'd have to hire someone who understands pfSense to go over all the settings and tell me if there's anything wrong with the setup.
-
@Octopuss said in Logging DNS queries:
I'm wondering why doesn't pfSense have a component that does this kind of monitoring.
You can turn up logging in unbound if you want to see more - but its going to be hard from that log to get some idea of how many are being done in an hour or a day, etc.
You can set other logging in unbound, in the custom option box
server: log-queries: yes log-replies: yesBut not really sure an easy way in unbound to see how many queries and what specific queries were sent upstream.. and to where for each query..
You can get a bunch of info using the cmd line control of unbound
unbound-control -c /var/unbound/unbound.conf
You could prob tell from
unbound-control -c /var/unbound/unbound.conf stats_noreset
Look at total.num.recursivereplies for how many queries were done upstream, etc..
-
@Octopuss said in Logging DNS queries:
I'd have to hire someone who understands pfSense to go over all the settings and tell me if there's anything wrong with the setup.
There is nothing wrong in your setup for dns - your not causing extra queries. 16K queries in a day is not excessive for typical network these days..
setting min ttl to 3600 is not going to reduce the number of queries your clients are doing if they do not have their own local cache. iot devices don't have their own local cache. Most OSes do, application have their own even. Firefox for example has its own cache.
keeping in mind when you set that unbound will restart and the cache will be cleared. But if you have some client asking every 60 seconds for something, vs unbound having to look that up upstream. Once you set the min ttl to 3600, it will only have to ask upstream every hour vs every minute.
If your client has its own local cache of dns, windows for example - if it wants to lookup www.something.com and the ttl was 60 seconds and it constantly wanted to look this up.. It would have to ask every 60 seconds.. Once you set min ttl of 3600, it will only ever have to ask for that every hour vs every minute.
-
@johnpoz But why on earth do I not see any requests for forum.netgate.com anymore no matter what interface I watch?
I swear this thing has a life of its own. -
@Octopuss have no idea where your running the tool, if you logging levels in the query. If your device is even using pfsense IP as dns, etc. You mention a phone - your phone could be using doh via privacy settings of hide me stuff, etc..
Your device your doing the query from could have forum.netgate.com cached..
There are multiple variables at play.. I would test from a pc or laptop or something where you have better insight to where its pointing.. I would do the query from cmd line with dig or nslookup and not some application like a browser that would have its own cache, and could also be doing doh for the dns and not even asking pfsense.
with a pc or laptop running a actual os, you can flush the local dns cache, you can view it where it points for dns, you can do a specific query that you know is not being done over doh, etc.
Ran the tool on my igb0 interface, which is my lan 192.168.9.253 interface on pfsense. Did a directed query to pfsense from my client on the lan at 192.168.9.100
I can see that source IP in dnstop, and can see destination of the forum.netgate.com query.
edit: if me I would run dnstop on your wan.. run it in a screen so you can disconnect from your ssh session. Let it run for 24 hours and see how many outbound dns queries you did.. I can almost promise setting the min ttl to 3600 is going to drastically reduce the number of queries you send to your isp dns by a very noticeable amount.
edit2: I just started a dnstop on my wan.. So 24 hours from now will take a look at how many outbound queries I did. I will make sure not to restart unbound on pfsense.. I just restarted it so its cache is empty.. Lets see how many total outbound queries it does in 24 hours.. Can compare that to your 16k
-
@johnpoz said in Logging DNS queries:
have no idea where your running the tool
Sorry everything is getting mixed up together.
I am on my PC and was the entire time. Phone has nothing to do with anything.
It's just that after some of the flipping settings back and forth (ending in the same state as before) the large amount of requests to this forum simply disappeared. I don't know why.And then it got even weirder.
I restarted pfSense about ten mins ago, and then internet stopped working completely. Nothing would get translated at all.
After much trial and error, I noticed it started working again after unchecking "Enable SSL/TLS Service Respond to incoming SSL/TLS queries from local clients", which was apparently on since the beginning and probably never gave me any trouble.The only things that I changed before that was unchecking DNSSEC support yesterday, and then when I tried the TTL setting I noticed another setting "Harden DNSSEC Data DNSSEC data is required for trust-anchored zones." was causing an error due to DNSSEC being disabled. So I disabled this as well.
That's it.
Why the hell did the entire thing stop working?edit: Screw this, the forum requests started again. I understand nothing anymore and I don't care.
-
@Octopuss said in Logging DNS queries:
"Enable SSL/TLS Service Respond to incoming SSL/TLS queries from local clients",
That would have zero to do with anything at all.. But a change in that would cause a restart of unbound. Its possible if you rebooted pfsense completely that unbound was not able to bind to your interfaces when pfsense started because they were not up yet when unbound started. Changing that setting would restart unbound and now that your interfaces were up it could bind to them to use.
-
@johnpoz And now it happened again.
I had to manually restart the DNS service on pfSense. Why did I even touch the whole thing in the first place? Now everything seems to be half broken.
I'll try restarting it again to see if it's triggered by a reboot. -
@Octopuss Yeah, for whatever reason DNS simply doesn't work unless I manually restart it after I reboot pfSense.
This has NEVER happened before! -
@Octopuss why are you rebooting your pfsense?
Look in the log.. unbound can not bind to an interface if its not up.. I set my unbound to only use localhost for outbound, this will be up as soon as the box starts for sure, and your local side should be up pretty much instant as well. But wan could take a bit to come up.
What do you have it bound to for outbound - all? Which is the default
If you bind outbound to only localhost, it will auto be converted to your wan interface IP when it sends outbound traffic, little reason to actually bind it to that interface.
-
@johnpoz Thanks.
It's bound to WAN interface indeed, but that has been working for... well since I started using pfSense years ago!
I just don't understand WTF is going on. -
@johnpoz said in Logging DNS queries:
@Octopuss why are you rebooting your pfsense?
Look in the log.. unbound can not bind to an interface if its not up.. I set my unbound to only use localhost for outbound, this will be up as soon as the box starts for sure, and your local side should be up pretty much instant as well. But wan could take a bit to come up.
What do you have it bound to for outbound - all? Which is the default
If you bind outbound to only localhost, it will auto be converted to your wan interface IP when it sends outbound traffic, little reason to actually bind it to that interface.
Ok that didn't work.
I changed it to localhost and upon reboot nothing would work until I manually restarted the service. -
@Octopuss I had to restore an old configuration from before last two days to get to a stable baseline, and found out this problem triggers when I disabled DNSSEC support.
I don't get it.edit: Yep, I simply cannot disable DNSSEC otherwise I have to manually restart the DNS service on every boot.
I might as well reconfigure everything from scratch because apparently something is rotten somewhere. -
@Octopuss You might want to make sure your not trying to do any of the advanced stuff with dnssec if you turn it off in advanced.
I have never seen an issue with those being on but dnssec being off on the normal check box.. But I only ever turn dnssec off for testing for someone else. And have never actually rebooted pfsense with those advanced setting still checked but dnssec turned off.
Like I said the only time I ever reboot pfsense is on update.
What version are you running of pfsense 24.11 or CE 2.7.2 ?
edit: I could try and duplicate your issue in one of my VMs of pfsense - but not going to reboot my main physical pfsense box ;) heheh
-
@johnpoz I am on 2.7.2.
I thought the advanced settings wouldn't matter if I disable the feature in the general tab.
Anyway, I'll try. Perhaps I found a bug. -
@Octopuss yeah if the normal checkbox is set to not do dnssec, those setting for sure shouldn't come in to play.. But yeah maybe you found something weird.. So if your on 2.7.2 I will try and duplicate in my VM of that.
-
@johnpoz Nope, even with the advaned settings unchecked it's still borked. It's even weirdly borked, because some websites work and some don't, namely this forum and Facebook, but others too I guess.
I really think I should do a reinstall, this seems hopelessly screwed.
