Simple DNS log per host? Pfblocker? Ntopng?
-
Hi all,
Last week I picked up a used SG-3100 for home use. I was totally unfamiliar with pfSense, but it turned out quite easy to set up VLANs + pfBlocker. All that is missing really is a way to monitor domains visited per host (device):
-which IoT devices call home (or worse)
-what domains our windows machines are pullingJust a simple “device X made these DNS queries (list)” would be all I need.
Can pfblocker be used for this? If so: how?
Ntopng?
Thanks!
Mike -
Just to add:
I had a quick look at ntopng. The flows per host is what I want, but I’d like that list logged to an archive with say two weeks retention. Can’t find a way to create / browse a log.Mike
-
I think the new pfblockerng_devel does that via python hooks into unbound. I don't use pfblockerng for DNS filtering. I use a Pi-hole. I point all devices to the Pi-hole and then it forwards to unbound (resolving) on pfsense. Pi-hole can give you most of what you want in it's web gui. There is an API to do more than that if you choose to.
You should ask in the pfblockerng sub-forum. Very friendly and helpful developer is attentive to the forum.
You will be, unsurprisingly, shocked by what you see. Figuring out exactly where all that traffic is coming from is more difficult than it would first appear. App developers will, for a fee, include frameworks in their app that exist for no other purpose other than gathering data. Weather app asks for background location permission. You grant it because a weather app needs to know where you are. The included framework in that weather app can now harvest that location data and phone it home to be sold.
Apple is trying to deal with some of this with its, not yet mandatory, disclosures in the app store. How they are going to verify and enforce is a mystery. No way they can dig into every app and every update to see what's what.
The genie is out of the bottle... There's way too much money to be made and there are way too many law and order types who don't think anyone should care as long as they're not doing anything wrong.
-
Thanks and good points. I’ll head over to the pfblocker group.
Meanwhile I ran a google search for ntopng logging. It led me towards an area I don’t want to go: dumping logs to an ELK server. I have no servers running that can run an ELK stack and I have no knowledge whatsoever doing CLI / unix stuff.Actually I’m thinking maybe pfsense is not for me, and should have chosen a different concept for a router.
Mike
-
@mikegraw376 Don't give up on pfsense too quickly. It's not a full on NG firewall right out of the box, but it's good stuff and you can't beat the price ;)
-
@jwj thanks for your encouragement :-). Actually I absolutely love the GUI and how straightforward and solid it works. Before this I used a USG 4 pro. I had VLANs, UniFi cameras in Separate VLANs with static mappings and granular FW rules using only some ports. I managed to transfer all that to pfsense without a pain and it’s all working great.
However some of the reasons to go pfsense route were:- more processing power -> USG performs poorly for IPS
- pfblocker and custom whitelist / blacklist
- FttH config
- DNS logging
So far it delivers on the first three goals.
Mike
-
@mikegraw376 I also came from a USG. The Unifi stuff is a great concept. It just seams like they got 60% done and called it a day to go make IoT stuff.
I also had great desire to do detailed DNS logging. I followed a path not unlike you. Looked at ntop and really wanted pfblocker to have DNS logging beyond what was blocked. In the end I found it just wasn't all that worthwhile. Too much noise to recognize the signal. I do find pi-hole useful. It's really easy to configure different block lists for different groups of hosts. You can also search for specific hosts or domain names trivially. It's just knowing what to look for in all of the data that is challenging.
Anyhow, I think you'll find this forum useful. You can get the attention of Netgate employees and other professional network and security engineers. All for free... I have learned a ton here.
-
@jwj thanks for sharing your story. I think I have found an acceptable solution for now. Agonising over where to push the logs to (don’t have any Linux servers) I suddenly realised I have a synology NAS. I ran a google search for “synology syslog”, I set it up and I’m now feeding the pfsense dns resolver logs.
On the synology I can filter on client IP address so I get a pretty good list of domain names.
Already I can see it’s a lot of noise, as you said. But so far I kinda like it, it’s what I’ve wanted to have for a few years but could not get on the USG.
Best, Mike -
@mikegraw376 That's great! I love remote syslog.
-
softd will log your traffic but most of the free softwares are not good, EventSentry is nice but they want my retirement for a license. If we could log the URLs visited as part of the ntop config that would be nice, there's a feature named Active Monitoring but I dont see it listed in the GUI anywhere. THe closest I came was the interface Interface >Protocol history which isnt timestamped or granular.
-
Just something to be aware of.
You might be able to log DNS from IoT devices for now ... simple queries.
But DoH will interfere with that soon
(terrible)If your kids are using Chrome , they'll (chrome will) bypas normal DNS, and use DoH.
All you will see is a series of HTTPS requests.
/Bingo -
@bingo600 said in Simple DNS log per host? Pfblocker? Ntopng?:
@mikegraw376
If your kids are using Chrome , they'll (chrome will) bypas normal DNS, and use DoH.All you will see is a series of HTTPS requests.
Thanks for that, I heard about this before. I’m not all that worried yet. DoH still has to be enabled manually at this point and my family members don’t have a clue about IT so I can still service all devices and disable it where applicable.
We mostly use macs and iOS devices.You reckon IoT crap will start to use DoH anytime soon?
Mike
-
@mikegraw376 said in Simple DNS log per host? Pfblocker? Ntopng?:
You reckon IoT crap will start to use DoH anytime soon?
Depends ....
DoH will enable the company to "sniff" all your requests , as they could use DoH servers they control. For Google i'd say yes (speakers etc) , just to collect more info.For an ESP8266 WiFi switch ...
It would prob depend on if they are already using HTTPS for fw-update or registering , then the code is already in the device.
A small company they would prob not have own DoH servers, so they will claim security as the benefit.In general i'd say for smaller companys ... Not yet.
/Bingo
-
@bingo600
Insightful, thanks. As I’m allergic to data collectors, tracking, sniffing etc I don’t use any google products. We use the apple Ecosystem exclusively for our personal devices. Then I have synology, UniFi and now pfsense.
I’m thinking about Philips hue, put them in a sealed off VLAN,
Actually I don’t want them to phone home and scramble it, I don’t want them to know when I turn on my lights. -
@mikegraw376 The bulb talks to some service, your controller app then talks to that service to control the bulb. That's how I(nternet!)oT works. It's not local. So, yeah, they are going to know when you turn your lights on and off. A selling point is you can turn off the lights from the beach halfway around the world.
Issue is they're often easy to compromise and turn into zombie agents of bad stuff.
Smart phone that knows your every step. Credit cards that know your every purchase. Facial recognition in your city. Let's not loose sight of the forest while we worry about the trees.
-
@mikegraw376 said in Simple DNS log per host? Pfblocker? Ntopng?:
I’m allergic to data collectors, tracking, sniffing etc I don’t use any google products.
Me neither , have the same attitude.
And i have a PinePhone , to avoid tracking there (well i can't avoid GSM Mast location) but else ...
@jwj
Let's not loose sight of the forest while we worry about the trees.
I'm doing my "forrest" one tree at a time, since i cant do them all.
/Bingo
-
@jwj valid points. Phones and facial recognition we can’t control. I just don’t want to help burglars giving them “nobody’s home, please come in” invitations. This is especially relevant during summer holidays, where we will be away for weeks.
Mike
-
@bingo600 If you don't want someone to know when you turn your lights off you can't have IoT light bulbs. Simple. The genie is out of the bottle and F'n around with DNS or firewall rules isn't going to put him back in.
What happens to these companies when they screw up their security and let somebody steal the data? Never update the firmware and the devices get turned into bots? Nothing much. Slap on the hand. Small fine. They don't care. Caring costs too much money. All the while people still line up to buy their stuff. It's an own goal.
-
@jwj I suppose that was addressed to me :-). Well you are right, maybe I’ll stick with a dumb home for now instead of a smart home.
Mike -
@jwj said in Simple DNS log per host? Pfblocker? Ntopng?:
@bingo600 If you don't want someone to know when you turn your lights off you can't have IoT light bulbs. Simple. The genie is out of the bottle and F'n around with DNS or firewall rules isn't going to put him back in.
It depends .... on your capabilities
I have put Tasmota FW in my Sonoff wifi switches , now i can control them via "local web" & VPNIf i had HUE i'd look for something like this, and kick out the genie.
https://diyhue.org/#primary
https://diyhue.github.io//Bingo
