PfBlockerNG w/ DNSBL > Squid(+Guard) for Content Filtering?
-
pfbng w/ dnsbl has been excellent. it just seems to work.
This combined with redirecting all DNS requests to pfsense (DNSBL) seems to me to be a very easy to implement and very effective method of content filtering your network.
https://doc.pfsense.org/index.php/Redirecting_all_DNS_Requests_to_pfSenseI'm posting this because it doesn't seem like many people are using pfbng & dnsbl for this type of content filtering and I've seen it mentioned that people should use squidguard for that.
However, this method seems to filter http & https without any client level modification or complex pfsense configuration.
Is there something obvious that I'm missing as to why this doesn't seem to be heavily utilized for this type (content based lists) of content filtering? Or it it and I just now figured it out?
Also, how could someone on my network bypass/defeat this filtering setup?
Hi pfBasic…
I'm not an expert either, but AFAIK one way to bypass this would be to hard code the IP address.
So instead of:
http://www.bannedcontent.com/something
use:
http://x.x.x.x/something
Depending on what 'something' is, and how the 'www.bannedcontent.com' is set up, a hard coded ip might get around DNSBL.
From what I have read Microsoft is hard coding ip addresses in some of their telemetry to make it harder for users to block it.
IP filtering is required to block this type of activity.SquidGuard will block that type of attack and offer a lot of other controls you may or may not need.
This is a very quick read that will fill you in on what it provides:
http://squidguard.org/about.htmlDepending on the nature of the threat and who has access to machines running on your network, an unauthorized VPN running on the network will also get around almost any blocking.
I'm still experimenting with a non-production system and pfSense, that has a very tight firewall with only a handful of ports open. A VPN using https port 443 goes out no problem.
This is without doubt one of the hardest of all exploits to block because blocking port 443 is not a port that can be closed or white listed in all but the simplest of use cases without breaking a lot of things. To stop this one you need to block the IP of the VPN, which means you need to know about the VPNs existence.
If 80/443 were forced through SquidGuard, I don't know if it would stop a VPN except if SquidGuard was operated as whitelist only. . If the only way out 80/443 was though SquidGuard, and some form of authentication were required, that should slow down or stop unauthorized traffic (and leave log trails) unless it used the web browser for access (which a lot of malware does).
(If someone who knows more about this, it would be great if they could comment.)
Hope that helps.
-
squidgard can offer more refined filtering.
lets say e.g.
dropbox.com/someuserA/malware.zip
dropbox.com/safeuserB/picture.jpgA DNSBL list might block the entire dropbox.com domain, which is not good, and is a big reason alexa is recommended to remove popular sites from the list.
Whilst squidguard could filter the first url whilst at the same time allowing the second url,
On the flipside I dont like sending all my http/https traffic through an intermediate proxy so I use pfblockerNG not squidguard.
What might be considered an ultimate solution is that only domains in the dnsbl are routed to squid, and all other domains are direct, this is how UK isp's filter IWF traffic.
In regards to blocking the resolved ip's of domains in dnsbl, you got 2 obvious issues.
1 - the processing time to resolve everything in the list to generate the ip table would be very significant, and could even get you blocked on dns resolvers for a flood of connections.
2 - ip addresses's can be shared by many domains so if you block ip's then you are probably also going to be blocking harmless sites as well sharing the same ip address. -
On the flipside I dont like sending all my http/https traffic through an intermediate proxy so I use pfblockerNG not squidguard.
What might be considered an ultimate solution is that only domains in the dnsbl are routed to squid, and all other domains are direct, this is how UK isp's filter IWF traffic.
Just curious what your concern is?
On the flipside I dont like sending all my http/https traffic through an intermediate proxy so I use pfblockerNG not squidguard.
Security? / Performance? / Something else?
Unless you give it your browser keys, SquidGuard isn't going to be able to examine https anyway.
-
performance and possible compatibility breakage on sites.
-
Originally I had tried squidguard with both transparent proxy or wpad. It was a pain to implement and never got great results, so I quit.
pfbng w/ dnsbl has been excellent. it just seems to work.
Recently I found out how to easily use shallalist by category with DNSBL and have it updated weekly with cron. Thanks to javcasta
https://forum.pfsense.org/index.php?topic=120072.0#msg664172This combined with redirecting all DNS requests to pfsense (DNSBL) seems to me to be a very easy to implement and very effective method of content filtering your network.
https://doc.pfsense.org/index.php/Redirecting_all_DNS_Requests_to_pfSenseI'm posting this because it doesn't seem like many people are using pfbng & dnsbl for this type of content filtering and I've seen it mentioned that people should use squidguard for that.
However, this method seems to filter http & https without any client level modification or complex pfsense configuration.
Is there something obvious that I'm missing as to why this doesn't seem to be heavily utilized for this type (content based lists) of content filtering? Or it it and I just now figured it out?
Also, how could someone on my network bypass/defeat this filtering setup?
Hi. Thanks for mentions.
I have a new version for this script: https://forum.pfsense.org/index.php?topic=120072.msg685621#msg685621Regards.
-
Hi All
I have been looking at this and have hit a problem i hope you can help me with. If i enter in a browser for example youporn.com it will be blocked as expected but if i enter www.youporn.com it is not blocked. I have tested this on multiple sites with the same result.
Is it posible to include the www without editing the block list by hand.Thanks
-
Hi All
I have been looking at this and have hit a problem i hope you can help me with. If i enter in a browser for example youporn.com it will be blocked as expected but if i enter www.youporn.com it is not blocked. I have tested this on multiple sites with the same result.
Is it posible to include the www without editing the block list by hand.Thanks
You will need to enable the TLD option. Otherwise only the listed domains are blocked. Will need to run a Force Reload-DNSBL for it to take effect.
-
Keep in mind that TLD needs a lot of RAM. It's dependent on the number of hosts you have blocked in your lists. I have 8GB RAM and TLD used it all then my system crashed, maybe someday I'll throw some more RAM in there.
-
The pkg has limits on how many domains are processed via TLD. So it should have prevented the unbound memory exhaustion issue. If it's reproducible let me know and I may tweak the limits. The more details the better.
-
Thanks for the help, but enabling TLD on my meager system has limited results because of the table size. Looks like i will
have to return to squid/squidgaurd. -
Thanks for the help, but enabling TLD on my meager system has limited results because of the table size. Looks like i will
have to return to squid/squidgaurd.Sorry nothing I can control as Unbound creates pointers and uses more memory. Maybe it's time to upgrade. :)
-
Just reading this and i wonder about the RAM.
I have 2GB and TLD enabled in DNSBL and pfSense says "Memory usage 26% of 2013 MiB".I have 10 host sources in DNSBL Feeds and EasyList enabled. So how can 8GB be not enough?
Is there a place I didn't see yet where i see how many domains are blocked?
Something like what you see if you update? -
I don't think that TLD alone was the reason my system crashed. I also use ZFS and RAM disk. I have probably about ten lists as well but a couple of them have a LOT of domains.
I think TLD worked as it was supposed to but since I'm using RAM disk and ZFS I don't think the system appreciated such a small amount of RAM available and gave it up.I might try enabling it again to see what happens later and report back. But at this point I don't really need TLD. When I do I'll just add more RAM so that I have enough for all of my domains.
-
The blue infoblock for TLD will indicate recommended RAM requirements. You can review the pfblockerng.log in the Update tab window during any manual force runs. Or review the log in the Log browser tab. The that log will show stats per download and also the TLD stats after all DNSBL feeds have been downloaded and processed.
-
@mrglasspoole hi;
I had to turn off ipv6 which i don't use(ignored in linux) and this is on a dual core AMD APU with 4 gigs.
I also have issues with abuse list at first before killing ipv6 in pf latest and running out of ram issue.
but at least it tells you and shuts down.
but would like the caching squid had and av.
