Content Filtering on CF
-
I've started working on a content filter for PFSense before I noticed the last post. I've looked into Dan's Guard which is GPL for home use and costs for commercial use and Squid Guard which is GPL and filters through URL black lists.
In my mind URL black list is good to an extent but impossible to get all the domains. So with that in mind word content filtering is a must.
My planned method is to use the internal web server already included with PFSense and use its proxy extension pointed to code written in PHP. PHP is a fast scripting language, easy to learn and already included on PFSense so it is ideal for this task.
Benefits to this approach:
1. This method will be able to work on any PFSense system including embedded systems.
2. Filter URLs .
3. Content Filter words.
4. Will work on any PFSense systems including embedded.
5. Point to an internal or an external proxy.
6. Will be licensed under the BSD License.
7. Because the filtering will be done with PHP it will be easy to add additional features.A basic proof of concept is working on my PFSense system.
-
some toughts
i 've used squid+squidguard modified pfsense on embeded, but restrictions apply … my CF life .. was reduced ... and die ... some googling and the answer was about write times to CF .... it's limited ... and now i am using a hard disk based version
-
A few updates to my post:
1. I am sending my mdmfs package to cmb (one of the principals of pfsense) for an initial eval. This allows one to make mfs (ram disk-ish) mounts via the GUI. I am using this on a few of my pfsense deployed boxes. This will provide users who want to run on CF the ability to run a full pfsense install on a reasonable sized CF. You then create mfs mounts for heavy writable directories (eg, /var/log and /tmp) to minimize writes to the CF.
As such this package with a few user selectable defaults COULD create mfs mounts for squid and squidguard to reduce heavy CF direct writes.
I am currently running a live pfsense box using this setup (proof of concept) with squid+squidguard using URL blacklists from MESD and some content filtering via regular expressions in the squidguard engine.
All this on a Via c3-800 with 256mb of RAM and a 1gb CF card - no HD. Response times seem decent thus far.
2. one problem I am already forseeing is the time it takes to run "squidGuard -C" on updated lists (db file creation of the blacklist files). This is CPU intensive and would make a low end box unresponsive during that time. I see no freely usable blacklists for squidguard that distribute the DB files already created. If someone knows of one, let me know.
3. I am using squidGuard since it is just GPL with out the lovely complications of the DansGuardian license model.
4. I've already coded a super simple package for squidguard for my core needs. More work is needed obviously to make it usable for anyone else… I'll dump screen shots after some cleanup and more testing. When that happens, that may be some time down the road unless I see more interest.
-
hi patord,
very intersting ,,, i am using squidguard package .. with features created by dvserg on russian forum .. but embeded version is awesome…
if you need some help to test, deploy or something ... let me know ..
an ideia to db files .... create an site with compacted db files and uncompress in boot time on device ...
-
I'll add $50 to the pot for DansGuardian as a transparent proxy on the generic PC install. I don't need it for work, but this is pretty much the last piece I need to get working to use pfSense at home.
-
Content filtering can be done using OpenDNS.
Use the following OpenDNS servers:208.67.222.222
208.67.220.220Then sign up for an account at OpenDNS define your network IP so that OpenDNS can identify you and then set what categories of sites you want blocked or add in the domains of your choice. Also you have a dynamic IP you can use DNS-O-Matic that is provided from OpenDNS to keep a track of your IP. So that it stays synched with OpenDNS. For additional security block UDP 53 (DNS) for everything but the OpenDNS servers.
DNS-O-Matic will be available in PFSense 1.3. For those that would like to have it now see:
http://forum.pfsense.org/index.php/topic,7311.msg41445.html#msg41445 -
Ok question…. do it have to be squid if I could meet your needs.... Theres better out there then squid for this stuff
-
Opendns is a really neat solution. I have been implementing it at several clients after reading about it in this post. It works very well! The only problem is you are unable to easily create groups so one group would be blocked and other would not. This could be done by creating some type of policy for DNS where based on your IP address you could some how forward DNS requests to Opendns for machines who have IP's in a restricted group or policy. You could also do it based on MAC address which would prevent people from tricking the system and changing their IP. A rule would also have to be created to block DNS requests to anywhere besides the pfSense so no one could circumvent the system. Or this might be tooooo complicated. This is assuming that Opendsn will not be bought by another company and turned intoa for profit.
Mark
-
. This is assuming that Opendsn will not be bought by another company and turned intoa for profit.
Mark
opendns makes its money from search pages that are displayed when you type in a bad domain. sometimes it displays a search page anyway :-)
-
Yes you are correct. I understand that is how they currently make their money…..but once a company has a massive customer base albeit non-paying, and becomes incredibly popular they have the possibility of getting gobbled up. Slimming down the functionality and then charging a premium for the more "advanced" features. Do not get me wrong. I will enjoy the ride for as long as possible.
Cheers,
Mark