Content Filtering on CF
-
I raised the bounty with $175 If Content Filtering is seen in a little easier way.. (CF)
http://forum.pfsense.org/index.php/topic,3660.msg23800.html#msg23800
-
@submicron:
It should be noted that squid+squidGuard (or DansGuardian) are very memory intensive. Since most WRAP and similar embedded devices are limited in the amount of memory available, this is going to be a severe limitation. I've used squid+squidGuard for a number of years now, so believe me when I tell you that putting such a solution onto the pfSense embedded platform is going to be no small feat.
Not all embedded hardware has limited RAM. I have a TNet Pro 1000 with 512 MB RAM that is expandable, but has a 256 MB CF card.
Great to see this thread come to life!
Take care -
PfSener
-
If we are talking about features for embedded hardware we have to look at the least powerful system (talking about our official minimum specs) as this means the feature has to go into base (unless we provide another way to install addons on embeddeds). You can run the embedded install on a very powerful machine as well but that's not the point here.
-
:-\ mmm, we do not make so much progress here.
Who has an idea how to make this more sexy so this thread is not gonna die in silence.
I'm realy interested in a good contentfilter as I wrote earlier ( http://forum.pfsense.org/index.php/topic,3660.msg23800.html#msg23800 ). Also the reaction on that post is worth while thinking off.Maybe an idea is to change the title from the post that started this thread to.. $1100 for a decent content filter (administrator is this possible ?? )
I know if you count the money that we come up to $975 (all contributors) but I'm willing to raise my share of the bounty to mach the $1100..Maybe that another problem is that there are to many contributers a develloper might think that it is to hard to get all the money from everybody. Lady's and gentleman devellopers, let's hear your pov 's ..
-
The biggest problem with this is it would be very difficult to implement on embedded, which is what you're requiring. If it could be a regular package, it'd probably be done by now, but that would only work on full installs. Since it can't be a package, and it's outside the realm of what we would want to add to the base system, it's unlikely this will get done any time soon.
-
After hearing here and then understanding the problems by doing a bit of experimenting with DansGuardian on a Linux box, I think Content Filtering on CF is a bridge too far. I think the bounty needs to be re-tabulated on the basis on a full install as a add-in module. Even as a full install, it is going to require a decent machine to do it, so the minimum specs are going to be higher than what you need for PFSense in its bare bones configuration, otherwise it will be slow.
So, I still stand by my contribution to the bounty for a full install version. I wonder how much support there would be from others on this as well?
-
As i Stated earlier, I am more looking into a PC build and i think a package add on would be ideal in my situation. I posted here because it seemed like the most likely place to post. Maybe we should start a new tread for those who dont mind havin a generic pc build. My bounty still stands for whatever type of Content filtering comes to pass, generic or embedded. I do agree that most embedded hardware would probably be too light for a good content filtering.
-
Bump.
Ill add $400 to any bounty for a full install module of any content filtering solutions (dansguard or similar). Ability to block sites by keywords contained on pages etc.
-
I've been using untangle.com's filtering behind a pfsense box to get content filtering at a certain location. In wanting to consolidate this I started evaluating if squidGuard (1.3beta) on a "full install" pfsense with the squid package is viable. My current testing shows promise.
Before I proceed with trying to make a package that may satisfy a good amount of interested folks:
1. I'd like confirmation of parties interested and their current bounties. This thread is pretty old. I suggest someone start a new "squidGuard" package bounty thread and start posting there since the title of this thread indicates it needs to run on the CF-embedded platform.
2. Your absolute minimum requirements to see if they are feasible.Here's what I envision for the initial releases:
a. this will be an installable package
b. it will require a "full install" pfsense on a box with very decent resources (RAM/processor/diskspace)
c. it will pull and install any needed packages from FreeBSD ports (eg, BerkeleyDB)
d. it will require the most recent squid package (pfsense) to be installed
e. at a minimum will use the MESD blacklists, and allow user to select which specific sublist to use
f. most allowable squidGuard rules expressions (time/dest/src/acl etc) to be definable via GUIFrom what I am seeing this is going to take a fair amount of time and effort to do this one.
-
I've started working on a content filter for PFSense before I noticed the last post. I've looked into Dan's Guard which is GPL for home use and costs for commercial use and Squid Guard which is GPL and filters through URL black lists.
In my mind URL black list is good to an extent but impossible to get all the domains. So with that in mind word content filtering is a must.
My planned method is to use the internal web server already included with PFSense and use its proxy extension pointed to code written in PHP. PHP is a fast scripting language, easy to learn and already included on PFSense so it is ideal for this task.
Benefits to this approach:
1. This method will be able to work on any PFSense system including embedded systems.
2. Filter URLs .
3. Content Filter words.
4. Will work on any PFSense systems including embedded.
5. Point to an internal or an external proxy.
6. Will be licensed under the BSD License.
7. Because the filtering will be done with PHP it will be easy to add additional features.A basic proof of concept is working on my PFSense system.
-
some toughts
i 've used squid+squidguard modified pfsense on embeded, but restrictions apply … my CF life .. was reduced ... and die ... some googling and the answer was about write times to CF .... it's limited ... and now i am using a hard disk based version
-
A few updates to my post:
1. I am sending my mdmfs package to cmb (one of the principals of pfsense) for an initial eval. This allows one to make mfs (ram disk-ish) mounts via the GUI. I am using this on a few of my pfsense deployed boxes. This will provide users who want to run on CF the ability to run a full pfsense install on a reasonable sized CF. You then create mfs mounts for heavy writable directories (eg, /var/log and /tmp) to minimize writes to the CF.
As such this package with a few user selectable defaults COULD create mfs mounts for squid and squidguard to reduce heavy CF direct writes.
I am currently running a live pfsense box using this setup (proof of concept) with squid+squidguard using URL blacklists from MESD and some content filtering via regular expressions in the squidguard engine.
All this on a Via c3-800 with 256mb of RAM and a 1gb CF card - no HD. Response times seem decent thus far.
2. one problem I am already forseeing is the time it takes to run "squidGuard -C" on updated lists (db file creation of the blacklist files). This is CPU intensive and would make a low end box unresponsive during that time. I see no freely usable blacklists for squidguard that distribute the DB files already created. If someone knows of one, let me know.
3. I am using squidGuard since it is just GPL with out the lovely complications of the DansGuardian license model.
4. I've already coded a super simple package for squidguard for my core needs. More work is needed obviously to make it usable for anyone else… I'll dump screen shots after some cleanup and more testing. When that happens, that may be some time down the road unless I see more interest.
-
hi patord,
very intersting ,,, i am using squidguard package .. with features created by dvserg on russian forum .. but embeded version is awesome…
if you need some help to test, deploy or something ... let me know ..
an ideia to db files .... create an site with compacted db files and uncompress in boot time on device ...
-
I'll add $50 to the pot for DansGuardian as a transparent proxy on the generic PC install. I don't need it for work, but this is pretty much the last piece I need to get working to use pfSense at home.
-
Content filtering can be done using OpenDNS.
Use the following OpenDNS servers:208.67.222.222
208.67.220.220Then sign up for an account at OpenDNS define your network IP so that OpenDNS can identify you and then set what categories of sites you want blocked or add in the domains of your choice. Also you have a dynamic IP you can use DNS-O-Matic that is provided from OpenDNS to keep a track of your IP. So that it stays synched with OpenDNS. For additional security block UDP 53 (DNS) for everything but the OpenDNS servers.
DNS-O-Matic will be available in PFSense 1.3. For those that would like to have it now see:
http://forum.pfsense.org/index.php/topic,7311.msg41445.html#msg41445 -
Ok question…. do it have to be squid if I could meet your needs.... Theres better out there then squid for this stuff
-
Opendns is a really neat solution. I have been implementing it at several clients after reading about it in this post. It works very well! The only problem is you are unable to easily create groups so one group would be blocked and other would not. This could be done by creating some type of policy for DNS where based on your IP address you could some how forward DNS requests to Opendns for machines who have IP's in a restricted group or policy. You could also do it based on MAC address which would prevent people from tricking the system and changing their IP. A rule would also have to be created to block DNS requests to anywhere besides the pfSense so no one could circumvent the system. Or this might be tooooo complicated. This is assuming that Opendsn will not be bought by another company and turned intoa for profit.
Mark
-
. This is assuming that Opendsn will not be bought by another company and turned intoa for profit.
Mark
opendns makes its money from search pages that are displayed when you type in a bad domain. sometimes it displays a search page anyway :-)
-
Yes you are correct. I understand that is how they currently make their money…..but once a company has a massive customer base albeit non-paying, and becomes incredibly popular they have the possibility of getting gobbled up. Slimming down the functionality and then charging a premium for the more "advanced" features. Do not get me wrong. I will enjoy the ride for as long as possible.
Cheers,
Mark