Unofficial E2guardian package for pfSense
-
Just updated on one of the e2guardian installations I have and blacklist category was applied to files after clicking "Re-apply current blacklist" without any other hack or pkg changes.
Being applied is one thing, and it actually taking effect when you go to a blacklisted website is another. In my case it seemed like the categories were in fact applied, however, since the E2Guardian daemon wasn't killed and the old daemon was running from memory. I could still visit black listed domains and not get the categories to show.
Therefore, what I am trying to say is, why did I have to manually actually kill E2Guardian via SSH to make it show the categories? Why wasn't unticking the "Enable E2Guardian" check-mark or any of the GUI stuff enough to make E2Guardian process stop? Even reinstallation, and uninstallation of E2Guardian didn't stop it.
Enabled E2Guardian from the GUI and enabled watch dog, went to a black listed sites and categories finally showed. However, very unclearly. It says said "porn" it should say Category: Porn clearly but that is a minor thing and can be fixed in the future.
I guess it's better to include the 'Category:' on html template.
Oops my bad, I will do that. However, I will push out some commits to your repo to make things a little more professional. :)
By the way, so far everything has been far more stable. I've pushed over 120GB through the proxy without issues so far, hope everything lasts. Even after I upgrade E2Guardian.If this works well, I will definitely start deploying it out. The only problems I see with E2Guardian now is those small errors with permissions and whatnot, which you say are now fixed.
And the other problem I see is that the phraselists are pretty old, and haven't been updated since years… Maybe those need a little checking up on by E2Guardian devs. I will try bring this to their attention. It works well still, however in some small cases it can overblock, but I've pushed a commit out to official repo which should help resolve those issues with problematic phraselists.I have two questions for you Marcello:
So as you know, I've fully clean installed my pfSense production box. And upgraded E2Guardian from 12 to 13. If I keep upgrading, do you think you've got things to a state where I won't have random corruption issues? Also the fact that E2Guardian after my upgrade was running, is some cause for concern which shouldn't be dismissed. If E2Guardian is updating, all instances of the old process should be killed, then the upgrade should take place.
The second question I have is:
Could you explain what each sections mean in the E2Guardian widget? And how many HTTP workers is 'enough' and how can I actually tell. Because the requests column sometimes shows over 1k. I don't know if that means all the workers are busy or not. I know there's a 'busy' column but it's a bit daunting and confusing just to look at and understand.
-
Being applied is one thing, and it actually taking effect when you go to a blacklisted website is another. In my case it seemed like the categories were in fact applied, however, since the E2Guardian daemon wasn't killed and the old daemon was running from memory. I could still visit black listed domains and not get the categories to show.
I did not had to kill or restart, just apply config as binaries was already ok to show categories. The missing part was to adapt shallist to have the category line inside it. The documentation says that -Q arg will kill(not kill -9) all current instances and start new one. This is the default options and I'm using it in 4 different networks without strange behaviors.
-
Being applied is one thing, and it actually taking effect when you go to a blacklisted website is another. In my case it seemed like the categories were in fact applied, however, since the E2Guardian daemon wasn't killed and the old daemon was running from memory. I could still visit black listed domains and not get the categories to show.
I did not had to kill or restart, just apply config as binaries was already ok to show categories. The missing part was to adapt shallist to have the category line inside it. The documentation says that -Q arg will kill(not kill -9) all current instances and start new one. This is the default options and I'm using it in 4 different networks without strange behaviors.
I think it is safe to say that if people uninstall E2Guardian, they actually want it to stop running. So in that case why did the daemon stay running? -Q didn't seem to be enough to actually apply changes to the blacklist. Which is weird, I had to forcefully kill the client.
Anyhow, everything seems fine for me now. However, I am only bringing it up to stop newer people installing E2Guardian running into these same issues, and to avoid problems when upgrading.
EDIT: It's really amazing to see the categories finally working. However they need a few changes, such as capital letters. Below it should say "Gambling" not "gamble".
PS: Ignore the IP being shown twice. After device it is meant to show hostname, it doesn't seem to work with my PC, but works well with IOS devices for example. -
Marcelloc
Maybe your hardware is faster than the one of pfsensation making the kill fast enough to finish on time before the file replacements. Pfsensation maybe is getting a race condition in his harware and the kill is to slow overlaping some instances with the file replacement.
Can you add a step in rhe update script to make sure all instances were killed or aborted before doing the file replacement? Thus avoiding any race condition no matter the hardware.
-
Marcelloc
Maybe your hardware is faster than the one of pfsensation making the kill fast enough to finish on time before the file replacements. Pfsensation maybe is getting a race condition in his harware and the kill is to slow overlaping some instances with the file replacement.
Can you add a step in rhe update script to make sure all instances were killed or aborted before doing the file replacement? Thus avoiding any race condition no matter the hardware.
I am suggesting exactly that! For a command when upgrading or uninstalling to actually kill the client and to have it force restarted. As regards to hardware… I am rocking a Dual Core at 3.1GHz, since I am using this at home for a handful of devices. It seems to handle everything just fine, and usage usually stays quite low. Load averages are usually around 0.20 max, unless downloading and caching massive content with Squid caching in realtime which it can go up slightly. I don't exactly know what it is, but I know for sure, E2Guardian must be gone when uninstalled.
-
I got a report from e2guardian on freebsd11 ( not pfSense) of some zumbie process that keeps running after the stop call.
This is something that will be really hard to identify as e2guardian developers do it on Linux.
I'm started studying c++ but will take a lot of time to ve good enough to debug threads.
Try changing the apply action on daemon tab to use stop and start instead o -G call.
-
I got a report from e2guardian on freebsd11 ( not pfSense) of some zumbie process that keeps running after the stop call.
This is something that will be really hard to identify as e2guardian developers do it on Linux.
I'm started studying c++ but will take a lot of time to ve good enough to debug threads.
Try changing the apply action on daemon tab to use stop and start instead o -G call.
It's already on stop and start by default. However I don't think that's working. For now I think one workaround you could add is a script to run and kill off all E2Guardian processes when "enabled E2Guardian" is unchecked. And also to kill all processes before upgrades. But yeah processing not being killed causes some issues, sometimes configuration updates are not taking effect, despite the stop and start option being selected on apply.
-
It's already on stop and start by default. However I don't think that's working.
How many time did you used the -Q option? This is the option I'm using since this field was added to gui.
It's a demon option to kill all running instances and start new ones.
Try it on console and see what it returns to you.
/usr/local/sbin/e2guardian -Q
-
if i may say, this package is not much really that helpful without its installation and setup guide.
-
if i may say, this package is not much really that helpful without its installation and setup guide.
This is a tutorial in Portuguese with basic setup instructions and some screenshots
https://eliasmoraispereira.wordpress.com/2017/06/21/pfsense-proxy-transparente-mitm-no-modo-splice-all-com-squid-e2guardian/
https://github.com/e2guardian/e2guardian/wiki
As almost all pfSense packages, configure it checking tabs and options from left to right.
The package installs a blacklist to make things easier, all you have to do is understand what a content filter do and follow the tabs.
suggestion: start with ip based authentication.
-
It's already on stop and start by default. However I don't think that's working.
How many time did you used the -Q option? This is the option I'm using since this field was added to gui.
It's a demon option to kill all running instances and start new ones.
Try it on console and see what it returns to you.
/usr/local/sbin/e2guardian -Q
I'm confused. My apply action is set up "Kill all running copies and start a new one". Isn't this what you're referring to? There's no -Q option in the GUI.
Edit: will e2 guardian work fine on the new pfsense 2.3.1_1? Everything has been stable since I've reinstalled. Don't want to screw it up again without knowing for sure any problems that arise are dealt with before hand.
-
I'm confused. My apply action is set up "Kill all running copies and start a new one". Isn't this what you're referring to? There's no -Q option in the GUI.
Restart by SO rc.d script is the stop and start script you told me you're using
the kill any running copy and start a new one is the -Q arg passed to the e2guradian process that will do all internally
Reload list and groups with a HUP is the -r options that softly restart the damon without killing any active connection.Edit: will e2 guardian work fine on the new pfsense 2.3.1_1? Everything has been stable since I've reinstalled. Don't want to screw it up again without knowing for sure any problems that arise are dealt with before hand.
Did not tested. I'll do always checking service, return erros, file permissions and commands at console. Some time pfSense pkg reinstall the binaries, sometimes not. This is an internal pfSense pkg processes that I cannot change.
If the update keeps the Unofficial repo, then the update may be painless.
 -
I'm confused. My apply action is set up "Kill all running copies and start a new one". Isn't this what you're referring to? There's no -Q option in the GUI.
Restart by SO rc.d script is the stop and start script you told me you're using
the kill any running copy and start a new one is the -Q arg passed to the e2guradian process that will do all internally
Reload list and groups with a HUP is the -r options that softly restart the damon without killing any active connection.Edit: will e2 guardian work fine on the new pfsense 2.3.1_1? Everything has been stable since I've reinstalled. Don't want to screw it up again without knowing for sure any problems that arise are dealt with before hand.
Did not tested. I'll do always checking service, return erros, file permissions and commands at console. Some time pfSense pkg reinstall the binaries, sometimes not. This is an internal pfSense pkg processes that I cannot change.
If the update keeps the Unofficial repo, then the update may be painless.
I thought you were talking about "kill all running copies and start new ones". Because that is what you've been effectively describing. That phrase. XD
My point is, if I can change configuration on E2 Guardian, without killing the process and have everything working fine. Then I'm happy with that.
The problem I have is those zombie processes running which are able to survive reinstall and upgrades, and they run the old configuration. This creates confusion, conflict and many other issues. Because the zombie process is pretty much in the RAM and isn't being killed.In regards to the update, I updated pfSense E2 Guardian wasn't started. I applied config and it started, weird the pfSense update didn't touch E2 Guardian files as far as I know. Why did it have to have the config applied again?
I really hope that E2 Guardian continues to run smoothly. The last two days since I reinstalled pfSense and re did my setup. Everything has been very smooth and stable. No unintentional crashes or anything. I've pushed over 150GB through it too lol.
-
pfsense wiki has a setup/how to guide for squid and wpad. although not that extensive but it tackled setting up wpad. and in youtube, you can see guide for squid/squidguard and wpad. however, e2guardian do not have public documentation aside from github wiki link you shown.
so it is really hard for novices to follow. yes, setup options are there from left to right but it doesnt say how to really setup this e2guardian with https and http filtering with wpad. it just shows you options to click.
you say e2guardian is a daemon which may work alone without squid or may work also with squid.
its like selling a sophisticated television but no exacting manual how to go with it and your on your own to figure out how to go with it.
-
its like you sell a sophisticated television but no exacting manual how to go with it.
The gui package helps a lot with e2guardian configuration but you need to know how content filter works, network, proxy and tcp-ip base.
You have the option to install only the e2guardian binaries from freebsd and configure all under console following wiki or any e2guardian step by step configuration for linux. Maybe this way you get more comfortable with the content filtering solution implemented by e2guardian.
This topic is also full of information about the package and configuration scenarios with e2guardian + tinyproxy, e2guardian+ squid, squid + e2guardian + squid, e2guardian + user authentication, e2guardian + ip authentication, config trouble shooting, etc…
-
this topic ? so you mean, to know all those information, we need to read this whole topic thread of 36 pages and running and connect the dots by ourselves.
-
You must be kidding me.
-
the e2guardian gui is learn-able but what were just asking, how do you configure, setup the http and https filter with wpad, firewall, etc. to get the ball running and that's it.
-
E2guardian is not an official package of pfsense that is why there is no wiki for it. This thread it does have some configuration samples is not about how to use e2guardian is about how to install it.
To learn how to use e2guardian you have to go to the e2guardian forums and dansguardian wiki/forums.
-
the e2guardian gui is learn-able but what were just asking, how do you configure, setup the http and https filter with wpad, firewall, etc. to get the ball running and that's it.
Wow man, you make it seem like absolute rocket science…
With E2 Guardian the main steps are simple:
-
Generate a CA certificate via pfSense certificate manager
-
Select that certificate on E2 Guardian
-
Figure out how you want to identify clients, identification is needed to know who is who and to assign people to groups. One of the simplest is IP based identification.
-
Setup your ACL's
-
Create groups, input who's in that group and what ACL's to apply and that's all for E2 Guardian setup.
After you have finished E2 Guardian setup, just setup your clients to use E2 Guardian as a proxy. Either manually or via GPO if you are in an organisation. And the same with the CA certificate if you go with HTTPS filtering via MITM.
For WPAD it isn't hard either, there are many guides, since Marcello made a WPAD package. It's even simpler to configure. Just go to the WPAD settings after installing the package, paste your script in there. Configure DHCP to send out your WPAD info (find details easily by Googling).
Everything here is doable. I've managed to do it myself, and I'm quite new to this too. E2 Guardian as a whole is amazing, but understand that the package is still unofficial and not everything is fool proof. And here and there you will have hiccups, but report it and after pestering Marcello a bit (:p) he can fix it. Or pass it onto the E2 Guardian team.
My personal thoughts on E2 Guardian is, it works. However there's much room for improvement but I'm happy with it as it is for now, it's really advanced and powerful. It is able to block based on content checking rather than just rely on black lists and it's very configurable on a who, what, when, where, why basis.
PS: Techbee, you need to first develop an understanding on how everything works it seems. You are trying to use something which you seem to have no knowledge on. It's very simple to Google things like WPAD and find out how it works and what it does. Anyways, I hope my little list helps you, or at least makes it clear that it isn't rocket science.
-
