Unofficial E2guardian package for pfSense
-
I have it in production and I'm not seeing 1/10 of these issues. Yesterday I've forced a number of install/uninstall/reinstall combination to see almost all possibilities I can handle with coding.
Not sure why your setup is so different from mine but if it's not happening on my labs I can't see what is going wrong.
On the latest update 0.4.2.3, I've included a fix to -HOST- placeholder crash(at least on my boxes.
Take a look on uninstall process. See if it removes e2guardian bsd package. If not, do it on console with pkg delete e2guardian.
The native gui pkg process is something that I can't change.
EDIT: after any install or upgrade ,I'm going to package, gui, click save then apply, to be sure all is in place.
Blacklist files can be reapplied under blacklist gui if you are seeing missing files on logs.
I've tried everything you've said. Even removing it via console and I'm still having trouble starting e2guardian. I even tried fixing the permissions but no joy.
I don't know what else I can say but squid seems to be in a worse position lol. It completely screws up the main part of some url's for example if a website is "example.com/test" for some odd reason it'll try going to "https:///test".
-
The etc files are left behind on uninstall. Maybe you should remove them manually in case they have being corrupted. Then reinstall with clean etc folder.
-
I've tried everything you've said. Even removing it via console and I'm still having trouble starting e2guardian. I even tried fixing the permissions but no joy.
what you get with :
ls -l /usr/local/sbin/e2guardian
and
killall e2guradian;/usr/local/sbin/e2guardian -N?
EDIT Tested on a clean install on 2.3.4 amd64. e2guardian package only.
>>> Installing pfSense-pkg-E2guardian4... Updating Unofficial repository catalogue... Fetching meta.txz: . done Fetching packagesite.txz: . done Processing entries: .. done Unofficial repository update completed. 13 packages processed. Updating pfSense-core repository catalogue... pfSense-core repository is up to date. Updating pfSense repository catalogue... pfSense repository is up to date. All repositories are up to date. Updating database digests format: . done The following 7 package(s) will be affected (of 0 checked): New packages to be INSTALLED: pfSense-pkg-E2guardian4: 0.4.2.3 [Unofficial] squid: 3.5.26 [pfSense] krb5: 1.15.1_4 [pfSense] pkgconf: 1.3.0,1 [pfSense] cyrus-sasl: 2.1.26_12 [pfSense] e2guardian: 4.1.1_11 [Unofficial] openssl: 1.0.2l,1 [Unofficial] Number of packages to be installed: 7 The process will require 29 MiB more space. 7 MiB to be downloaded. [1/7] Fetching pfSense-pkg-E2guardian4-0.4.2.3.txz: .......... done [2/7] Fetching squid-3.5.26.txz: .......... done [3/7] Fetching krb5-1.15.1_4.txz: .......... done [4/7] Fetching pkgconf-1.3.0,1.txz: ....... done [5/7] Fetching cyrus-sasl-2.1.26_12.txz: .......... done [6/7] Fetching e2guardian-4.1.1_11.txz: .......... done [7/7] Fetching openssl-1.0.2l,1.txz: .......... done Checking integrity... done (0 conflicting) [1/7] Installing pkgconf-1.3.0,1... [1/7] Extracting pkgconf-1.3.0,1: .......... done [2/7] Installing krb5-1.15.1_4... [2/7] Extracting krb5-1.15.1_4: .......... done [3/7] Installing cyrus-sasl-2.1.26_12... *** Added group `cyrus' (id 60) *** Added user `cyrus' (id 60) [3/7] Extracting cyrus-sasl-2.1.26_12: .......... done [4/7] Installing squid-3.5.26... ===> Creating groups. Creating group 'squid' with gid '100'. ===> Creating users Creating user 'squid' with uid '100'. ===> Pre-installation configuration for squid-3.5.26 [4/7] Extracting squid-3.5.26: .......... done [5/7] Installing e2guardian-4.1.1_11... [5/7] Extracting e2guardian-4.1.1_11: .......... done [6/7] Installing pfSense-pkg-E2guardian4-0.4.2.3... [6/7] Extracting pfSense-pkg-E2guardian4-0.4.2.3: ......... done Saving updated package information... done. Loading package configuration... done. Configuring package components... Loading package instructions... Custom commands... Executing custom_php_install_command()...Checking E2guardian Blacklists... One moment please...Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |--- /usr/local/www/pkg_edit.orig.php 2017-04-05 17:12:56.478730000 -0300 |+++ /usr/local/www/pkg_edit.php 2017-04-05 17:13:51.614222000 -0300 -------------------------- Patching file /usr/local/www/pkg_edit.php using Plan A... Hunk #1 succeeded at 656 (offset 5 lines). done Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |--- /usr/local/www/pkg.orig.php 2017-04-05 17:18:25.349676000 -0300 |+++ /usr/local/www/pkg.php 2017-04-05 17:20:49.204578000 -0300 -------------------------- Patching file /usr/local/www/pkg.php using Plan A... Hunk #1 succeeded at 329 (offset 5 lines). done Checking Blacklist... 1% 2% 3% 4% 5% 6% 7% 8% 9% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%Blacklist check done, continuing package config sync. done. Executing custom_php_resync_config_command()...done. Menu items... done. Services... done. Writing configuration... done. [7/7] Installing openssl-1.0.2l,1... Extracting openssl-1.0.2l,1: .......... done Message from cyrus-sasl-2.1.26_12: You can use sasldb2 for authentication, to add users use: saslpasswd2 -c username If you want to enable SMTP AUTH with the system Sendmail, read Sendmail.README NOTE: This port has been compiled with a default pwcheck_method of auxprop. If you want to authenticate your user by /etc/passwd, PAM or LDAP, install ports/security/cyrus-sasl2-saslauthd and set sasl_pwcheck_method to saslauthd after installing the Cyrus-IMAPd 2.X port. You should also check the /usr/local/lib/sasl2/*.conf files for the correct pwcheck_method. If you want to use GSSAPI mechanism, install ports/security/cyrus-sasl2-gssapi. If you want to use SRP mechanism, install ports/security/cyrus-sasl2-srp. If you want to use LDAP auxprop plugin, install ports/security/cyrus-sasl2-ldapdb. Message from squid-3.5.26: o You can find the configuration files for this package in the directory /usr/local/etc/squid. o The default cache directory is /var/squid/cache/. The default log directory is /var/log/squid/. Note: You must initialize new cache directories before you can start squid. Do this by running "squid -z" as 'root' or 'squid'. If your cache directories are already initialized (e.g. after an upgrade of squid) you do not need to initialize them again. o When using DiskD storage scheme remember to read documentation: http://wiki.squid-cache.org/Features/DiskDaemon and alter your kern.ipc defaults in /boot/loader.conf. DiskD will not work reliably without this. Last recomendations were: kern.ipc.msgmnb=8192 kern.ipc.msgssz=64 kern.ipc.msgtql=2048 o The default configuration will deny everyone but the local host and local networks as defined in RFC 1918 for IPv4 and RFCs 4193 and 4291 for IPv6 access to the proxy service. Edit the "http_access allow/deny" directives in /usr/local/etc/squid/squid.conf to suit your needs. o If AUTH_SQL option is set, please, don't forget to install one of following perl modules depending on database you like: databases/p5-DBD-mysql databases/p5-DBD-Pg databases/p5-DBD-SQLite To enable Squid, set squid_enable=yes in either /etc/rc.conf, /etc/rc.conf.local or /etc/rc.conf.d/squid Please see /usr/local/etc/rc.d/squid for further details. Note: If you just updated your Squid installation from an earlier version, make sure to check your Squid configuration against the 3.4 default configuration file /usr/local/etc/squid/squid.conf.sample. /usr/local/etc/squid/squid.conf.documented is a fully annotated configuration file you can consult for further reference. Additionally, you should check your configuration by calling 'squid -f /path/to/squid.conf -k parse' before starting Squid. Message from e2guardian-4.1.1_11: ===> Please Note: ******************************************************************************* This port has created a log file named e2guardian.log that can get quite large. Please read the newsyslog(8) man page for instructions on configuring log rotation and compression. This port has been converted using old dansguardian-devel port Let me know how it works (or not). (Patches always welcome.) ******************************************************************************* Message from pfSense-pkg-E2guardian4-0.4.2.3: Please visit Services - E2guardian Server menu to configure the package and enable it. Message from openssl-1.0.2l,1: Edit /usr/local/openssl/openssl.cnf to fit your needs. >>> Cleaning up cache... done. SuccessAfter install,
-
Created CA
-
selected it to e2g config and enable ssl filtering as well
-
selected watchdog
-
save
-
under general settings
-
selected Ip address as authentication
-
changed filter mode to filtered
-
save, apply
Both parent squid and e2guardian up and running with no interaction or hack via console.
-
-
The etc files are left behind on uninstall. Maybe you should remove them manually in case they have being corrupted. Then reinstall with clean etc folder.
should not happen since 0.4.2.2.
-
what you get with :
ls -l /usr/local/sbin/e2guardian
-rwxr-xr-x 1 root wheel 2099000 Jun 27 00:21 /usr/local/sbin/e2guardiankillall e2guradian;/usr/local/sbin/e2guardian -N
[2.3.4-RELEASE][root@pfSense.kortex]/root: killall e2guradian;/usr/local/sbin/e2guardian -N No matching processes were found Error reading file /usr/local/etc/e2guardian/lists/blacklists/adv/domains: No such file or directory Error opening file: /usr/local/etc/e2guardian/lists/blacklists/adv/domains Error reading: /usr/local/etc/e2guardian/lists/bannedsitelist.g_Default Error opening bannedsitelist Error opening filter group config: /usr/local/etc/e2guardian/e2guardianf1.conf Error in reading filter group files Error reading filter group conf file(s). Error parsing the e2guardian.conf file or other e2guardian configuration files -
The etc files are left behind on uninstall. Maybe you should remove them manually in case they have being corrupted. Then reinstall with clean etc folder.
Really don't wanna lose my configs :( – It took a while to configure everything to a usable state.
-
The etc files are left behind on uninstall. Maybe you should remove them manually in case they have being corrupted. Then reinstall with clean etc folder.
Really don't wanna lose my configs :( – It took a while to configure everything to a usable state.
The etc files are left behind on uninstall. Maybe you should remove them manually in case they have being corrupted. Then reinstall with clean etc folder.
should not happen since 0.4.2.2.
-
The etc files are left behind on uninstall. Maybe you should remove them manually in case they have being corrupted. Then reinstall with clean etc folder.
Really don't wanna lose my configs :( – It took a while to configure everything to a usable state.
The etc files are left behind on uninstall. Maybe you should remove them manually in case they have being corrupted. Then reinstall with clean etc folder.
should not happen since 0.4.2.2.
I feel maybe something got corrupted somewhere down the line, since I one day updated via console by accident. When hitting 13 and updating the actual system (due to the "pkg" bug).
Here's the output I get when uninstalling…
>>> Removing pfSense-pkg-E2guardian4... Checking integrity... done (0 conflicting) Deinstallation has been requested for the following 1 packages (of 0 packages in the universe): Installed packages to be REMOVED: pfSense-pkg-E2guardian4-0.4.2.3 Number of packages to be removed: 1 [1/1] Deinstalling pfSense-pkg-E2guardian4-0.4.2.3... Removing E2guardian4 components... Menu items... done. Services... done. Loading package instructions... Deinstall commands... Remove modified xml files... Removing package crons... Disabling automtic parent squid script... Removing conf files... done. [1/1] Deleting files for pfSense-pkg-E2guardian4-0.4.2.3: pfSense-pkg-E2guardian4-0.4.2.3: missing file /usr/local/etc/e2guardian/squidparent.conf [1/1] Deleting files for pfSense-pkg-E2guardian4-0.4.2.3...... pfSense-pkg-E2guardian4-0.4.2.3: missing file /usr/local/pkg/e2guardian_ips.xml [1/1] Deleting files for pfSense-pkg-E2guardian4-0.4.2.3...... pfSense-pkg-E2guardian4-0.4.2.3: missing file /usr/local/pkg/e2guardian_users.xml [1/1] Deleting files for pfSense-pkg-E2guardian4-0.4.2.3....... done Removing E2guardian4 components... Configuration... done. >>> Removing stale packages... done. SuccessThis is the output I get when installing…
>>> Installing pfSense-pkg-E2guardian4... Updating Unofficial repository catalogue... Fetching meta.txz: . done Fetching packagesite.txz: . done Processing entries: .. done Unofficial repository update completed. 13 packages processed. Updating pfSense-core repository catalogue... pfSense-core repository is up to date. Updating pfSense repository catalogue... pfSense repository is up to date. All repositories are up to date. Checking integrity... done (0 conflicting) The following 2 package(s) will be affected (of 0 checked): New packages to be INSTALLED: pfSense-pkg-E2guardian4: 0.4.2.3 [Unofficial] e2guardian: 4.1.1_11 [Unofficial] Number of packages to be installed: 2 The process will require 3 MiB more space. [1/2] Installing e2guardian-4.1.1_11... [1/2] Extracting e2guardian-4.1.1_11: .......... done [2/2] Installing pfSense-pkg-E2guardian4-0.4.2.3... [2/2] Extracting pfSense-pkg-E2guardian4-0.4.2.3: ......... done Saving updated package information... done. Loading package configuration... done. Configuring package components... Loading package instructions... Custom commands... Executing custom_php_install_command()...Checking E2guardian Blacklists... One moment please...Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |--- /usr/local/www/pkg_edit.orig.php 2017-04-05 17:12:56.478730000 -0300 |+++ /usr/local/www/pkg_edit.php 2017-04-05 17:13:51.614222000 -0300 -------------------------- Patching file /usr/local/www/pkg_edit.php using Plan A... Ignoring previously applied (or reversed) patch. Hunk #1 ignored at 656. 1 out of 1 hunks ignored--saving rejects to /usr/local/www/pkg_edit.php.rej done Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |--- /usr/local/www/pkg.orig.php 2017-04-05 17:18:25.349676000 -0300 |+++ /usr/local/www/pkg.php 2017-04-05 17:20:49.204578000 -0300 -------------------------- Patching file /usr/local/www/pkg.php using Plan A... Ignoring previously applied (or reversed) patch. Hunk #1 ignored at 329. 1 out of 1 hunks ignored--saving rejects to /usr/local/www/pkg.php.rej done Checking Blacklist... done. Executing custom_php_resync_config_command()...done. Menu items... done. Services... done. Writing configuration... done. Message from e2guardian-4.1.1_11: ===> Please Note: ******************************************************************************* This port has created a log file named e2guardian.log that can get quite large. Please read the newsyslog(8) man page for instructions on configuring log rotation and compression. This port has been converted using old dansguardian-devel port Let me know how it works (or not). (Patches always welcome.) ******************************************************************************* Message from pfSense-pkg-E2guardian4-0.4.2.3: Please visit Services - E2guardian Server menu to configure the package and enable it. >>> Cleaning up cache... done. SuccessI am seriously getting really frustrated, because Squid is even more of a buggy piece of sh** without E2Guardian. Some websites it can't even load… And when using Squid directly, I don't have Squid Guard so I have no filtering at this point in time, just open dns. I hope I have provided enough info, something seriously seems wrong here. :/
EDIT 2: FINALLY I FIXED IT WOOOW, MY HEAD IS RELIEVED!
I had to download the blacklist again, then set permissions, then re-apply blacklist. And then press the 'play button' and it started!!
-
Really don't wanna lose my configs :( – It took a while to configure everything to a usable state.
They will be there. :)
All config stays on pfSense config.xml. The etc files just reflect what were saved on GUI. -
-rwxr-xr-x 1 root wheel 2099000 Jun 27 00:21 /usr/local/sbin/e2guardianbinary permission is ok.
[2.3.4-RELEASE][root@pfSense.kortex]/root: killall e2guradian;/usr/local/sbin/e2guardian -N No matching processes were found Error reading file /usr/local/etc/e2guardian/lists/blacklists/adv/domains: No such file or directory Error opening file: /usr/local/etc/e2guardian/lists/blacklists/adv/domains Error reading: /usr/local/etc/e2guardian/lists/bannedsitelist.g_Default Error opening bannedsitelist Error opening filter group config: /usr/local/etc/e2guardian/e2guardianf1.conf Error in reading filter group files Error reading filter group conf file(s). Error parsing the e2guardian.conf file or other e2guardian configuration filesblacklist and conf files were not applied via gui.
To force a blacklist apply, save config under blacklist tab.
-
[quote] This is the output I get when installing... Looks fine. [quote] EDIT 2: [u][b]FINALLY I FIXED IT WOOOW, MY HEAD IS RELIEVED![/b][/u] I had to download the blacklist again, then set permissions, then re-apply blacklist. And then press the 'play button' and it started!! [/quote] what permissions did you had to fix? If you want to force a blacklist download during install process, remove /usr/local/pkg/blacklist.tgz file after deinstall. [/quote] -
@pfsensation, I could reproduce the erros on lab.
The problem is with reinstall. It does the uninstall process that removes conf files but do not remove the e2guardian bsd package.
This way, some files 'get lost' in the process.
I'm working on it to fix and will push a fix.
thanks for all your feedbacks! 8)
-
fetch this file and try resintalling, upgrading, removing version e2guardian pkg v 0.4.2.5
fetch -o /usr/local/pkg/e2guardian.inc https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/master/pkg-e2guardian4/files/usr/local/pkg/e2guardian.incThis fetch 'fixes' uninstall process from previous versions ( < 0.4.2.4) by replacing with current 0.4.2.5 e2guardian.inc file.
After upgrading, apply settings under services -> e2guardian.
What's new on 0.4.2.5
-
Reduced uninstall remove file process to do not break reinstalls
-
Improved watchdog script and gui realtime view.
-
-
Really don't wanna lose my configs :( – It took a while to configure everything to a usable state.
They will be there. :)
All config stays on pfSense config.xml. The etc files just reflect what were saved on GUI.I've encountered a pfSense crash, again it seems to be related to E2Guardian. Wasn't able to collect any details on it. Then I realised there was an update for E2Guardian, I updated now and ended up with the same issue (couldn't start the service). Downloaded the blacklist again, then press play and it started.
So maybe force or keep old blacklist when upgrading / installing?
Also wanted to add that ShallaList categories still don't show D: – Just says "blocked site".
-
I've encountered a pfSense crash, again it seems to be related to E2Guardian. Wasn't able to collect any details on it. Then I realised there was an update for E2Guardian, I updated now and ended up with the same issue (couldn't start the service). Downloaded the blacklist again, then press play and it started.
Did you updated the inc file before the update? I did it on 3 different installs and upgrade was fine.
-
I've encountered a pfSense crash, again it seems to be related to E2Guardian. Wasn't able to collect any details on it. Then I realised there was an update for E2Guardian, I updated now and ended up with the same issue (couldn't start the service). Downloaded the blacklist again, then press play and it started.
Did you updated the inc file before the update? I did it on 3 different installs and upgrade was fine.
What inc file are you referring to? I haven't touched any specific inc files.
-
fetch this file and try resintalling, upgrading, removing version e2guardian pkg v 0.4.2.5
fetch -o /usr/local/pkg/e2guardian.inc https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/master/pkg-e2guardian4/files/usr/local/pkg/e2guardian.incThis fetch 'fixes' uninstall process from previous versions ( < 0.4.2.4) by replacing with current 0.4.2.5 e2guardian.inc file.
After upgrading, apply settings under services -> e2guardian.
What's new on 0.4.2.5
-
Reduced uninstall remove file process to do not break reinstalls
-
Improved watchdog script and gui realtime view.
from this post. ::)
But if you are on 0.4.2.5, you don't need this fetch anymore.
-
-
fetch this file and try resintalling, upgrading, removing version e2guardian pkg v 0.4.2.5
fetch -o /usr/local/pkg/e2guardian.inc https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/master/pkg-e2guardian4/files/usr/local/pkg/e2guardian.incThis fetch 'fixes' uninstall process from previous versions ( < 0.4.2.4) by replacing with current 0.4.2.5 e2guardian.inc file.
After upgrading, apply settings under services -> e2guardian.
What's new on 0.4.2.5
-
Reduced uninstall remove file process to do not break reinstalls
-
Improved watchdog script and gui realtime view.
from this post. ::)
But if you are on 0.4.2.5, you don't need this fetch anymore.
Let's see how this goes. Have you tried setting up HTTPS transparently? If I try forwarding port 443 to 8080 on the pfSense box, it breaks HTTPS. However it works fine on HTTP (port 80) this is how I have it setup already.
I know it can be done without breaking HTTPS because smoothwall has this capability. You can get onto HTTPS sites just by installing their CA certificate (without setting up proxy settings on Android). And I am mentioning Android in particular because it doesn't have WPAD or any auto detection, so all the magic must be happening on the actual router itself, it is making HTTPS traffic go through the filter.
Let me know if this can be done, it would be useful to have for guest devices. Not 100% sure but I think Squid has this, if we could set it up for certain IP's or ranges it would be fantastic. :)
It's annoying to go around and actually manually configure the proxy settings for Android in particular, and some apps.
EDIT: I think that Fortinet uses some kind of ARP poisoning as one way of working. Probably right up there with our brute force method of using NAT redirects. Even on IOS devices, they haven't always been the best when picking up the proxy settings from WPAD.
-
-
e2guardian mitm does not work in transparent mode. If you you want transparent mode you have to turn off mitm and block only by blacklist.
I do not know the technicality why but I think is because to use SSL forging you need to authenticate the connection to a user/machine and that has to be in explicit mode.
-
Let's see how this goes. Have you tried setting up HTTPS transparently? If I try forwarding port 443 to 8080 on the pfSense box, it breaks HTTPS. However it works fine on HTTP (port 80) this is how I have it setup already.
Only when e2guardian code supports transparent ssl. Current version does not has it.
If you forward 443 to e2g, you may filter without mitm.
A working setup is e2g in sandwich mode (squid tranparent +splice all -> e2g without mitm -> automatic parent). You can deny access to sites but with no intercetion and no client config.You can create groups acls that has proxy configured and interception and groups from squid splice all.
EDIT
I've included a request on e2guardian github project. hope they can do it soon
https://github.com/e2guardian/e2guardian/issues/254
