Unofficial E2guardian package for pfSense



  • @Cino:

    Need to figure out an easy/clean way to convert my old Dansguardian config to E2guardian (within the config.xml)

    Easy as renaming it on xml(except for some config change from dansguardian to e2guardian) but I can help with a php script if you want.



  • FYI:

    When I go to ACLs -> Antivirus, the lower menu bar disappears. 
    When I go to ACLs -> Search Engine, Both menu bars disappear.



  • @Stewart:

    FYI:

    When I go to ACLs -> Antivirus, the lower menu bar disappears. 
    When I go to ACLs -> Search Engine, Both menu bars disappear.

    It's a missing div on pkg_edit.php I've updated on install script.

    EDIT

    to apply the fix/update manually, under system_patches package, create a new patch with this info

    
    --- pkg_edit.orig.php        2017-04-05 16:25:04.960401000 +0000
    +++ pkg_edit.php 2017-04-03 22:56:33.184313000 +0000
    @@ -651,6 +651,7 @@
     if ($savehelp) {
            $savebutton->setHelp($savehelp);
     }
    +?> $form = new Form($savebutton);
    
     $form->addGlobal(new Form_Input(
    
    

    folder /usr/local/www/




  • How do we download the blacklists for them to work?  On the Blacklist Tab I'm using http://www.shallalist.de/Downloads/shallalist.tar.gz as the Blacklist URL but when I go into the Default Site List and remove the "#" from in front of items on the list, e2guardian won't start.

    I'm changing:
    #.Include
    to
    .Include

    and the error when starting is:

    /root: /usr/local/etc/rc.d/e2guardian.sh start
    kern.ipc.somaxconn: 16384 -> 16384
    kern.maxfiles: 131072 -> 131072
    kern.maxfilesperproc: 104856 -> 104856
    kern.threads.max_threads_per_proc: 4096 -> 4096
    Starting e2guardian.
    Error reading file /usr/local/etc/e2guardian/lists/blacklists/adult/domains: No such file or directory
    Error opening file: /usr/local/etc/e2guardian/lists/blacklists/adult/domains
    Error reading: /usr/local/etc/e2guardian/lists/bannedsitelist.g_Default
    Error opening bannedsitelist
    Error opening filter group config: /usr/local/etc/e2guardian/e2guardianf1.conf
    Error reading filter group conf file(s).
    Error parsing the e2guardian.conf file or other e2guardian configuration files
    /usr/local/etc/rc.d/e2guardian.sh: WARNING: failed to start e2guardian
    

    I'm guessing it's because the blacklist isn't really downloaded and applied.  I've tried changing the Update Frequency on the Blacklist Tab to Download and Update Now and I get an alert that says "E2guardian - Blacklist update process started" but I don't know how to see if it is doing anything.  After 20 minutes it still doesn't start if I uncomment the lines.



  • @Stewart:

    How do we download the blacklists for them to work?

    Select the option Download and update now, save, then back it to never or the update frequency You selected before.

    I'll change it to a Download and update button soon to keep it easy to use.

    Once it's downloaded and updated, I'll receive an system alert on gui.

    Thanks for the feedback




  • @marcelloc:

    @Stewart:

    FYI:

    When I go to ACLs -> Antivirus, the lower menu bar disappears. 
    When I go to ACLs -> Search Engine, Both menu bars disappear.

    It's a missing div on pkg_edit.php I've updated on install script.

    EDIT

    to apply the fix/update manually, under system_patches package, create a new patch with this info

    
    --- pkg_edit.orig.php        2017-04-05 16:25:04.960401000 +0000
    +++ pkg_edit.php 2017-04-03 22:56:33.184313000 +0000
    @@ -651,6 +651,7 @@
     if ($savehelp) {
            $savebutton->setHelp($savehelp);
     }
    +?> $form = new Form($savebutton);
    
     $form->addGlobal(new Form_Input(
    
    

    folder /usr/local/www/

    Applying the patch gives me:

    /usr/bin/patch --directory=/usr/local/www/ -t -p1 -i /var/patches/58e51d1fade80.patch --check --forward --ignore-whitespace
    
    Hmm...  Looks like a unified diff to me...
    The text leading up to this was:
    --------------------------
    |--- pkg_edit.orig.php        2017-04-05 16:25:04.960401000 +0000
    |+++ pkg_edit.php 2017-04-03 22:56:33.184313000 +0000
    --------------------------
    Patching file pkg_edit.php using Plan A...
    Hunk #1 failed at 651.
    1 out of 1 hunks failed while patching pkg_edit.php
    done
    


  • @marcelloc:

    @Stewart:

    How do we download the blacklists for them to work?

    Select the option Download and update now, save, then back it to never or the update frequency You selected before.

    I'll change it to a Download and update button soon to keep it easy to use.

    Once it's downloaded and updated, I'll receive an system alert on gui.

    Thanks for the feedback

    Hmmm.  That's what I did.  I get the alert of "E2guardian - Blacklist update process started" but never one that it completes.  How can I see what's going on?



  • @Stewart:

    @marcelloc:

    @Stewart:

    How do we download the blacklists for them to work?

    Select the option Download and update now, save, then back it to never or the update frequency You selected before.

    I'll change it to a Download and update button soon to keep it easy to use.

    Once it's downloaded and updated, I'll receive an system alert on gui.

    Thanks for the feedback

    Hmmm.  That's what I did.  I get the alert of "E2guardian - Blacklist update process started" but never one that it completes.  How can I see what's going on?

    I just ran the Download and Update Now and shortly thereafter got 2 instances less than a minute apart of "E2guardianBlacklist applied, check site and URL access lists for categories", but it still gives me the same error.  I checked the folder structure and there is no /usr/local/etc/e2guardian/lists/blacklists/adult folder.  Do we need to manually edit the config file to match the directory structure?

    EDIT:
    The List in Config doesn't match the structure in /usr/local/etc/e2guardian/lists/blacklists/ at all.  Is that normal?

    EDIT2:
    Or I could just select from the Include box instead, like a normal person…



  • [quote]
    .patch --check --forward --ignore-whitespace
    
    Hmm...  Looks like a unified diff to me...
    The text leading up to this was:
    --------------------------
    |--- pkg_edit.orig.php        2017-04-05 16:25:04.960401000 +0000
    |+++ pkg_edit.php 2017-04-03 22:56:33.184313000 +0000
    --------------------------
    Patching file pkg_edit.php using Plan A...
    Hunk #1 failed at 651.
    1 out of 1 hunks failed while patching pkg_edit.php
    done
    [/quote]
    
    What pfSense version are you using?
    I'm on 2.3.3
    


  • @marcelloc:

    @Cino:

    Need to figure out an easy/clean way to convert my old Dansguardian config to E2guardian (within the config.xml)

    Easy as renaming it on xml(except for some config change from dansguardian to e2guardian) but I can help with a php script if you want.

    I ran into some errors over the weekend when I tried it. I need to take another look and try again. I did rename everything from that had danguardian using find/replace. This time, I'll just focus on the ACL, Groups, IPs sections of the XML.

    Thank you for the offer, I think Im good for now unless there is a need for it by the community



  • So, it appears to be working if I set a computer to proxy at port 8080 but not when passed to Squid transparently.  From your previous response I see the IP and port should be defined on the Daemon tab but that appears to already be set properly by default and the config file shows proxyip=127.0.0.1 and proxyport=3128.  On the General tab I've tried several options and combinations for Auth Plugins but it doesn't seem to be filtering through Squid.

    Squid is running on port 3128 with all boxes in advanced being empty.  Would it matter that I still have Squidguard installed but disabled?



  • @marcelloc:

    [quote]
    .patch --check --forward --ignore-whitespace
    
    Hmm...  Looks like a unified diff to me...
    The text leading up to this was:
    --------------------------
    |--- pkg_edit.orig.php        2017-04-05 16:25:04.960401000 +0000
    |+++ pkg_edit.php 2017-04-03 22:56:33.184313000 +0000
    --------------------------
    Patching file pkg_edit.php using Plan A...
    Hunk #1 failed at 651.
    1 out of 1 hunks failed while patching pkg_edit.php
    done
    [/quote]
    
    What pfSense version are you using?
    I'm on 2.3.3
    
    2.3.3-RELEASE-p1 (amd64)
    built on Thu Mar 09 07:17:41 CST 2017
    FreeBSD 10.3-RELEASE-p17
    
    It doesn't bother me all that much.  I can make the change manually.  It was more for your benefit so that you could get it fixed for future installs.
    


  • @Stewart:

    It doesn't bother me all that much.  I can make the change manually.  It was more for your benefit so that you could get it fixed for future installs.

    Sure. I'll test all install process(including blacklist) on a 2.3.3_p1 version too.



  • @Stewart:

    Hmmm.  That's what I did.  I get the alert of "E2guardian - Blacklist update process started" but never one that it completes.  How can I see what's going on?

    
    /usr/local/www/e2guardian.php fetch_blacklist 
    
    

    This is the script that fetches and apply the blacklist.



  • Awesome, thanks!  Now that it seems to be working, any advice on getting it to work with Squid transparently?



  • @Stewart:

    Awesome, thanks!  Now that it seems to be working, any advice on getting it to work with Squid transparently?

    Dansguardian as two work modes.

    The sandwich mode, where there is a squid -> e2guardian -> tinyproxy or any other cache

    Or the starndard mode where it stays in front of squid or any other cache and try to forward client authentication requests

    What mode are you using?

    E2guardian transparent mode while in front of squid is setup manually, creating a nat rule pointing to its listening port.



  • I've moved the download and apply blacklist options from combo values to buttons on xml




  • It might be just my setup but there isn't any defaults for the 4 options under Anti-Virus. The boxes have no text. Also, under Search Engine there are no options only an Add/Save button. Click on Add, no options; only save. I downloaded the latest xml files this morning.

    running on
    2.3.4-DEVELOPMENT (amd64)
    built on Tue Apr 04 16:31:17 CDT 2017
    FreeBSD 10.3-RELEASE-p17



  • I'll test it today.

    I'm also including the ssl group options introduced on e2guardian since 3.1 version



  • @marcelloc:

    @Stewart:

    Awesome, thanks!  Now that it seems to be working, any advice on getting it to work with Squid transparently?

    Dansguardian as two work modes.

    The sandwich mode, where there is a squid -> e2guardian -> tinyproxy or any other cache

    Or the starndard mode where it stays in front of squid or any other cache and try to forward client authentication requests

    What mode are you using?

    E2guardian transparent mode while in front of squid is setup manually, creating a nat rule pointing to its listening port.

    It would be transparent.  SquidGuard doesn't need any NAT rules and just integrates with Squid.  Does e2Guardian not behave the same way?  I want it transparent with Squid so do I need a NAT rule that points LAN 80 to LAN 8080 and then e2G will automatically forward to Squid?



  • E2guardian is a daemon, not a squid helper. You need to configure ip, port , forward rules as a normal proxy daemon.



  • Ah, it's completely independent of Squid, then.  Where does it sit in-line?  I'll check the manuals tomorrow but does it sit between Squid and the user or between Squid and the WAN?  I'm sure the forwards needed are in the documentation so I'll see if I can find the ports needed in the morning.  Thanks for all your help!



  • @Stewart:

    Ah, it's completely independent of Squid, then.  Where does it sit in-line?  I'll check the manuals tomorrow but does it sit between Squid and the user or between Squid and the WAN?  I'm sure the forwards needed are in the documentation so I'll see if I can find the ports needed in the morning.  Thanks for all your help!

    E2guardian replaces squidguard and in some cases can do all squid job too but it needs a cache peer to fetch urls.

    The idea of this package is to have a full-feature replacement for squidguard and depending on you user authentication needs, dansguardian package itsel will do the job as it include tynyproxy for url fetch.



  • I've updated the binaries and GUI to include dnsauth and icap antivirus integration. The idea is to call clamv exactly the same way squid does to avoid package conflict.



  • Latest TODO as far as I can see

    • add gui  group ssl options introduced since E2guardian 3.1

    • fix search acl tab and configuration

    • increase tiny proxy config values to March E2guardian start process configuration

    • Configure clamav icap



  • A method of configuring it to listen on a CARP address via the GUI would be great too - Squid you can do this from the 'Advanced Features - Custom Options'… or is this not possible with e2g, looks like multi interface listen causes problems at the moment?



  • @danjeman:

    It looks like multi interface listen causes problems at the moment?

    Not anymore. I've hard coded a config option util multi auth psr port is better implemented on gui.



  • How do we update?  Rerun the script?  That's what I've done and I've got some changes.



  • @Stewart:

    How do we update?  Rerun the script?  That's what I've done and I've got some changes.

    Yes, rerun. Few days ago I've updated the e2guardian binaries to include icap and dnsauth.



  • I reran the script and suddenly e2Guardian wouldn't load.  The error was:

    /usr/local/etc/rc.d/e2guardian.sh: WARNING: failed to start e2guardian
    Error parsing the e2guardian.conf file or other e2guardian configuration files
    Error loading auth plugins
    auth_plugin_load() returned NULL pointer with config file: /usr/local/etc/e2guardian/authplugins/proxy-header.conf 
    Unable read plugin config plugname variable /usr/local/etc/e2guardian/authplugins/proxy-header.conf 
    

    I was able to get it to start by uncommenting the last 3 lines in /usr/local/etc/e2guardian/authplugins/proxy-header.conf:

    
    low case
    header = ''
    
    plugname = 'proxy-header'
    
    

    Not sure what uncommenting them does but it allows it to load.



  • Well, it loads but it doesn't do anything.  Whether I try to create a NAT rule or just set the browser to 8080, no pages load.  They all just time out.  I'm watching /var/log/system.log but it isn't showing anything.



  • I'll what's different from my working e2guardian.  Did you selected any authentication method?



  • I've tried Proxy-Basic and none.  Now, some pages load.  Others are blocked but show a category of NA.  I also can't get to the pfSense GUI from the local network.  Is there a way to reset to defaults in case I've done something?

    Edit:  i am getting a bunch of:

    Apr 10 17:25:20 	e2guardian 	53019 	stats reset to freechildren 16 busychildren 2 numchildren 18
    Apr 10 17:25:20 	e2guardian 	53019 	freechildren 15 + busychildren 2 != numchildren 18
    Apr 10 17:25:20 	e2guardian 	53019 	stats reset to freechildren 16 busychildren 2 numchildren 18
    Apr 10 17:25:20 	e2guardian 	53019 	freechildren 16 + busychildren 1 != numchildren 18
    Apr 10 17:25:20 	e2guardian 	53019 	stats reset to freechildren 17 busychildren 1 numchildren 18
    Apr 10 17:25:21 	e2guardian 	53019 	freechildren 16 + busychildren 1 != numchildren 18
    Apr 10 17:25:21 	e2guardian 	53019 	stats reset to freechildren 17 busychildren 1 numchildren 18
    Apr 10 17:25:21 	e2guardian 	53019 	freechildren 15 + busychildren 2 != numchildren 18
    Apr 10 17:25:21 	e2guardian 	53019 	stats reset to freechildren 16 busychildren 2 numchildren 18
    Apr 10 17:25:21 	e2guardian 	53019 	freechildren 14 + busychildren 3 != numchildren 18
    Apr 10 17:25:21 	e2guardian 	53019 	stats reset to freechildren 15 busychildren 3 numchildren 18
    Apr 10 17:25:21 	e2guardian 	53019 	freechildren 14 + busychildren 3 != numchildren 18
    Apr 10 17:25:21 	e2guardian 	53019 	stats reset to freechildren 15 busychildren 3 numchildren 18
    Apr 10 17:26:36 	e2guardian 	53019 	freechildren 16 + busychildren 1 != numchildren 18
    Apr 10 17:26:36 	e2guardian 	53019 	stats reset to freechildren 17 busychildren 1 numchildren 18
    Apr 10 17:26:36 	e2guardian 	53019 	freechildren 16 + busychildren 1 != numchildren 18
    Apr 10 17:26:36 	e2guardian 	53019 	stats reset to freechildren 17 busychildren 1 numchildren 18
    Apr 10 17:26:36 	e2guardian 	53019 	freechildren 17 + busychildren 0 != numchildren 18
    Apr 10 17:26:36 	e2guardian 	53019 	stats reset to freechildren 18 busychildren 0 numchildren 18
    Apr 10 17:26:37 	e2guardian 	53019 	freechildren 17 + busychildren 0 != numchildren 18
    Apr 10 17:26:37 	e2guardian 	53019 	stats reset to freechildren 18 busychildren 0 numchildren 18
    Apr 10 17:26:39 	e2guardian 	53019 	freechildren 17 + busychildren 0 != numchildren 18
    Apr 10 17:26:39 	e2guardian 	53019 	stats reset to freechildren 18 busychildren 0 numchildren 18 
    

    in the log now.



  • Unselect any autentication, save and apply(and maybe restart the service). The small documentation on proxy-header says that you must select a http header field to identify as users, and then add them on groups

    
    # Proxy-header auth plugin
    # FredB August 2016
    # Identifies users with header;
    # relies upon the upstream proxy.
    # Eg: in groups file
    # Mozilla/5.0 (Windows NT 6.1; rv:47.0) Gecko/20100101 Firefox/47.0=filter3
    # here:
    # header = 'user-agent'
    
    


  • Have updated looks ok so far except on the menus if you go to IPs then ACLs changes to Access Lists - will clear cache etc. and check again…

    Don't believe sync has ever worked with the default 'Sync to configured system backup server' either - this is the log entry..

    Apr 11 11:57:52 php-fpm 69222 /pkg_edit.php: [E2guardian] xmlrpc sync is enabled but there is no system backup hosts to push squid config.
    Apr 11 11:57:52 php-fpm 69222 /pkg_edit.php: Reloading E2guardian

    Every other package syncs fine with this setting.



  • @marcelloc:

    Unselect any autentication, save and apply(and maybe restart the service). The small documentation on proxy-header says that you must select a http header field to identify as users, and then add them on groups

    
    # Proxy-header auth plugin
    # FredB August 2016
    # Identifies users with header;
    # relies upon the upstream proxy.
    # Eg: in groups file
    # Mozilla/5.0 (Windows NT 6.1; rv:47.0) Gecko/20100101 Firefox/47.0=filter3
    # here:
    # header = 'user-agent'
    
    

    Is there a difference between "None" and not having any selected?



  • With no authentication set I now get an Access Denied page from Squid.

    When I access a page the Squid logs show:

    1491926079.428      3 127.0.0.1 TCP_MISS/403 4185 GET http://forum.pfsense.com/ - HIER_NONE/- text/html
    1491926079.455    248 127.0.0.1 TCP_MISS/403 4287 GET http://forum.pfsense.com/ - ORIGINAL_DST/127.0.0.1 text/html
    1491926079.499      2 127.0.0.1 TCP_MEM_HIT/200 13066 GET http://localhost:3128/squid-internal-static/icons/SN.png - HIER_NONE/- image/png
    1491926079.599      1 127.0.0.1 TAG_NONE/409 4118 CONNECT urs.microsoft.com:443 - HIER_NONE/- text/html
    1491926079.601      1 127.0.0.1 TAG_NONE/409 4118 CONNECT urs.microsoft.com:443 - HIER_NONE/- text/html
    

    /var/log/e2guardian/access.log shows:

    2017.4.11 13:29:04 - 192.168.1.100 http://forum.pfsense.com/  GET 3849 0  1 403 text/html  Default   - -
    2017.4.11 13:29:04 - 192.168.1.100 https://urs.microsoft.com:443  CONNECT 4118 0  1 200 -  Default   - -
    2017.4.11 13:29:04 - 192.168.1.100 https://urs.microsoft.com:443  CONNECT 4118 0  1 200 -  Default   - -
    
    

    No matter what page I go to, I get those same 5 lines.  The first 2 vary depending on where I go but the last 3 are exactly the same.  If I point directly to the Squid proxy port then pages load.  If I pass through transparently then it works.  If I point to the port 8080 for e2guardian it fails.  I've gone back and undone the changes I made to proxy-header.conf and everything loads properly now.  I've deselected all auth plugins on the General page.

    EDIT:  Well, not every time.  I also get

    1491928118.800 86400440 192.168.1.157 TAG_NONE/200 0 CONNECT 216.58.218.2:443 - HIER_NONE/- -
    1491928118.800 86400301 192.168.1.157 TAG_NONE_TIMEDOUT/409 0 CONNECT pagead2.googlesyndication.com:443 - HIER_NONE/- text/html;charset=utf-8
    
    

    EDIT 2:  Here's my config:

    forcequicksearch = off
    reverseaddresslookups = off
    reverseclientiplookups = off
    logclienthostnames = off
    createlistcachefiles = on
    prefercachedlists = off
    maxcontentfiltersize = 256
    maxcontentramcachescansize = 1000
    maxcontentfilecachescansize = 2000
    filecachedir = '/tmp'
    deletedownloadedtempfiles = on
    initialtrickledelay = 20
    trickledelay = 20
    downloadmanager = '/usr/local/etc/e2guardian/downloadmanagers/fancy.conf'
    downloadmanager = '/usr/local/etc/e2guardian/downloadmanagers/trickle.conf'
    downloadmanager = '/usr/local/etc/e2guardian/downloadmanagers/default.conf'
    contentscannertimeout = 60
    contentscanexceptions = off
    mapauthtoports = off
    recheckreplacedurls = off
    forwardedfor = off
    usexforwardedfor = off
    logconnectionhandlingerrors = on
    logsslerrors = off
    logchildprocesshandling = off
    maxchildren = 120
    minchildren = 8
    minsparechildren = 8
    preforkchildren = 10
    maxsparechildren = 64
    maxagechildren = 500
    maxips = 0
    ipcfilename = '/tmp/.dguardianipc'
    urlipcfilename = '/tmp/.dguardianurlipc'
    ipipcfilename = '/tmp/.dguardianipipc'
    nodaemon = off
    nologger = off
    logadblocks = off
    loguseragent =
    daemonuser = 'clamav'
    daemongroup = 'nobody'
    softrestart = on
    cacertificatepath = '/etc/ssl/demoCA/cacert.pem'
    caprivatekeypath = '/etc/ssl/demoCA/private/cakey.pem'
    certprivatekeypath = '/etc/ssl/demoCA/private/serverkey.pem'
    generatedcertpath = '/usr/local/etc/e2guardian/ssl/generatedcerts'
    
    


  • Wanted to say, thanks for bringing this project back alive. Been waiting an extremely long time for E2 Guardian, as it seems to tick all my boxes in being able to scan HTTPS traffic properly, without having to rely on a black list or DNS based blocking. Most people don't seem to understand the importance of HTTPS scanning, even knowing that there's always new proxies, and websites coming along to bypass blocks.

    How is the status of the package now? I see Stewart is saying that it isn't working, is it working for everyone else?



  • I've tested normal proxy and SSL interception. Did not started testing authentication and antivirus integration



  • @pfsensation:

    Wanted to say, thanks for bringing this project back alive. Been waiting an extremely long time for E2 Guardian, as it seems to tick all my boxes in being able to scan HTTPS traffic properly, without having to rely on a black list or DNS based blocking. Most people don't seem to understand the importance of HTTPS scanning, even knowing that there's always new proxies, and websites coming along to bypass blocks.

    How is the status of the package now? I see Stewart is saying that it isn't working, is it working for everyone else?

    I've wiped the box I was having problems with and will go back to testing once my other projects simmer down since I couldn't ever get it to work properly.  I'll report back here once I get back on it.


Log in to reply