Unofficial E2guardian package for pfSense

marcelloc · May 9, 2017, 2:31 PM

Hello Again

I thought to post this as well.
Since a lot of people are interested in getting MITM working with Chrome 58, I copied the CertificateAuthority.cpp & CertificateAuthority.hpp from version 4.1 to e2guardian v3.5.1 and compiled again (replace the v3.5.1 files with the ones from V4.1)

I'm buiding it with these changes and 4096 max clients util 4.1 is fine on FreeBSD . Thanks for the contribution.

EDIT: Done
https://github.com/marcelloc/Unofficial-pfSense-packages/commit/36678fe4cb3868065f5f84d90796c76fe515045c

marcelloc · May 9, 2017, 3:29 PM

@Morlac:

In e2guardian.conf, the following is checked (these should be done automatically for you when you enable/save the settings in the GUI):
cacertificatepath = '/etc/ssl/demoCA/cacert.pem'
caprivatekeypath = '/etc/ssl/demoCA/private/cakey.pem'
certprivatekeypath = '/etc/ssl/demoCA/private/serverkey.pem'
generatedcertpath = '/usr/local/etc/e2guardian/ssl/generatedcerts'

In e2guardianf1.conf, I made sure the following are configured:
reportinglevel = 3
ssldeniedrewrite = 'on'
htmltemplate = 'template.html'

the sslsiteregexplist file did not exist so I Created an empty one… not sure if its needed or not.

sslsiteregexplist = '/usr/local/etc/e2guardian/lists/sslsiteregexplist'

sslmitm = on

Also included on package
https://github.com/marcelloc/Unofficial-pfSense-packages/commit/f83a44d1ef96ec2a81d785ac6695072c9aa99e8d

pfsensation · May 9, 2017, 4:08 PM

@Bismarck:

@pfsensation:

However I will try enabling it manually like you did. Unfortunately, what I'm most worried about at the minute is the fact that pfsense doesn't update after installing E2G. I'm stuck on 2.3.3.

Just in case you have problems to update to 2.3.4-RELEASE because of bugged pkg:

https://forum.pfsense.org/index.php?topic=130071.msg716776#msg716776

Thank you very much that small guide on fixing the issue. It's weird though, no one else seems to have reported it. I will try this out and see if my pfsense finally updates after I get back home.

marcelloc · May 9, 2017, 4:12 PM

@pfsensation:

Have you already updated the package? Would love to get Chrome working with MITM.

yes. :)

I'll start testing on my vm soon.

pfsensation · May 9, 2017, 9:53 PM

@marcelloc:

@pfsensation:

Have you already updated the package? Would love to get Chrome working with MITM.

yes. :)

I'll start testing on my vm soon.

Awesome!

Is there a way to exclude URL's and IP's? For example in Squid we can set certain URL's/IP's to bypass the proxy altogether. This is useful in situations where apps have certificate pinning enabled and will not accept any other certification other than the one the developer has baked in. Not sure if there's a way to do this already (maybe exclusions section?), but definitely need this when I do enable HTTPS MITM, since it is a home network and a lot of mobile devices will be on it.

pfsensation · May 15, 2017, 9:28 PM

So I was looking through the source code of the smoothwall block page at my college, hoping that I could play around with it and tinker it a bit to work with E2Guardian at home just for the lulz or just see how modified it is from the Dansguardian it started from.

I ended up finding a lot of interesting stuff such as how they optimise loading using base64 stuff, RegEx like code, and some interesting javascript implementations to load up a CA cert popup. In addition to that, adding code to increase compatibility with browsers and as such.
Could you have a look through it Marcello, and see if anything useful could be extracted from this and used in E2Guardian? I've already managed to try things like base64 encoding with E2G logo, it does seem slightly quicker. For organisations where many devices and block pages are presented, this may help a tonne and speed things up while reducing load on the server.

Link to source: https://ybin.me/p/f3e93b53881d53d1#wDDjXNWK8ThA//iIuYtOOJLkxDRPWN3ygvRSrVJgQbo=

marcelloc · May 21, 2017, 3:08 AM

@pfsensation:

@marcelloc:

@pfsensation:

Have you already updated the package? Would love to get Chrome working with MITM.

yes. :)

I'll start testing on my vm soon.

Awesome!

Is there a way to exclude URL's and IP's?

You mean a do way to do not connect to e2guardian based on ip or hosts? If so, as package do not has a transparent proxy configuration implemented yet, you can do it on proxy configuration tab on your browser.
If you create more then one group on e2guardian, you can do an ip based authentication, and allow some source ips to do not be filtered by your proxy rules.

marcelloc · May 21, 2017, 3:13 AM

@pfsensation:

Could you have a look through it Marcello, and see if anything useful could be extracted from this and used in E2Guardian?

Sure. I'll se it as soon as possible. Thank's for the contribution. :)

pfsensation · May 22, 2017, 9:18 AM

@marcelloc:

@pfsensation:

@marcelloc:

@pfsensation:

Have you already updated the package? Would love to get Chrome working with MITM.

yes. :)

I'll start testing on my vm soon.

Awesome!

Is there a way to exclude URL's and IP's?

You mean a do way to do not connect to e2guardian based on ip or hosts? If so, as package do not has a transparent proxy configuration implemented yet, you can do it on proxy configuration tab on your browser.
If you create more then one group on e2guardian, you can do an ip based authentication, and allow some source ips to do not be filtered by your proxy rules.

I meant the destination IP's. For example, dropbox has SSL pinning, therefore it needs to completely bypass E2Guardian, and it's inspection. It won't accept nor work with a mimicked certificate. There's a couple more examples of certain programs I can give that won't work with SSL decryption or any type of inspection, such as Discord, Skype etc. Therefore if you can make an option to do this, that would be very helpful so those programs can connect directly to the internet.

Hope that made sense.

Thanks a lot for the continued work! :)

EDIT: Not sure if you've had a chance to take a look. But Fred B, has replied to your issue report on GitHub.

jetberrocal · May 25, 2017, 4:38 PM

I am getting an error on the logs:

Error reading file /usr/local/usr/local/etc/e2guardian/lists/sslsiteregexplist.g_Default: No such file or directory
Error opening sslsiteregexplist

usr/local/etc/e2guardian/lists/sslsiteregexplist.g_Default do exist but /usr/local/usr/local does not exist.

What should I do?

marcelloc · May 25, 2017, 4:59 PM

@jetberrocal:

What should I do?

It's an error on config file. I've fixed it. Thanks for the feedback

To get the fixed file, under console exec this:


fetch -o /usr/local/pkg/e2guardianfx.conf.template  https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/master/pkg-e2guardian/files/usr/local/pkg/e2guardianfx.conf.template

jetberrocal · May 25, 2017, 5:04 PM

Instead of getting the denied access page I am getting "The proxy server is refusing connections".
I loaded shallalist. Default ACL I selected all categories to be denied. I have must off the settings in defaults including to use the html template.

I added Google as an exception.

I search for pfsense and loaded pfsense web page successfully. Search for "porn" and selected the first item, then get the error expressed above.

I am using squid.

marcelloc · May 25, 2017, 5:10 PM

@jetberrocal:

Instead of getting the denied access page I am getting "The proxy server is refusing connections".

this is the message you see when squid is the front end that denies and ssl page without interception enabled.

jetberrocal · May 25, 2017, 5:16 PM

@marcelloc:

@jetberrocal:

What should I do?

It's an error on config file. I've fixed it. Thanks for the feedback

To get the fixed file, under console exec this:
fetch -o /usr/local/pkg/e2guardianfx.conf.template  https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/master/pkg-e2guardian/files/usr/local/pkg/e2guardianfx.conf.template

I already did the fetch, but maybe I am doing something wrong because still giving me the same erroro. I restarted the service and did a click save on general tab, but still had the error.

Should I have to do the fetch in a particular folder? I am doing it in the /root folder.

jetberrocal · May 25, 2017, 5:26 PM

@marcelloc:

@jetberrocal:

Instead of getting the denied access page I am getting "The proxy server is refusing connections".

this is the message you see when squid is the front end that denies and ssl page without interception enabled.

Thanks. Do you know a non ssl web site that should be block by shalla list?

Edit:
I added my web site to the Banned Config and test it (my site is not ssl).

It work correctly.

marcelloc · May 25, 2017, 5:50 PM

@jetberrocal:

still had the error.

It needs a fix on inc file too. I forgot to update on repo

fetch -o /usr/local/pkg/e2guardian.inc https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/master/pkg-e2guardian/files/usr/local/pkg/e2guardian.inc

Mr. Jingles · May 25, 2017, 7:41 PM

A fresh first time install on 2.3.4 gives scary errors :o

[2.3.4-RELEASE][admin@woof]/root: cd /root
[2.3.4-RELEASE][admin@woof]/root: fetch https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/master/pkg-e2guardian/ files/install_e2guardian_23.sh
install_e2guardian_23.sh 100% of 3075 B 16 MBps 00m00s
[2.3.4-RELEASE][admin@woof]/root: sh ./install_e2guardian_23.sh
fetching /usr/local/pkg/e2guardian.xml from github
fetching /usr/local/pkg/e2guardian_antivirus_acl.xml from github
fetching /usr/local/pkg/e2guardian_blacklist.xml from github
fetching /usr/local/pkg/e2guardian_config.xml from github
fetching /usr/local/pkg/e2guardian_content_acl.xml from github
fetching /usr/local/pkg/e2guardian_file_acl.xml from github
fetching /usr/local/pkg/e2guardian_groups.xml from github
fetching /usr/local/pkg/e2guardian_header_acl.xml from github
fetching /usr/local/pkg/e2guardian_ldap.xml from github
fetching /usr/local/pkg/e2guardian_limits.xml from github
fetching /usr/local/pkg/e2guardian_log.xml from github
fetching /usr/local/pkg/e2guardian_phrase_acl.xml from github
fetching /usr/local/pkg/e2guardian_search_acl.xml from github
fetching /usr/local/pkg/e2guardian_pics_acl.xml from github
fetching /usr/local/pkg/e2guardian_sync.xml from github
fetching /usr/local/pkg/e2guardian_site_acl.xml from github
fetching /usr/local/pkg/e2guardian_url_acl.xml from github
fetching /usr/local/pkg/e2guardian.inc from github
fetching /usr/local/pkg/pkg_e2guardian.inc from github
fetching /usr/local/pkg/e2guardian.conf.template from github
fetching /usr/local/pkg/e2guardian_ips_header.template from github
fetching /usr/local/pkg/e2guardian_rc.template from github
fetching /usr/local/pkg/e2guardian_users_footer.template from github
fetching /usr/local/pkg/e2guardian_users_header.template from github
fetching /usr/local/pkg/e2guardianfx.conf.template from github
fetching /usr/local/www/e2guardian.php from github
fetching /usr/local/www/e2guardian_about.php from github
fetching /usr/local/www/e2guardian_ldap.php from github
fetching /usr/local/www/shortcuts/pkg_e2guardian.inc from github
fetching /usr/local/pkg/tinyproxy.inc from github
Locking pkg-1.10.1_1
Updating FreeBSD repository catalogue…
pkg: Repository FreeBSD load error: access repo file(/var/db/pkg/repo-FreeBSD.sqlite) failed: No such file or directory
Fetching meta.txz: 100% 944 B 0.9kB/s 00:01
Fetching packagesite.txz: 100% 6 MiB 6.0MB/s 00:01
Processing entries: 100%
FreeBSD repository update completed. 26276 packages processed.
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.

pkg-1.10.1_1 is locked and may not be modified

pkg-1.10.1_1 is locked and may not be modified

pkg-1.10.1_1 is locked and may not be modified
The following 8 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
e2guardian: 3.5.1 [FreeBSD]
tinyproxy: 1.8.4,1 [FreeBSD]
xproto: 7.0.31 [FreeBSD]
fontconfig: 2.12.1,1 [FreeBSD]
pkg-devel: 1.10.99.4 [FreeBSD]
libfontenc: 1.1.3_1 [FreeBSD]
pixman: 0.34.0 [FreeBSD]
cyrus-sasl: 2.1.26_12 [FreeBSD]

Number of packages to be installed: 8

The process will require 28 MiB more space.
6 MiB to be downloaded.
[1/8] Fetching e2guardian-3.5.1.txz: 100% 398 KiB 407.6kB/s 00:01
[2/8] Fetching tinyproxy-1.8.4,1.txz: 100% 45 KiB 46.4kB/s 00:01
[3/8] Fetching xproto-7.0.31.txz: 100% 59 KiB 60.2kB/s 00:01
[4/8] Fetching fontconfig-2.12.1,1.txz: 100% 345 KiB 353.5kB/s 00:01
[5/8] Fetching pkg-devel-1.10.99.4.txz: 100% 4 MiB 4.4MB/s 00:01
[6/8] Fetching libfontenc-1.1.3_1.txz: 100% 18 KiB 18.2kB/s 00:01
[7/8] Fetching pixman-0.34.0.txz: 100% 256 KiB 262.6kB/s 00:01
[8/8] Fetching cyrus-sasl-2.1.26_12.txz: 100% 467 KiB 478.5kB/s 00:01
Checking integrity…
pkg-1.10.1_1 is locked and may not be modified
Assertion failed: (cun != NULL), function pkg_conflicts_check_chain_conflict, file pkg_jobs_conflicts.c, line 481.
Child process pid=52839 terminated abnormally: Abort trap
No packages matched for pattern 'e2guardian'

Checking integrity... done (0 conflicting)
Package(s) not found!
Fetching e2guardian-3.5.1.txz: 100% 424 KiB 434.2kB/s 00:01
Installing e2guardian-3.5.1...
Extracting e2guardian-3.5.1: 100%
Message from e2guardian-3.5.1:
===> Please Note:

This port has created a log file named e2guardian.log that can get
quite large. Please read the newsyslog(8) man page for instructions
on configuring log rotation and compression.

This port has been converted using old dansguardian-devel port
Let me know how it works (or not). (Patches always welcome.)

3creating menu and services...
Hmm... Looks like a unified diff to me...
The text leading up to this was:

-- /usr/local/www/pkg_edit.orig.php 2017-04-05 17:12:56.478730000 -0300

+++ /usr/local/www/pkg_edit.php 2017-04-05 17:13:51.614222000 -0300

Patching file /usr/local/www/pkg_edit.php using Plan A...

Hunk #1 succeeded at 656 (offset 5 lines).

done

Hmm... Looks like a unified diff to me...

The text leading up to this was:

-- /usr/local/www/pkg.orig.php 2017-04-05 17:18:25.349676000 -0300

+++ /usr/local/www/pkg.php 2017-04-05 17:20:49.204578000 -0300

Patching file /usr/local/www/pkg.php using Plan A...

Hunk #1 succeeded at 329 (offset 5 lines).

done

Unlocking pkg-1.10.1_1

[2.3.4-RELEASE][admin@woof]

Mr. Jingles · May 25, 2017, 7:47 PM

I did the last command you posted:

fetch -o /usr/local/pkg/e2guardian.inc https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/b3bfccd0335b30fc9b9f56856e215daabd3a6b9d/pkg-e2guardian/files/usr/local/pkg/e2guardian.inc

I went into the Daemon tab, added 8888 at the bottom, did not enable e2guardian, only pressed 'save'.

And now I'm waiting for some minutes to see the tab in my browser change from 'connecting…' to something more useful.

jetberrocal · May 25, 2017, 8:09 PM

@marcelloc:

@jetberrocal:

still had the error.

It needs a fix on inc file too. I forgot to update on repo
fetch -o /usr/local/pkg/e2guardian.inc https://raw.githubusercontent.com/marcelloc/Unofficial-pfSense-packages/b3bfccd0335b30fc9b9f56856e215daabd3a6b9d/pkg-e2guardian/files/usr/local/pkg/e2guardian.inc

It was partially fixed by fetching the new template:
May 25 15:44:42 e2guardian 67568 Error opening sslsiteregexplist
May 25 15:44:42 e2guardian 67568 Error reading file /usr/local/etc/e2guardian/lists/sslsiteregexplist.g_Default: No such file or directory

Now it is looking into /usr/local/etc, but still is not finding the file. I guess the fix will be to do also the fetch of the new .inc?

Can you place the file in a more user friendly folder name?

I connected by ssh to the console and could do a copy/paste of the fetch.

Still getting:
May 25 16:06:47 e2guardian 70630 Error opening sslsiteregexplist
May 25 16:06:47 e2guardian 70630 Error reading file /usr/local/etc/e2guardian/lists/sslsiteregexplist.g_Default: No such file or directory

jetberrocal · May 25, 2017, 8:35 PM

I have something missing between e2g and squid.
I activated squid authentication against local table and enabled Proxy-Basic in e2g, but web browsers are not asking for user/pass.

What can I provide to get help?

Unofficial E2guardian package for pfSense

the sslsiteregexplist file did not exist so I Created an empty one… not sure if its needed or not.

3creating menu and services... Hmm... Looks like a unified diff to me... The text leading up to this was:

3creating menu and services...
Hmm... Looks like a unified diff to me...
The text leading up to this was: