Unofficial E2guardian package for pfSense

marcelloc

Is there a parameter for the install script to not install it in the first place?

not yet.

You can remove it from config.xml using viconfig, a bad config.xml file breaks your firewall.

Mr. Jingles


Error reading file /usr/local/etc/e2guardian/lists/blacklists/adult/domains: No such file or directory
Error opening file: /usr/local/etc/e2guardian/lists/blacklists/adult/domains
Error reading: /usr/local/etc/e2guardian/lists/bannedsitelist.g_Default
Error opening bannedsitelist Error opening filter group config: /usr/local/etc/e2guardian/e2guardianf1.conf

Still looks like you did not applied a blacklist yet. :(
What blacklist are you using? I'll test with the same here to see if I get same results.

Thank you Marcello :D

The shallalist in your first post.

I downloaded and retried it five times.

The GUI says I applied it (pic).

How can I safely remove icap, clam, e2guardian, tinyproxy, and all others?

So I can start fresh again, and can you give me the exact install commands?

pfsense_e2guardian_01.jpg_thumb

jkrueger2020

Sorry to ask another question, but does E2Guardian support Man in the Middle for SSL? When I set my web browser to use port 3128 (from the Squid proxy server) and I load Amazon.com, the certificate is issued by "internal-ca" as I would expect. But when I change the port to 8080 to use E2Guardian, the SSL is issued by Amazon - not "internal-ca." This is causing keyword filtering not to work for SSL websites.

Any suggestions for what I may have missed?

jetberrocal

@jkrueger2020:

Sorry to ask another question, but does E2Guardian support Man in the Middle for SSL? When I set my web browser to use port 3128 (from the Squid proxy server) and I load Amazon.com, the certificate is issued by "internal-ca" as I would expect. But when I change the port to 8080 to use E2Guardian, the SSL is issued by Amazon - not "internal-ca." This is causing keyword filtering not to work in SSL mode.

Any suggestions for what I may have missed?

Yes it does. Thanks to marcelloc now we have e2g with mitm support.

Select the Groups Tab. Edit the group that you want to enable mitm.

Select "Filter ssl sites …" in Group options. Save.

Remember to set the Certificate for SSL mitm in General Tab. Save.

marcelloc

I'm changing the packages under unofficial repo to have uninstall and update under system -> Package manger. E2guardian will take some time as it needs a manual compiled binaries. But packages like wpad and filer are already updated.

jkrueger2020

@jetberrocal:

Yes it does. Thanks to marcelloc now we have e2g with mitm support.

Select the Groups Tab. Edit the group that you want to enable mitm.

Select "Filter ssl sites …" in Group options. Save.

Remember to set the Certificate for SSL mitm in General Tab. Save.

Thanks for the quick reply! So I got the correct SSL cert now ('internal-ca), but SSL Keyword filtering seems to be spotty. For example, if I Google using a banned (not weighted) keyword (temporarily I've set the word "Jonathan" to be banned), the search results still display. If I click on the Wikipedia article (which is HTTPS), it gets blocked, but I would have expected the search results on Google (also HTTPS) to have been blocked too. If I go to Amazon (again HTTPS) and search for "Jonathan" I also can see search results, and if I click any of the links, they show up just fine - completely ignoring the banned keyword.

Any ideas why?

jetberrocal

Searches are another story. What I do is to force the search engine to do safe search.

Actually I am now testing this.

I am having problem with Google but Yahoo and Bing are working.

From e2g forums: This is the current list that works for urlregexplist


"(^http://[0-9a-z]+\.google\.[a-z]+[-/%.0-9a-z]*/search\?)(.*)(&?)(safe=[^&]*)"->"\1\2\3"
# ... and add 'safe=vss'
"(^http://[0-9a-z]+\.google\.[a-z]+[-/%.0-9a-z]*/search\?)"->"\1safe=strict&"

#"(http[s]?://[0-9a-z]+.bing.com/images/search\?.*)"->"\1&adlt=strict"
"(http[s]?://[0-9a-z]+.bing\.[a-z]+[-/%.0-9a-z]*/search\?.*)"->"\1&adlt=strict"

# Yahoo - remove 'vm=...' and add 'vm=r'
"(^http://[.0-9a-z]+\.yahoo\.[a-z]+[-/%.0-9a-z]*/search)(.*)(&?)(vm=[^&]*)"->"\1\2\3"
"(^http://[.0-9a-z]+\.yahoo\.[a-z]+[-/%.0-9a-z]*/search+.*\?)"->"\1vm=r&"

You go to the ACLs - Url Lists that you are using for testing.  Go down to Modify section, then enable and write the code in the provided box. Save and Activate.

The Google regex need love to fix it, but I am not good with regex.
[/s][/s]

jkrueger2020

@jetberrocal:

Searches are another story. What I do is to force the search engine to do safe search.

Actually I am now testing this.

I am having problem with Google but Yahoo and Bing are working.

From e2g forums: This is the current list that works for urlregexplist


"(^http://[0-9a-z]+\.google\.[a-z]+[-/%.0-9a-z]*/search\?)(.*)(&?)(safe=[^&]*)"->"\1\2\3"
# ... and add 'safe=vss'
"(^http://[0-9a-z]+\.google\.[a-z]+[-/%.0-9a-z]*/search\?)"->"\1safe=strict&"

#"(http[s]?://[0-9a-z]+.bing.com/images/search\?.*)"->"\1&adlt=strict"
"(http[s]?://[0-9a-z]+.bing\.[a-z]+[-/%.0-9a-z]*/search\?.*)"->"\1&adlt=strict"

# Yahoo - remove 'vm=...' and add 'vm=r'
"(^http://[.0-9a-z]+\.yahoo\.[a-z]+[-/%.0-9a-z]*/search)(.*)(&?)(vm=[^&]*)"->"\1\2\3"
"(^http://[.0-9a-z]+\.yahoo\.[a-z]+[-/%.0-9a-z]*/search+.*\?)"->"\1vm=r&"

You go to the ACLs - Url Lists that you are using for testing.  Go down to Modify section, then enable and write the code in the provided box. Save and Activate.

The Google regex need love to fix it, but I am not good with regex.

The issue with SSL filtering is bigger than just search results though. For example, here's a random page on Amazon. The name "Jonathan" appears 13 times on the page, but pfSense isn't blocking it. And this isn't a search results page. My understanding is that 1 instance of the banned keyword should block the page (given that I put the <jonathan> tag under the banned Keywords - not weighted)
https://www.amazon.com/Jonathan-Thomas-Sarbacher/dp/B01NBALPRR/ref=sr_1_1?ie=UTF8&qid=1496099352&sr=8-1&keywords=Jonathan

Either I have something configured incorrectly or SSL Keyword filtering has bugs that need to be fixed. I'm not sure which.

Any thoughts?[/s][/s]</jonathan>

jetberrocal

Since you are testing I guess you only have one ACL for the Phrase List and the Groups you have some but the user you are using for testing belongs to the Group were you selected the Phrase List with the Banned word.

Let say the ACL is the Default, make sure you select Default in the Group you are testing for the Phrase List box.

Just in case assume the words are case sensitive (there is a setting to make this case insensitive)

Use the sample text to follow the correct syntax.

jkrueger2020

@jetberrocal:

Since you are testing I guess you only have one ACL for the Phrase List and the Groups you have some but the user you are using for testing belongs to the Group were you selected the Phrase List with the Banned word.

Let say the ACL is the Default, make sure you select Default in the Group you are testing for the Phrase List box.

Just in case assume the words are case sensitive (there is a setting to make this case insensitive)

Use the sample text to follow the correct syntax.

Yes, I'm using all the default groups and lists. I have not created anything new. I'm not using any authentication or users. On General > Lower Case Options I have "force lower case." On General > Phrase Filter Mode I have "smart only." And just to be overly thorough, I entered the following keywords on ACLs > Phrase List > Default > Banned:

<jonathan>< Jonathan >
< jonathan >
<jonathan>As I understand it, that should have more than covered the possibilities, but the SSL page on Amazon that I referenced in my last post isn't blocked. It still gets displayed as though I hadn't attempted to filter it. I'm starting to think this is a bug.</jonathan></jonathan>

jetberrocal

The error is happening to me.

I checked the config files and it is being generated correctly.
I also disable Exceptions box in case the word falls in the exceptions.

So it seems to be an e2g 3.5.1 problem.

I guess this has to be checked on the e2g forum.

By the way I used the word <jet>. Search on Google and selected the link
https://www.google.com.pr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=13&cad=rja&uact=8&ved=0ahUKEwjq5qXotpbUAhUE6yYKHTncD78QFghvMAw&url=https%3A%2F%2Fjetprogramusa.org%2F&usg=AFQjCNGFZZgNdXX2OXYga7BOLHmLFjdZ_g</jet>

jkrueger2020

@jetberrocal:

The error is happening to me.

I checked the config files and it is being generated correctly.
I also disable Exceptions box in case the word falls in the exceptions.

So it seems to be an e2g 3.5.1 problem.

I guess this has to be checked on the e2g forum.

By the way I used the word <jet>. Search on Google and selected the link
https://www.google.com.pr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=13&cad=rja&uact=8&ved=0ahUKEwjq5qXotpbUAhUE6yYKHTncD78QFghvMAw&url=https%3A%2F%2Fjetprogramusa.org%2F&usg=AFQjCNGFZZgNdXX2OXYga7BOLHmLFjdZ_g</jet>

Thanks for helping to check this! At least I know I'm not the only one now.

I've logged the issue on the E2Guardian Google Groups: https://groups.google.com/forum/#!topic/e2guardian/NfBZ1Ux_lEY

If no one replies within a day or so, I'm going to log it as an issue on GitHub.

Thanks again!

Jonathan

jetberrocal

@jkrueger2020:

@jetberrocal:

The error is happening to me.

I checked the config files and it is being generated correctly.
I also disable Exceptions box in case the word falls in the exceptions.

So it seems to be an e2g 3.5.1 problem.

I guess this has to be checked on the e2g forum.

By the way I used the word <jet>. Search on Google and selected the link
https://www.google.com.pr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=13&cad=rja&uact=8&ved=0ahUKEwjq5qXotpbUAhUE6yYKHTncD78QFghvMAw&url=https%3A%2F%2Fjetprogramusa.org%2F&usg=AFQjCNGFZZgNdXX2OXYga7BOLHmLFjdZ_g</jet>

Thanks for helping to check this! At least I know I'm not the only one now.

I've logged the issue on the E2Guardian Google Groups: https://groups.google.com/forum/#!topic/e2guardian/NfBZ1Ux_lEY

If no one replies within a day or so, I'm going to log it as an issue on GitHub.

Thanks again!

Jonathan

If they ask for the conf files they are in /usr/local/etc/e2guardian/

pfsensation

When using Mitm I'm getting "Error too many redirects" from time to time. It can become quite frustrating when you're trying to browse, and you have to keep refreshing and going back on the page until it finally works.

Anyone else experiencing this? I vaguely remember seeing this being reported before to the E2G team, I assume it's fixed in 4.1 If it has, maybe we can back port?

Also marcelloc, have you been able to test newer versions of E2G on pfsense? Hoping it can be stable enough soon for us to update.

marcelloc

@pfsensation:

Also marcelloc, have you been able to test newer versions of E2G on pfsense? Hoping it can be stable enough soon for us to update.

Not yet, just that version that was crashing on BSD.

marcelloc

I've updated the install process to use freebsd package style instead of manual fetch from script. you can also enable the unofficial repository to install it using GUI.

See the updated install instructions on the first post of this topic.

Unfortunately, this package is still available only for AMD64 architecture.

jetberrocal

@pfsensation:

When using Mitm I'm getting "Error too many redirects" from time to time. It can become quite frustrating when you're trying to browse, and you have to keep refreshing and going back on the page until it finally works.

Anyone else experiencing this? I vaguely remember seeing this being reported before to the E2G team, I assume it's fixed in 4.1 If it has, maybe we can back port?

Also marcelloc, have you been able to test newer versions of E2G on pfsense? Hoping it can be stable enough soon for us to update.

I found this link related to your problem
https://github.com/e2guardian/e2guardian/issues/92

I think you need to add the particular site to a ssl cert exception conf file.

Mr. Jingles

Was this answer:

@marcelloc:

I'm changing the packages under unofficial repo to have uninstall and update under system -> Package manger. E2guardian will take some time as it needs a manual compiled binaries. But packages like wpad and filer are already updated.

Meant as a reply to me:

@Mr.:

How can I safely remove icap, clam, e2guardian, tinyproxy, and all others?

So I can start fresh again, and can you give me the exact install commands?

If so, it means I can't completely remove it now?

marcelloc

You can remove packages under console using pkg binary. (pkg info, pkg delete, etc…)

I'm changing the package structure to be able to install, remove, update it easier.

pfsensation

I installed E2Guardian from the unnoficial repo AMD64. Now I have two entries for E2Guardian in the web config. How can I remove the old manual one? Or any, as they are basically the same instance with two entries in the tab.

Another question is, why isn't E2Guardian decrypting/phrase matching content properly? If you go to Youtube and type in "porn x" inappropriate images appear, and there seem to be more than enough bad words on the search results for it to be blocked. But it doesn't get blocked even though I have all the phraselists enabled for pornography. However, when I refresh then it blocks. Why isn't it blocking on the first search? Is it just checking URL and ignoring?
Also having the same issues on Yandex, when you search for "porn" in images, it loads up. If you refresh then it blocks.

I guess this is a pretty big bug. I don't think my configuration or setup is wrong because after refresh it is blocking the site just fine, maybe someone can confirm? I remember it working fine before the latest update. Not sure what could be wrong.

EDIT: wow this is weird… Searching "Porn x" from YouTube homepage right now blocks it. If you search something else then porn x. It bypasses the proxy, if you refresh the page it blocks it again. Why would it behave this way? I'm fully confused and tearing my hair out.

EDIT2: Now it seems to block searching that term from any page. If you try searching it 2/3x it eventually loads up. And bypasses block page.

Oh just to clarify. E2G is correctly decrypting HTTPS. I can see the internal CA in my browser when going to YouTube.