Unofficial E2guardian package for pfSense

jkrueger2020

@jkrueger2020:

I had expected the results to be blocked, but they were not. Neither was the URL I clicked from the search results:

I had a feeling the problem was going to be caused by something stupid that I did wrong. And it was. Apparently, it doesn't matter if the checkbox on ACLs > Phrase Lists is checked to be Enabled. They aren't REALLY enabled unless the General > Weighted Phrase Mode is set to something other than "Off." I hadn't even realized I had missed turning that setting on. Obviously this isn't your fault - it's a quirk of E2Guardian. And now that I know the quirk, I can work around it.

Anyway, on a side note, Marcelloc, words cannot express how truly grateful I am for your work in helping E2Guardian come to pfSense. I really do mean that. I'm so excited to finally be able to start configuring pfSense with keyword blocking! I was about to give up hope of finding a good solution and then I found this. Many, many thanks for your efforts!

jetberrocal

@jkrueger2020:

@jkrueger2020:

I had expected the results to be blocked, but they were not. Neither was the URL I clicked from the search results:

I had a feeling the problem was going to be caused by something stupid that I did wrong. And it was. Apparently, it doesn't matter if the checkbox on ACLs > Phrase Lists is checked to be Enabled. They aren't REALLY enabled unless the General > Weighted Phrase Mode is set to something other than "Off." I hadn't even realized I had missed turning that setting on. Obviously this isn't your fault - it's a quirk of E2Guardian. And now that I know the quirk, I can work around it.

Anyway, on a side note, Marcelloc, words cannot express how truly grateful I am for your work in helping E2Guardian come to pfSense. I really do mean that. I'm so excited to finally be able to start configuring pfSense with keyword blocking! I was about to give up hope of finding a good solution and then I found this. Many, many thanks for your efforts!

Did you confirmed the problem was the General setting? When you enabled it the phrase blocking you tested started to work?

jkrueger2020

@jetberrocal:

Did you confirmed the problem was the General setting? When you enabled it the phrase blocking you tested started to work?

Yes, once the General setting was enabled, phrase blocking worked just fine. Quite confusing for a noob such as myself, but now that I know to look in two spots, I'm good. Thanks for the assistance!

jetberrocal

I am wondering how do I remove tinyproxy from the Services.

Since I use squid I do not really have a need for tinyproxy. How do I remove it from the Status/Services?

I guess I can "pkg remove tinyproxy" but this won't remove it from the page.

Is there a parameter for the install script to not install it in the first place?

marcelloc

@jetberrocal:

Is there a parameter for the install script to not install it in the first place?

not yet.

You can remove it from config.xml using viconfig, a bad config.xml file breaks your firewall.

Mr. Jingles

@marcelloc:

@Mr.:


Error reading file /usr/local/etc/e2guardian/lists/blacklists/adult/domains: No such file or directory
Error opening file: /usr/local/etc/e2guardian/lists/blacklists/adult/domains
Error reading: /usr/local/etc/e2guardian/lists/bannedsitelist.g_Default
Error opening bannedsitelist Error opening filter group config: /usr/local/etc/e2guardian/e2guardianf1.conf

Still looks like you did not applied a blacklist yet. :(
What blacklist are you using? I'll test with the same here to see if I get same results.

Thank you Marcello :D

The shallalist in your first post.

I downloaded and retried it five times.

The GUI says I applied it (pic).

How can I safely remove icap, clam, e2guardian, tinyproxy, and all others?

So I can start fresh again, and can you give me the exact install commands?

pfsense_e2guardian_01.jpg_thumb

jkrueger2020

Sorry to ask another question, but does E2Guardian support Man in the Middle for SSL? When I set my web browser to use port 3128 (from the Squid proxy server) and I load Amazon.com, the certificate is issued by "internal-ca" as I would expect. But when I change the port to 8080 to use E2Guardian, the SSL is issued by Amazon - not "internal-ca." This is causing keyword filtering not to work for SSL websites.

Any suggestions for what I may have missed?

jetberrocal

@jkrueger2020:

Sorry to ask another question, but does E2Guardian support Man in the Middle for SSL? When I set my web browser to use port 3128 (from the Squid proxy server) and I load Amazon.com, the certificate is issued by "internal-ca" as I would expect. But when I change the port to 8080 to use E2Guardian, the SSL is issued by Amazon - not "internal-ca." This is causing keyword filtering not to work in SSL mode.

Any suggestions for what I may have missed?

Yes it does. Thanks to marcelloc now we have e2g with mitm support.

Select the Groups Tab. Edit the group that you want to enable mitm.

Select "Filter ssl sites …" in Group options. Save.

Remember to set the Certificate for SSL mitm in General Tab. Save.

marcelloc

I'm changing the packages under unofficial repo to have uninstall and update under system -> Package manger. E2guardian will take some time as it needs a manual compiled binaries. But packages like wpad and filer are already updated.

jkrueger2020

@jetberrocal:

Yes it does. Thanks to marcelloc now we have e2g with mitm support.

Select the Groups Tab. Edit the group that you want to enable mitm.

Select "Filter ssl sites …" in Group options. Save.

Remember to set the Certificate for SSL mitm in General Tab. Save.

Thanks for the quick reply! So I got the correct SSL cert now ('internal-ca), but SSL Keyword filtering seems to be spotty. For example, if I Google using a banned (not weighted) keyword (temporarily I've set the word "Jonathan" to be banned), the search results still display. If I click on the Wikipedia article (which is HTTPS), it gets blocked, but I would have expected the search results on Google (also HTTPS) to have been blocked too. If I go to Amazon (again HTTPS) and search for "Jonathan" I also can see search results, and if I click any of the links, they show up just fine - completely ignoring the banned keyword.

Any ideas why?

jetberrocal

Searches are another story. What I do is to force the search engine to do safe search.

Actually I am now testing this.

I am having problem with Google but Yahoo and Bing are working.

From e2g forums: This is the current list that works for urlregexplist


"(^http://[0-9a-z]+\.google\.[a-z]+[-/%.0-9a-z]*/search\?)(.*)(&?)(safe=[^&]*)"->"\1\2\3"
# ... and add 'safe=vss'
"(^http://[0-9a-z]+\.google\.[a-z]+[-/%.0-9a-z]*/search\?)"->"\1safe=strict&"

#"(http[s]?://[0-9a-z]+.bing.com/images/search\?.*)"->"\1&adlt=strict"
"(http[s]?://[0-9a-z]+.bing\.[a-z]+[-/%.0-9a-z]*/search\?.*)"->"\1&adlt=strict"

# Yahoo - remove 'vm=...' and add 'vm=r'
"(^http://[.0-9a-z]+\.yahoo\.[a-z]+[-/%.0-9a-z]*/search)(.*)(&?)(vm=[^&]*)"->"\1\2\3"
"(^http://[.0-9a-z]+\.yahoo\.[a-z]+[-/%.0-9a-z]*/search+.*\?)"->"\1vm=r&"

You go to the ACLs - Url Lists that you are using for testing.  Go down to Modify section, then enable and write the code in the provided box. Save and Activate.

The Google regex need love to fix it, but I am not good with regex.
[/s][/s]

jkrueger2020

@jetberrocal:

Searches are another story. What I do is to force the search engine to do safe search.

Actually I am now testing this.

I am having problem with Google but Yahoo and Bing are working.

From e2g forums: This is the current list that works for urlregexplist


"(^http://[0-9a-z]+\.google\.[a-z]+[-/%.0-9a-z]*/search\?)(.*)(&?)(safe=[^&]*)"->"\1\2\3"
# ... and add 'safe=vss'
"(^http://[0-9a-z]+\.google\.[a-z]+[-/%.0-9a-z]*/search\?)"->"\1safe=strict&"

#"(http[s]?://[0-9a-z]+.bing.com/images/search\?.*)"->"\1&adlt=strict"
"(http[s]?://[0-9a-z]+.bing\.[a-z]+[-/%.0-9a-z]*/search\?.*)"->"\1&adlt=strict"

# Yahoo - remove 'vm=...' and add 'vm=r'
"(^http://[.0-9a-z]+\.yahoo\.[a-z]+[-/%.0-9a-z]*/search)(.*)(&?)(vm=[^&]*)"->"\1\2\3"
"(^http://[.0-9a-z]+\.yahoo\.[a-z]+[-/%.0-9a-z]*/search+.*\?)"->"\1vm=r&"

You go to the ACLs - Url Lists that you are using for testing.  Go down to Modify section, then enable and write the code in the provided box. Save and Activate.

The Google regex need love to fix it, but I am not good with regex.

The issue with SSL filtering is bigger than just search results though. For example, here's a random page on Amazon. The name "Jonathan" appears 13 times on the page, but pfSense isn't blocking it. And this isn't a search results page. My understanding is that 1 instance of the banned keyword should block the page (given that I put the <jonathan> tag under the banned Keywords - not weighted)
https://www.amazon.com/Jonathan-Thomas-Sarbacher/dp/B01NBALPRR/ref=sr_1_1?ie=UTF8&qid=1496099352&sr=8-1&keywords=Jonathan

Either I have something configured incorrectly or SSL Keyword filtering has bugs that need to be fixed. I'm not sure which.

Any thoughts?[/s][/s]</jonathan>

jetberrocal

Since you are testing I guess you only have one ACL for the Phrase List and the Groups you have some but the user you are using for testing belongs to the Group were you selected the Phrase List with the Banned word.

Let say the ACL is the Default, make sure you select Default in the Group you are testing for the Phrase List box.

Just in case assume the words are case sensitive (there is a setting to make this case insensitive)

Use the sample text to follow the correct syntax.

jkrueger2020

@jetberrocal:

Since you are testing I guess you only have one ACL for the Phrase List and the Groups you have some but the user you are using for testing belongs to the Group were you selected the Phrase List with the Banned word.

Let say the ACL is the Default, make sure you select Default in the Group you are testing for the Phrase List box.

Just in case assume the words are case sensitive (there is a setting to make this case insensitive)

Use the sample text to follow the correct syntax.

Yes, I'm using all the default groups and lists. I have not created anything new. I'm not using any authentication or users. On General > Lower Case Options I have "force lower case." On General > Phrase Filter Mode I have "smart only." And just to be overly thorough, I entered the following keywords on ACLs > Phrase List > Default > Banned:

<jonathan>< Jonathan >
< jonathan >
<jonathan>As I understand it, that should have more than covered the possibilities, but the SSL page on Amazon that I referenced in my last post isn't blocked. It still gets displayed as though I hadn't attempted to filter it. I'm starting to think this is a bug.</jonathan></jonathan>

jetberrocal

The error is happening to me.

I checked the config files and it is being generated correctly.
I also disable Exceptions box in case the word falls in the exceptions.

So it seems to be an e2g 3.5.1 problem.

I guess this has to be checked on the e2g forum.

By the way I used the word <jet>. Search on Google and selected the link
https://www.google.com.pr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=13&cad=rja&uact=8&ved=0ahUKEwjq5qXotpbUAhUE6yYKHTncD78QFghvMAw&url=https%3A%2F%2Fjetprogramusa.org%2F&usg=AFQjCNGFZZgNdXX2OXYga7BOLHmLFjdZ_g</jet>

jkrueger2020

@jetberrocal:

The error is happening to me.

I checked the config files and it is being generated correctly.
I also disable Exceptions box in case the word falls in the exceptions.

So it seems to be an e2g 3.5.1 problem.

I guess this has to be checked on the e2g forum.

By the way I used the word <jet>. Search on Google and selected the link
https://www.google.com.pr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=13&cad=rja&uact=8&ved=0ahUKEwjq5qXotpbUAhUE6yYKHTncD78QFghvMAw&url=https%3A%2F%2Fjetprogramusa.org%2F&usg=AFQjCNGFZZgNdXX2OXYga7BOLHmLFjdZ_g</jet>

Thanks for helping to check this! At least I know I'm not the only one now.

I've logged the issue on the E2Guardian Google Groups: https://groups.google.com/forum/#!topic/e2guardian/NfBZ1Ux_lEY

If no one replies within a day or so, I'm going to log it as an issue on GitHub.

Thanks again!

Jonathan

jetberrocal

@jkrueger2020:

@jetberrocal:

The error is happening to me.

I checked the config files and it is being generated correctly.
I also disable Exceptions box in case the word falls in the exceptions.

So it seems to be an e2g 3.5.1 problem.

I guess this has to be checked on the e2g forum.

By the way I used the word <jet>. Search on Google and selected the link
https://www.google.com.pr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=13&cad=rja&uact=8&ved=0ahUKEwjq5qXotpbUAhUE6yYKHTncD78QFghvMAw&url=https%3A%2F%2Fjetprogramusa.org%2F&usg=AFQjCNGFZZgNdXX2OXYga7BOLHmLFjdZ_g</jet>

Thanks for helping to check this! At least I know I'm not the only one now.

I've logged the issue on the E2Guardian Google Groups: https://groups.google.com/forum/#!topic/e2guardian/NfBZ1Ux_lEY

If no one replies within a day or so, I'm going to log it as an issue on GitHub.

Thanks again!

Jonathan

If they ask for the conf files they are in /usr/local/etc/e2guardian/

pfsensation

When using Mitm I'm getting "Error too many redirects" from time to time. It can become quite frustrating when you're trying to browse, and you have to keep refreshing and going back on the page until it finally works.

Anyone else experiencing this? I vaguely remember seeing this being reported before to the E2G team, I assume it's fixed in 4.1 If it has, maybe we can back port?

Also marcelloc, have you been able to test newer versions of E2G on pfsense? Hoping it can be stable enough soon for us to update.

marcelloc

@pfsensation:

Also marcelloc, have you been able to test newer versions of E2G on pfsense? Hoping it can be stable enough soon for us to update.

Not yet, just that version that was crashing on BSD.

marcelloc

I've updated the install process to use freebsd package style instead of manual fetch from script. you can also enable the unofficial repository to install it using GUI.

See the updated install instructions on the first post of this topic.

Unfortunately, this package is still available only for AMD64 architecture.