Unofficial E2guardian package for pfSense

marcelloc

how many users connected?

pfsensation

how many users connected?

Just a handful to be honest. For now only 4 devices doing MITM, and another 3 without MITM. It's a home environment so, not that big of a load. My machine load averages are barely going past 0.20, considering what I have running and this is a dual core machine. Pretty much same exact setup minus WPAD package on 3.5.1, barely any hiccups at all after initial setup. I was able to run it for over a week without a single issue.

With 4.1, today everything suddenly started loading super slow. I restarted the service, and everything was well again. Leading me to believe this is an issue with 4.1. Have you had a chance at all to take a look at why the SSL regex wasn't working?
For now, I'm glad 4.1, at least runs on pfSense, I'm just worried about the stability. Obviously I am running this at home, so some downtime isn't that big of a deal. But others may use it in their business or something. Especially as there is no filtering solution for pfSense which supports MITM anymore. SquidGuard doesn't work.

marcelloc

@pfsensation:

Have you had a chance at all to take a look at why the SSL regex wasn't working?

I've tested th sslregex with youtube today. worked fine(with the restart service after apply).


#SSL site modifying Regular Expressions

# Enforce restricted mode in YouTube
#
"(^https://www.youtube.com)"->"https://restrict.youtube.com"
"(^https://m.youtube.com)"->"https://restrict.youtube.com"
"(^https://youtubei.googleapis.com)"->"https://restrict.youtube.com"
"(^https://youtube.googleapis.com)"->"https://restrict.youtube.com"
"(^https://www.youtube-nocookie.com)"->"https://restrict.youtube.com"
#

I'll test on a big environment next week, about 720 workstations. Hope it goes fine.

@pfsensation:

Especially as there is no filtering solution for pfSense which supports MITM anymore. SquidGuard doesn't work.

That's true. My test machines did not crashed any time but it's a single user with multiple tab test.

Can you test it on 2.4 beta too?

pfsensation

@marcelloc:

I've tested th sslregex with youtube today. worked fine(with the restart service after apply).


#SSL site modifying Regular Expressions

# Enforce restricted mode in YouTube
#
"(^https://www.youtube.com)"->"https://restrict.youtube.com"
"(^https://m.youtube.com)"->"https://restrict.youtube.com"
"(^https://youtubei.googleapis.com)"->"https://restrict.youtube.com"
"(^https://youtube.googleapis.com)"->"https://restrict.youtube.com"
"(^https://www.youtube-nocookie.com)"->"https://restrict.youtube.com"
#

It seems I've run into the same problem lol. For Youtube Restricted mode, I've created a separate config so I can easily switch it on and off as required on a per group basis. It didn't work before, however, now after saving that config again, and restarting. It seems to be functioning as expected. Currently I am unaware of if these weird things are happening because I upgraded E2Guardian over SSH, or not, as no one else seems to have replied regarding the update yet.

@marcelloc:

I'll test on a big environment next week, about 720 workstations. Hope it goes fine.

Awesome! Let us know how it goes!

@marcelloc:

Can you test it on 2.4 beta too?

Unfortunately, I'll be unable to test this thoroughly as I need my home network running. And I don't believe I can test long enough or simulate proper real world load in a virtual setup, since the crashes don't happen immediately and often take a couple of hours.

I've just had my E2Guardian service stop again, here's the errors I managed to catch this time around in the logs.

2 21:22:55	e2guardian	573	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 2 21:26:50	check_reload_status		Linkup starting re0
Jun 2 21:26:50	kernel		re0: link state changed to DOWN
Jun 2 21:26:51	check_reload_status		Reloading filter
Jun 2 21:28:31	php-fpm	68338	/index.php: Successful login for user 'admin' from: 172.16.1.1
Jun 2 21:50:35	php-fpm	89081	/index.php: Successful login for user 'admin' from: 172.16.1.8
Jun 2 21:52:24	php-fpm	81052	/pkg_edit.php: Reloading E2guardian
Jun 2 21:53:27	pfsense.kortex		nginx: 2017/06/02 21:53:27 [error] 23127#100120: *93 open() "/usr/local/www/vendor/datatable/css/jquery.dataTables.min.css" failed (2: No such file or directory), client: 172.16.1.8, server: , request: "GET /vendor/datatable/css/jquery.dataTables.min.css HTTP/1.1", host: "pfsense.kortex", referrer: "https://pfsense.kortex/e2guardian_about.php"
Jun 2 21:53:27	pfsense.kortex		nginx: 2017/06/02 21:53:27 [error] 23127#100120: *93 open() "/usr/local/www/vendor/datatable/css/jquery.dataTables.min.css" failed (2: No such file or directory), client: 172.16.1.8, server: , request: "GET /vendor/datatable/css/jquery.dataTables.min.css HTTP/1.1", host: "pfsense.kortex", referrer: "https://pfsense.kortex/e2guardian_about.php"

Seems like there's missing web config files? pfSense.kortex leads to 172.16.1.1 which is my pfsense box.

I'm also getting the errors below from nginx, I'm guessing it's due to the WPAD package?

Jun 2 21:22:50	nginx		2017/06/02 21:22:50 [emerg] 98077#100105: bind() to 172.16.1.1:80 failed (48: Address already in use)
Jun 2 21:22:50	nginx		2017/06/02 21:22:50 [emerg] 98077#100105: bind() to 172.16.1.1:80 failed (48: Address already in use)
Jun 2 21:22:50	nginx		2017/06/02 21:22:50 [emerg] 98077#100105: bind() to 172.16.1.1:80 failed (48: Address already in use)
Jun 2 21:22:50	nginx		2017/06/02 21:22:50 [emerg] 98077#100105: bind() to 172.16.1.1:80 failed (48: Address already in use)

I have followed all your installation notes, and have configured it as it's meant to be. I have changed the webconfig port to HTTPS (443) then turned off WebGUI redirection. However… I do have a NAT rule redirecting HTTP traffic through E2Guardian, as it doesn't have this function automatically. Since it doesn't seem practical to fumble around with user authentication and CA's if friends and family come along.

EDIT: Found out how to reproduce the issue and make E2Guardian crash. I am using IP based authenticated since this is a home network and I want things as simple as possible. For all members of the family, and close relatives, they have been given a static IP and have been assigned to a group. However, if anyone else that hasn't specifically been assigned to a group tries to access a blocked site, E2Guardian crashes.

Aren't unauthenticated users meant to be assiged to the default group? That's how it was in 3.5.1?

How did I test this?
On my phone from time to time, I change my mac address in order to simulate a guest device without a assigned group, and to test the default group ACL's. As soon as I go to any blocked site using that, E2Guardian crashes. Furthermore, in the block page it doesn't even show the group as "Default". Am I going to have to manually put in like 200 guest IP's into the Default group IP tab? This worked perfectly before without any issues.

In case you are wondering… This is what appears on the logs when this all happens:

Jun 3 00:33:46	kernel		pid 7075 (e2guardian), uid 106: exited on signal 11
Jun 3 00:35:03	e2guardian	23849	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 3 00:35:04	kernel		pid 24676 (e2guardian), uid 106: exited on signal 11
Jun 3 00:35:11	check_reload_status		Syncing firewall
Jun 3 00:35:11	php-fpm	23959	/pkg_edit.php: [E2guardian] - Save settings package call pr: bp: rpc:no
Jun 3 00:35:12	check_reload_status		Syncing firewall
Jun 3 00:35:14	php-fpm	24910	/pkg_edit.php: Starting E2guardian
Jun 3 00:35:14	php-fpm	37347	/pkg_edit.php: Reloading E2guardian
Jun 3 00:35:18	e2guardian	37162	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 3 00:35:19	kernel		pid 44972 (e2guardian), uid 106: exited on signal 11
Jun 3 00:35:37	e2guardian	49484	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 3 00:35:44	kernel		pid 49895 (e2guardian), uid 106: exited on signal 11
Jun 3 00:36:10	check_reload_status		Syncing firewall
Jun 3 00:36:10	php-fpm	49681	/pkg_edit.php: [E2guardian] - Save settings package call pr: bp: rpc:no
Jun 3 00:36:11	check_reload_status		Syncing firewall
Jun 3 00:36:13	php-fpm	72183	/pkg.php: Starting E2guardian
Jun 3 00:36:15	php-fpm	83374	/pkg.php: Reloading E2guardian
Jun 3 00:36:17	e2guardian	83358	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 3 00:36:30	kernel		pid 92522 (e2guardian), uid 106: exited on signal 11
Jun 3 00:36:38	check_reload_status		Syncing firewall
Jun 3 00:36:38	php-fpm	83374	/pkg_edit.php: [E2guardian] - Save settings package call pr: bp: rpc:no
Jun 3 00:36:38	check_reload_status		Syncing firewall
Jun 3 00:36:42	check_reload_status		Syncing firewall
Jun 3 00:36:42	php-fpm	96436	/pkg_edit.php: [E2guardian] - Save settings package call pr: bp: rpc:no
Jun 3 00:36:43	check_reload_status		Syncing firewall
Jun 3 00:36:44	php-fpm	99121	/pkg_edit.php: Starting E2guardian

EDIT 2: Tried manually adding my spoofed MAC address IP to the default group. And I still ended up with a crash. Even after a reinstallation… :(
Now I've created a new group and added the entire range of IP's excluding the ones already in a group, that didn't seem to do the trick either. I guess I actually manually have to input those IP's?

**EDIT 3: After some even more digging… I found out that it wasn't related to the IP's or ranges being input into E2Guardian. I narrowed the issue being down the the IP having to be in the same subnet, if it's outside then it will cause E2Guardian to crash.

For example my DHCP range is from 172.16.1.2-172.16.2.255. Any 2.x IP, going to any block page would cause E2Guardian to crash. This was not the behaviour in 3.5.1, no such issue. I forced my IP to be 172.16.1.50 and this time E2Guardian didn't crash on the block page which 100% confirms this.**

I give up lol… Made my range only one subnet. And now having the issue again. What kinda weird bug is this? It seems very likely that it's a problem with the authentication. Since it ONLY crashes when the devices "group" cannot be found. And it doesn't automatically put that device in the default group or anything.

marcelloc

This missing files on about page should not be therw. It's from postfix package I'll fix it.

The socket already in use is a service recall pfSensedo on wpad every config save. I'll try to ignore these calls from system and restart only from gui save.

pfsensation

@marcelloc:

This missing files on about page should not be therw. It's from postfix package I'll fix it.

The socket already in use is a service recall pfSensedo on wpad every config save. I'll try to ignore these calls from system and restart only from gui save.

Check my edit, I have found a way to reproduce the crashes of E2Guardian easily.

marcelloc

@pfsensation:

Check my edit, I have found a way to reproduce the crashes of E2Guardian easily.

I've update my ports with current e2guardian code and removed the debug patches I've created while was trying to fix ILLEGAL INSTRUCTION bug we found before.

At least on 2.4 it's stable. I"ll update the binaries to 2.3 and also fix the about page.

marcelloc

pkg version 0.1.1 updates for 2.3 and 2.4 8)

No crashes on my tests until now.

pfsensation

@marcelloc:

pkg version 0.1.1 updates for 2.3 and 2.4 8)

No crashes on my tests until now.

I've upgraded to your latest package and I'm on pfsense 2.3.4, I'm still getting those crashes. :(

Have you been able to recreate the issue? Why is it giving so many issues on 2.3.4 if it works fine on 2.4?
Have you tried using multiple IP's? Currently I've assigned every single IP in my dhcp range to a group and I'm still having problems.

Could it be something related to my custom block page? Since it also shows filtergroup, host name and IP address? I posted the source code here. I'll have to try with the default page later on and see if that makes a difference. Although I don't think that's the cause.

marcelloc

I'm using your page and also testing on 2.3 64 bits.

pfsensation

@marcelloc:

I'm using your page and also testing on 2.3 64 bits.

Alright let me know how it goes. That's basically the same setup as mine. By the way 4.1 completely fixed the error too many redirects issue. I haven't had that error at least since updating from 3.5.1 :)

marcelloc

How exactly can I reproduce the error?

Default install
Ip address authentication
squid as parent proxy(I'm using lan address to be able to test user authentication too)
two groups(default + admin)
home network on default group(192.168.1.0/255.255.255.0)
specific devices ips on admin group
ssl interception on both groups
sslregex on youtube urls

Is there a specific site or rule or deny rule that is crashing you box?

I'm also trying:

opening multiple browser tabs
refreshing online new sites(that refreshes itself every x seconds)
stopping site fetch during browser load, then loading another site, etc

marcelloc

Did you saw e2g binaries being updated during reinstall process? Pkg info should show e2guardian-4.1.1_1 pkg

Cino

I figured I would give E2guardian4 a spin and anytime something is blocked (word phase or blocked domain is what i tested so far), the daemon would crash. I think I've figure what is causing it, the block page.

When I used the block page provided by pfsensation, it crashes after it's display

pid 70160 (e2guardian), uid 65534: exited on signal 11

When I erase the block page, it doesn't put the default one back in there. So I graded the template from e2guardian github but that one doesn't want to load (haven't troubleshooting it yet)


[2.3.5-DEVELOPMENT][root@pfsense.home.lan]/root: /usr/local/etc/rc.d/e2guardian.sh restart
kern.ipc.somaxconn: 16384 -> 16384
kern.maxfiles: 131072 -> 131072
kern.maxfilesperproc: 104856 -> 104856
kern.threads.max_threads_per_proc: 4096 -> 4096
e2guardian not running? (check /var/run/e2guardian.pid).
Starting e2guardian.
Syntax error at first: -REASONGIVEN-
Error reading default HTML Template file: /usr/local/share/e2guardian/languages/ukenglish/template.html
Error opening filter group config: /usr/local/etc/e2guardian/e2guardianf1.conf
Error in reading filter group files
Error reading filter group conf file(s).
Error parsing the e2guardian.conf file or other e2guardian configuration files
/usr/local/etc/rc.d/e2guardian.sh: WARNING: failed to start e2guardian

https://github.com/e2guardian/e2guardian/blob/v4.1/contrib/template.html


		<title>E2Guardian - Access Denied</title>

| 
				Access Denied!
			 |			
| 

				-USER- 
			 |			

					Your Organization
				 |				 
					Access to the page:

					[-URL-](-URL-)

					 ... has been denied for the following reason:

						 -REASONGIVEN- 

						You are seeing this error because what you attempted to access appears to contain,
						or is labeled as containing, material that has been deemed inappropriate.

						If you have any questions contact your [Network Administrator](mailto:-ADMIN-?subject=access%20denied&body=-URL-%20--%20-REASON-). 
					 Powered by [E2Guardian](http://www.e2guardian.org) 

						 [![Valid HTML 4.01 Transitional](http://www.w3.org/Icons/valid-html401)](http://validator.w3.org/check?uri=referer) 
						 [![Valid CSS!](http://jigsaw.w3.org/css-validator/images/vcss)](http://jigsaw.w3.org/css-validator/) 

				 |

Since that wouldn't load, I took the spanish template and used google to translate it for me. No more crashing when pages are blocked.
/usr/local/share/e2guardian/languages/spanish/template.html


<title>E2guardian Access Denied</title>

<center>

	 **Access denied!**  |

	 **-USER- **  |

	 YOUR COMPANY  |	 
	 Access to the website

	[-URL-](-URL-)

	 Has been denied for the following reason:

	 **-REASONGIVEN-**
	 You are seeing this error message because the page you are

attempts to access contains, or is classified as containing,

material that is considered inappropriate.

If you have questions, please contact 
 with the System Administrator or the Network Administrator.

	 Powered by [e2guardian](http://www.e2guardian.org?block)  |

</center>

marcelloc

Ok. Try to add on pfsensation error page the html tags at the beginning and at the end. This is the way I'm using here.

custom_error_page.txt

Cino

Still crashes for me using the attached file. I took out the image too, and that didn't help. I'll have to revisit it later today or tomorrow.

pfsensation

@Cino:

Still crashes for me using the attached file. I took out the image too, and that didn't help. I'll have to revisit it later today or tomorrow.

I've been very busy today so I didn't get a chance to play around anymore. But if it really is to do with my block page causing the crash. Look at what is different from the official, namely, my block page shows client Group and Host name. Remove those and see if it crashes, if it doesn't that could be it. Something wrong with "-Filtergroup-" or "-Host-".

Edit: My page has a bypass link too. So it may be any of those tbh, bypass link creates a code. Judging by what Marcelloc was facing with illegal chars etc. That could somehow be related.

pfsensation

Not able to go on my pc right now but from my phone. I removed the code below.


#### Your details are below:

      _-USER- -HOST-_ _-FILTERGROUP-_ _-IP-_

      [Acknowledge](-BYPASS-)

It doesn't seem to be crashing anymore. I'll test a little more when possible. Not sure exactly which place holder is causing the issue, but it most likely maybe the filter group. Since the filter group doesn't show.

marcelloc

@pfsensation:

Not able to go on my pc right now but from my phone. I removed the code below.
 

#### Your details are below:

      _-USER- -HOST-_ _-FILTERGROUP-_ _-IP-_
      

      [Acknowledge](-BYPASS-)
It doesn't seem to be crashing anymore. I'll test a little more when possible. Not sure exactly which place holder is causing the issue, but it most likely maybe the filter group. Since the filter group doesn't show.

As I don't use bypass key, try just removing it from page. Maybe this is the difference from our config.

marcelloc

Also I identified why the apply button was not reloading completely the daemon. E2guardian has 3 ways to reload it without killing the process and the option I was using was the lightest one. Next gui version will have this fix.

If we can confirm that the -BYPASS- is crashing the daemon, I can open another ticket on e2guardian repo.