Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Unofficial E2guardian package for pfSense

    Cache/Proxy
    70
    1.2k
    1.1m
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • marcellocM
      marcelloc
      last edited by

      I've opened a new issue on e2g git

      
Client: START------------------------------- Line: 1011 Function: dbshowheader
 Line: 1018 Function: dbshowheadereader bool: POST http://1xx.1yy.zz3.4/dout.aspx?s=16434986&p=10000003&client=DynGate HTTP/1.0
 Line: 1018 Function: dbshowheadereader bool: Cache-Control: no-cache
 Line: 1018 Function: dbshowheadereader bool: Pragma: no-cache
 Line: 1018 Function: dbshowheadereader bool: Content-Type: application/octet-stream
 Line: 1018 Function: dbshowheadereader bool: Accept: */*
 Line: 1018 Function: dbshowheadereader bool: User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; DynGate)
 Line: 1018 Function: dbshowheadereader bool: Content-Transfer-Encoding: binary
 Line: 1018 Function: dbshowheadereader bool: Connection: keep-alive
 Line: 1018 Function: dbshowheadereader bool: Content-Length: 68
 Line: 1018 Function: dbshowheadereader bool: Host: 1xx.1yy.zz3.4
Client: END------------------------------- Line: 1023 Function: dbshowheader
54837  got past line 2350 proxy header out  Line: 2561 Function: handleConnection
54837  exchange_timeout is 20000 Line: 2562 Function: handleConnection
tunnel tw poll returned ok:1
Start of tunnel loop: throughput:4942 target:-1
getLine !SSL read into buffer; bufflen: 0
getLine terminate string !SSL: 0
firstime: header:in after getLine  Line: 2098 Function: in
firstime: header:in after getLine: rc: 0 truncated: 1 Line: 2103 Function: in
6982468823791959581Call : from HTTPHeader.cpp to dbshowheader but header is empty Line: 1028 Function: dbshowheader
Segmentation fault

      At least here with e2g compiled with debug, it crashes when empty header happens

      If you want to try the same debug version to see if it crashes on the same place, follow this:

      
fetch https://github.com/marcelloc/Unofficial-pfSense-packages/raw/master/repo/e2guardian-4.1.1_debug.txz
fetch https://github.com/marcelloc/Unofficial-pfSense-packages/raw/master/repo/e2guardian-4.1.1_1.txz

      To force debug install, run

      pkg add -f e2guardian-4.1.1_debug.txz
killall e2guardian && /usr/local/sbin/e2guardian -N

      This must be done using console to see debug output

      To back to non debug binaries:

      pkg add -f e2guardian-4.1.1_1.txz
killall  e2guardian && /usr/local/sbin/e2guardian

      Treinamentos de Elite: http://sys-squad.com

      Help a community developer! ;D

      1 Reply Last reply Reply Quote 0
      • P
        pfsensation
        last edited by

        I realised the sudden memory spikes too. I had 350MB, didn't worry too much as I assumed it was for cache + I have 2GB total.

        I had another crash today after 2 days, it's better than the nearly instant crash every 30 minites - 2 hours I was getting before.

        Jun 7 02:56:44	check_reload_status		Syncing firewall
Jun 7 02:56:46	php-fpm	18746	/pkg.php: Restarting e2g by sending -Q action to e2g binaries
Jun 7 02:56:46	php-fpm	19894	/pkg.php: Restarting e2g by sending -Q action to e2g binaries
Jun 7 02:56:48	php-fpm	20300	/status_services.php: The command '/usr/local/etc/rc.d/e2guardian.sh stop' returned exit code '1', the output was 'kern.ipc.somaxconn: 16384 -> 16384 kern.maxfiles: 131072 -> 131072 kern.maxfilesperproc: 104856 -> 104856 kern.threads.max_threads_per_proc: 4096 -> 4096 e2guardian not running? (check /var/run/e2guardian.pid).'
Jun 7 02:56:54	e2guardian	20294	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 02:56:54	e2guardian	19797	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 02:56:55	e2guardian	23430	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 02:56:55	e2guardian	19797	I seem to be running already!
Jun 7 02:56:55	e2guardian	23430	I seem to be running already!
Jun 7 02:56:55	root		/usr/local/etc/rc.d/e2guardian.sh: WARNING: failed to start e2guardian
Jun 7 02:57:52	check_reload_status		Syncing firewall
Jun 7 02:57:52	php-fpm	56014	/pkg_edit.php: [E2guardian] - Save settings package call pr:1 bp: rpc:no
Jun 7 02:57:52	check_reload_status		Syncing firewall
Jun 7 02:57:55	php-fpm	78906	/pkg.php: Restarting e2g by sending -Q action to e2g binaries
Jun 7 02:57:55	php-fpm	80648	/pkg.php: Restarting e2g by sending -Q action to e2g binaries
Jun 7 02:58:00	e2guardian	81133	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 02:58:00	e2guardian	80510	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 02:58:01	e2guardian	80510	I seem to be running already!
Jun 7 03:01:00	root		rc.update_bogons.sh is starting up.
Jun 7 03:01:00	root		rc.update_bogons.sh is sleeping for 31584
Jun 7 03:14:47	check_reload_status		Syncing firewall
Jun 7 03:14:47	php-fpm	9104	/pkg_edit.php: [E2guardian] - Save settings package call pr:1 bp: rpc:no
Jun 7 03:14:47	check_reload_status		Syncing firewall
Jun 7 03:14:49	php-fpm	19850	/pkg.php: Restarting e2g by sending -Q action to e2g binaries
Jun 7 03:14:55	e2guardian	23043	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 03:24:23	check_reload_status		Syncing firewall
Jun 7 03:24:23	php-fpm	60117	/pkg_edit.php: [E2guardian] - Save settings package call pr:1 bp: rpc:no
Jun 7 03:24:23	check_reload_status		Syncing firewall
Jun 7 03:24:25	php-fpm	11356	/pkg.php: Restarting e2g by sending -Q action to e2g binaries
Jun 7 03:24:25	php-fpm	15238	/pkg.php: Restarting e2g by sending -Q action to e2g binaries
Jun 7 03:24:30	e2guardian	15501	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 03:24:31	e2guardian	15228	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 03:24:31	e2guardian	15228	I seem to be running already!
Jun 7 03:24:39	e2guardian	21488	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 03:31:24	check_reload_status		Syncing firewall
Jun 7 03:31:24	php-fpm	64424	/pkg_edit.php: [E2guardian] - Save settings package call pr:1 bp: rpc:no
Jun 7 03:31:24	check_reload_status		Syncing firewall
Jun 7 03:31:26	php-fpm	41060	/pkg.php: Restarting e2g by sending -Q action to e2g binaries
Jun 7 03:31:32	e2guardian	44437	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 03:31:46	e2guardian	49314	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 03:31:46	php-fpm	46997	/pkg.php: Restarting e2g by sending -Q action to e2g binaries
Jun 7 03:31:52	e2guardian	49618	Reporting_level is : 0 file /usr/local/etc/e2guardian/e2guardianf3.conf
Jun 7 11:47:24	root		rc.update_bogons.sh is beginning the update cycle.
Jun 7 11:47:26	root		Bogons V4 file downloaded: 15 addresses added.
Jun 7 11:47:26	root		Bogons V4 file downloaded: 16 addresses deleted.
Jun 7 11:47:27	root		Bogons V6 file downloaded: 302 addresses added.
Jun 7 11:47:27	root		Bogons V6 file downloaded: 69 addresses deleted.
Jun 7 11:47:27	root		rc.update_bogons.sh is ending the update cycle.
Jun 7 20:39:37	kernel		pid 49681 (e2guardian), uid 106: exited on signal 11
        1 Reply Last reply Reply Quote 0
        • P
          pfsensation
          last edited by

          Any updates on this? Is there anyway I can downgrade for now?

          4.1 is giving me a lot of hassle with crashing. Maybe you could make a script to automatically restart it if it crashes Marcello?

          1 Reply Last reply Reply Quote 0
          • marcellocM
            marcelloc
            last edited by

            I have a watchdog script. I'll share here when I get home.

            Treinamentos de Elite: http://sys-squad.com

            Help a community developer! ;D

            1 Reply Last reply Reply Quote 0
            • J
              jetberrocal
              last edited by

              @marcelloc:

              I have a watchdog script. I'll share here when I get home.

              Let say the e2g has the watchdog script working.

              From the perspective of the clients, is e2g as of now, doing all the features OK and they will not notice the crashes?

              I want to know because if the features are not working OK then I will have to wait for the fixes to be able to use it in production.  Also if the crashes can be noticed by the clients then I have to wait for the fixes to be able to use it in production.  Maybe the crashes can be tolerated if they are noticed but in sporadic instances.

              1 Reply Last reply Reply Quote 0
              • marcellocM
                marcelloc
                last edited by

                I guess the current crashes are related to a old exit status 11 that happens for a Long Time since dansguandian but with current thread version, the main process dies too. The issue is related with empty headers and connection that doesn't answer.

                Treinamentos de Elite: http://sys-squad.com

                Help a community developer! ;D

                1 Reply Last reply Reply Quote 0
                • marcellocM
                  marcelloc
                  last edited by

                  whatchdog script for e2guardian

                  #!/bin/sh
for a in 5 10 15 20 25 30 35 40 45 50 55
do
if [ -f /var/run/e2guardian.pid ];then
 cat /var/run/e2guardian.pid | xargs ps
 if [ $? -ne 0 ]; then
   /usr/local/etc/rc.d/e2guardian.sh start
   tail -50 /var/log/e2guardian/access.log >> /var/log/e2guardian/last.log
   echo "`date` start" >> /var/log/e2guardian/start.log
 fi
fi
sleep 5
done

                  Run it every minute via cron. /var/log/e2guardian/start.log will log the crashes.

                  Treinamentos de Elite: http://sys-squad.com

                  Help a community developer! ;D

                  1 Reply Last reply Reply Quote 0
                  • P
                    pfsensation
                    last edited by

                    @marcelloc:

                    whatchdog script for e2guardian

                    #!/bin/sh
for a in 5 10 15 20 25 30 35 40 45 50 55
do
if [ -f /var/run/e2guardian.pid ];then
 cat /var/run/e2guardian.pid | xargs ps
 if [ $? -ne 0 ]; then
   /usr/local/etc/rc.d/e2guardian.sh start
   tail -50 /var/log/e2guardian/access.log >> /var/log/e2guardian/last.log
   echo "`date` start" >> /var/log/e2guardian/start.log
 fi
fi
sleep 5

                    Run it every minute via cron. /var/log/e2guardian/start.log will log the crashes.

                    Just tried setting this up now. When I copy and paste your code into Cron, it becomes all jumbled. Here's an example of what happened :

                    #!/bin/sh
for a in 5 10 15 20 25 30 35 40 45 50 55
do
if [ -f /var/run/e2guardian.pid ];then
 cat /var/run/e2guardian.pid | xargs ps
 if [ $? -ne 0 ]; then
   /usr/local/etc/rc.d/e2guardian.sh start
   tail -50 /var/log/e2guardian/access.log >> /var/log/e2guardian/last.log
   echo "`date` start" >> /var/log/e2guardian/start.log
 fi
fi
sleep 5

                    Went from the above code, to the below code

                    #!/bin/shfor a in 5 10 15 20 25 30 35 40 45 50 55doif [ -f /var/run/e2guardian.pid ];then cat /var/run/e2guardian.pid | xargs ps if [ $? -ne 0 ]; then   /usr/local/etc/rc.d/e2guardian.sh start   tail -50 /var/log/e2guardian/access.log >> /var/log/e2guardian/last.log   echo "`date` start" >> /var/log/e2guardian/start.log fifisleep 5

                    Will this cause any problems?

                    And also, since this cron job is running every minute. What if the case is that E2Guardian crashes just after the cron job is run, then you have to wait an entire minute before E2Guardian is put back up. It could be incredibly annoying in a production environment.

                    1 Reply Last reply Reply Quote 0
                    • P
                      pfsensation
                      last edited by

                      @jetberrocal:

                      @marcelloc:

                      I have a watchdog script. I'll share here when I get home.

                      Let say the e2g has the watchdog script working.

                      From the perspective of the clients, is e2g as of now, doing all the features OK and they will not notice the crashes?

                      I want to know because if the features are not working OK then I will have to wait for the fixes to be able to use it in production.  Also if the crashes can be noticed by the clients then I have to wait for the fixes to be able to use it in production.  Maybe the crashes can be tolerated if they are noticed but in sporadic instances.

                      Pretty sure at some point it will be noticed, and it will become annoying. If the cause is actually an empty header, and someone is accessing a site which is causing the empty headers, you may end up with more crashes at once. And then you have to wait a minute for it to come back etc etc… I use this at home, and I can say that 4.1 is extremely fast, and nice to use in general. But the crashes for now are making it more hassle than it's worth. I'm stuck in between for now, 3.5.1 gave me redirect issues, and sometimes some slowness (not a huge deal) but redirect issue was annoying. But 4.1 is very very quick, fixes the redirect issues and sudden lag but is prone to crashes. And I guess it has some sort of memory leak? I hope the watchdog script at least takes some frustration out. D:

                      @Marcelloc, I have a NAT rule to redirect port 80 to my pfsense box IP and port 8080. Should I do the same for 443? Or will that completely break it? I don't mind being having to install the certificate, but is there a way to just force all the traffic through E2Guardian. In some cases, such as Android, it seems other methods such as WPAD etc, don't work.

                      1 Reply Last reply Reply Quote 0
                      • J
                        jetberrocal
                        last edited by

                        Pfsensation:

                        You have to add spaces where they stick together.

                        Example
                        55doif -> 55 do if

                        It seems you lost carriege returns. But whats important is spaces between commands and parameters.

                        1 Reply Last reply Reply Quote 0
                        • P
                          pfsensation
                          last edited by

                          @jetberrocal:

                          Pfsensation:

                          You have to add spaces where they stick together.

                          Example
                          55doif -> 55 do if

                          It seems you lost carriege returns. But whats important is spaces between commands and parameters.

                          Thanks for that. I will go edit the cron now

                          1 Reply Last reply Reply Quote 0
                          • marcellocM
                            marcelloc
                            last edited by

                            Create a file on filesystem with the script and then call the file on cron.

                            Treinamentos de Elite: http://sys-squad.com

                            Help a community developer! ;D

                            1 Reply Last reply Reply Quote 0
                            • J
                              jetberrocal
                              last edited by

                              @Marcelloc, I have a NAT rule to redirect port 80 to my pfsense box IP and port 8080. Should I do the same for 443? Or will that completely break it? I don't mind being having to install the certificate, but is there a way to just force all the traffic through E2Guardian. In some cases, such as Android, it seems other methods such as WPAD etc, don't work.

                              That rule will brake wpad retrival. Did you add that before testing Androids?

                              1 Reply Last reply Reply Quote 0
                              • P
                                pfsensation
                                last edited by

                                @jetberrocal:

                                @Marcelloc, I have a NAT rule to redirect port 80 to my pfsense box IP and port 8080. Should I do the same for 443? Or will that completely break it? I don't mind being having to install the certificate, but is there a way to just force all the traffic through E2Guardian. In some cases, such as Android, it seems other methods such as WPAD etc, don't work.

                                That rule will brake wpad retrival. Did you add that before testing Androids?

                                Hmm how can I get that setup with WPAD still working? Somehow WPAD seems to be working. My pc is set to automatically detect proxy settings and it does, however it didn't work on IOS maybe this is it.

                                I tested with this rule on android, works for http traffic. It'll successfully go to the block page if the website isn't allowed.

                                1 Reply Last reply Reply Quote 0
                                • marcellocM
                                  marcelloc
                                  last edited by

                                  Create the rule and try to fetch the wpad file. If it fetches, the is no conflict.

                                  Treinamentos de Elite: http://sys-squad.com

                                  Help a community developer! ;D

                                  1 Reply Last reply Reply Quote 0
                                  • P
                                    pfsensation
                                    last edited by

                                    @marcelloc:

                                    Create the rule and try to fetch the wpad file. If it fetches, the is no conflict.

                                    Able to fetch it no problems, as long as I type http, since web config is on https.

                                    1 Reply Last reply Reply Quote 0
                                    • P
                                      pfsensation
                                      last edited by

                                      @marcelloc:

                                      Create a file on filesystem with the script and then call the file on cron.

                                      Didn't work at first, but added "Done" at the end of the script. Now it seems to be working fine.

                                      1 Reply Last reply Reply Quote 0
                                      • marcellocM
                                        marcelloc
                                        last edited by

                                        @pfsensation:

                                        @marcelloc:

                                        Create a file on filesystem with the script and then call the file on cron.

                                        Didn't work at first, but added "Done" at the end of the script. Now it seems to be working fine.

                                        Bad copy and paste. Sorry. I've fixed the post

                                        Treinamentos de Elite: http://sys-squad.com

                                        Help a community developer! ;D

                                        1 Reply Last reply Reply Quote 0
                                        • marcellocM
                                          marcelloc
                                          last edited by

                                          Can you test if a full MITM e2guardian setup with no exceptions crashes? current binaries from package are not on the latest version but it's close to it. If you can confirm a crash, I'LL build with current latest code.

                                          Treinamentos de Elite: http://sys-squad.com

                                          Help a community developer! ;D

                                          1 Reply Last reply Reply Quote 0
                                          • marcellocM
                                            marcelloc
                                            last edited by

                                            I found an workaround util 4.1.1 gets fixed 8)

                                            • Configure squid to intercept SSL connections with splice all mode (this checks only remote certificate)

                                            • Configure e2guardian Parent proxy Settings with your squid ssl interface configured.

                                            Testing with steps I know that crashes the daemon but it's still alive with and without MITM.

                                            Treinamentos de Elite: http://sys-squad.com

                                            Help a community developer! ;D

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post