Unofficial E2guardian package for pfSense
-
So I've just had E2Guardian crash on me again… This time I have snapped all the logs. Here's the last few lines on the access.log
18.04.15 16:19:11 172.16.1.6 172.16.1.6 https://r1---sn-8pgbpohxqp5-ajtl.googlevideo.com:443 *TRUSTED* Site match: googlevideo.com CONNECT 138829 0 - 3 200 - 172.16.1.6 wireless_users - - - - - 18.04.15 16:19:13 172.16.1.3 172.16.1.3 https://use.fontawesome.com - - 5060 0 - 2 0 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:15 172.16.1.3 172.16.1.3 http://push.bitdefender.net/poll?push_id=01c53a8c-ce4e-4a87-948a-bb420093349e - GET 16 0 - 2 200 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:18 172.16.1.6 172.16.1.6 https://m.youtube.com/api/stats/watchtime?ns=yt&el=detailpage&cpn=Bnm_BnpPYGGmvaw6&docid=9YZgy1dpesY&ver=2&referrer=https%3A%2F%2Fm.youtube.com%2Fwatch%3Fv%3DQFe1PcLZGQE%26itct%3DCBYQpDAYAiITCPGnsKHIvNoCFRekFQodY%252BkPo1IKbWFkcyBsZXdpcw%253D%253D%26lact%3D12&cmt=155.448&plid=AAVp5J4QQ2VIWDNA&ei=RG3TWvv3LdbzWtWSgZgM&fmt=134&fs=0&rt=49.006&of=sidReqaeP4yRcfYsI6hTkw&euri&lact=39362&cl=192651238&state=playing&vm=CAEQABgE&volume=100&c=MWEB&cver=1.20180412&cplayer=UNIPLAYER&cbrand=generic&cbr=Firefox&cbrver=59.0&cmodel=android%204.4&cos=Android&cosver=4.4.4&cplatform=TABLET&hl=en_GB&cr=GB&len=796.261&rtn=89&feature=related&afmt=251&idpj=-4&ldpj=-25&rti=49&muted=0&st=136.465&et=155.448&conn=3 - GET 0 0 - 3 204 - 172.16.1.6 wireless_users - - - - - 18.04.15 16:19:20 172.16.1.6 172.16.1.6 https://m.youtube.com/api/stats/qoe?event=streamingstats&fmt=134&afmt=251&cpn=Bnm_BnpPYGGmvaw6&ei=RG3TWvv3LdbzWtWSgZgM&el=detailpage&docid=9YZgy1dpesY&ns=yt&fexp=23707874%2C23708904%2C23708906%2C23708910%2C23710476%2C23712544%2C23712994%2C23713711%2C23716256%2C23716972%2C23718632%2C23720634%2C23721898%2C23723618%2C23723926%2C23724493%2C23725949%2C23726426%2C23728352%2C23731308%2C23731896%2C23732330%2C23732385%2C23732729%2C23733751%2C23734535%2C23735135%2C23735347%2C23736055%2C23736249%2C23736874%2C9422596%2C9449243%2C9475691%2C9477414%2C9485000%2C9486081%2C9487194%2C9489759&cl=192651238&seq=4&c=MWEB&cver=1.20180412&cplayer=UNIPLAYER&cbrand=generic&cbr=Firefox&cbrver=59.0&cmodel=android%204.4&cos=Android&cosver=4.4.4&cplatform=TABLET&vps=50.154:PL&bwm=50.154:3569482:1.432&bwe=50.154:1955673&cmt=50.154:156.525&bh=50.154:51.736 - POST 0 0 - 3 204 - 172.16.1.6 wireless_users - - - - - 18.04.15 16:19:21 172.16.1.3 172.16.1.3 http://push.bitdefender.net/poll?push_id=6a8e596b-71f5-4665-91c3-b18339d68777 - GET 16 0 - 2 200 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:21 172.16.1.6 172.16.1.6 https://endpoint1.collection.eu.sumologic.com/receiver/v1/http/ZaVnC4dhaV3f1zhS5xhQcAjcICG2O_Vtd8mu-39iqkdF1iAhhFWHpSA5bUBK_SrbgvG8GdcqjH_DGs-NplOisVzYbr-qoEfcff0MhUJH4RV79DiuIFEF0w== - POST 0 0 - 3 200 - 172.16.1.6 wireless_users - - - - - 18.04.15 16:19:22 172.16.1.3 172.16.1.3 https://easylist.to - - 148068 0 - 2 0 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:25 172.16.1.3 172.16.1.3 http://push.bitdefender.net/poll?push_id=01c53a8c-ce4e-4a87-948a-bb420093349e - GET 16 0 - 2 200 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:29 172.16.1.7 172.16.1.7 https://play.googleapis.com *TRUSTED* Site match: play.googleapis.com - 1166 0 - 4 0 - 172.16.1.7 wireless_users - - - - - 18.04.15 16:19:29 172.16.1.6 172.16.1.6 https://s3-eu-west-1.amazonaws.com/smhw-frontend-prod/VERSION.json - GET 0 0 - 3 304 - 172.16.1.6 wireless_users - - - - - 18.04.15 16:19:31 172.16.1.3 172.16.1.3 http://push.bitdefender.net/poll?push_id=6a8e596b-71f5-4665-91c3-b18339d68777 - GET 16 0 - 2 200 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:34 172.16.1.3 172.16.1.3 http://go.microsoft.com/fwlink/?LinkID=252669&clcid=0x409 - POST 0 0 - 2 302 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:35 172.16.1.3 172.16.1.3 http://push.bitdefender.net/poll?push_id=01c53a8c-ce4e-4a87-948a-bb420093349e - GET 16 0 - 2 200 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:36 172.16.1.6 172.16.1.6 https://m.youtube.com/youtubei/v1/log_event?alt=json&key=8Q4STEHLGCilw_Y9_11qcW8 - POST 48 0 - 3 200 - 172.16.1.6 wireless_users - - - - - 18.04.15 16:19:41 172.16.1.3 172.16.1.3 http://push.bitdefender.net/poll?push_id=6a8e596b-71f5-4665-91c3-b18339d68777 - GET 16 0 - 2 200 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:45 172.16.1.3 172.16.1.3 http://push.bitdefender.net/poll?push_id=01c53a8c-ce4e-4a87-948a-bb420093349e - GET 16 0 - 2 200 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:45 172.16.1.3 172.16.1.3 https://nexus.officeapps.live.com:443 - CONNECT 152 0 - 2 200 - 172.16.1.3 Forid - - - - - 18.04.15 16:19:46 172.16.1.3 172.16.1.3 https://beacons5.gvt3.com - - 3362 0 - 2 0 - 172.16.1.3 Forid - - - - -We've got a tasty collection of enormous URL's, post requests and a tonne of random telemetry. @Marcelloc does this help?
-
Can you run e2g with debug binaries, browse these latest sites to get the error message?
what authentication are you using to get you user on logs?
-
Can you run e2g with debug binaries, browse these latest sites to get the error message?
what authentication are you using to get you user on logs?
I just had another crash, see below:
18.04.15 19:08:18 172.16.1.3 172.16.1.3 https://client.wns.windows.com - - 4430 0 - 2 0 - 172.16.1.3 Forid - - - - - 18.04.15 19:09:05 172.16.1.17 172.16.1.17 http://www.msftconnecttest.com/connecttest.txt - GET 22 -40 - 2 200 - 172.16.1.17 Forid - - - - - 18.04.15 19:09:12 172.16.1.17 172.16.1.17 http://cdn.content.prod.cms.msn.com/singletile/summary/alias/experiencebyname/today?market=en-GB&source=appxmanifest&tenant=amp&vertical=finance - GET 1008 0 - 2 200 - 172.16.1.17 Forid - - - - - 18.04.15 19:09:12 172.16.1.17 172.16.1.17 http://cdn.content.prod.cms.msn.com/singletile/summary/alias/experiencebyname/today?market=en-GB&source=appxmanifest&tenant=amp&vertical=sports - GET 1137 0 - 2 200 - 172.16.1.17 Forid - - - - - 18.04.15 19:09:13 172.16.1.17 172.16.1.17 http://cdn.content.prod.cms.msn.com/singletile/summary/alias/experiencebyname/today?market=en-GB&source=appxmanifest&tenant=amp&vertical=news - GET 1248 0 - 2 200 - 172.16.1.17 Forid - - - - - 18.04.15 19:09:16 172.16.1.17 172.16.1.17 http://cdn.content.prod.cms.msn.com/singletile/summary/alias/experiencebyname/today?market=en-GB&source=appxmanifest&tenant=amp&vertical=sports - GET 1137 0 - 2 200 - 172.16.1.17 Forid - - - - - 18.04.15 19:09:16 172.16.1.17 172.16.1.17 http://cdn.content.prod.cms.msn.com/singletile/summary/alias/experiencebyname/today?market=en-GB&source=appxmanifest&tenant=amp&vertical=finance - GET 1008 0 - 2 200 - 172.16.1.17 Forid - - - - - 18.04.15 19:09:16 172.16.1.17 172.16.1.17 http://cdn.content.prod.cms.msn.com/singletile/summary/alias/experiencebyname/today?market=en-GB&source=appxmanifest&tenant=amp&vertical=news - GET 1248 0 - 2 200 - 172.16.1.17 Forid - - - - - 18.04.15 19:09:21 172.16.1.17 172.16.1.17 https://client-office365-tas.msedge.net - - 12400 0 - 2 0 - 172.16.1.17 Forid - - - - - 18.04.15 19:09:21 172.16.1.17 172.16.1.17 https://config.edge.skype.com *TRUSTED* Site match: skype.com - 6894 0 - 2 0 - 172.16.1.17 Forid - - - - - 18.04.15 19:09:41 - 172.16.1.10 https://mmg-fna.whatsapp.net:443 *TRUSTED* Site match: whatsapp.net CONNECT 358 0 - 1 200 - 172.16.1.10 Default - - - - - 18.04.15 19:09:58 172.16.1.17 172.16.1.17 https://client.wns.windows.com - - 4430 0 - 2 0 - 172.16.1.17 Forid - - - - - 18.04.15 19:10:05 - 172.16.1.14 http://clients3.google.com/generate_204 *TRUSTED* Site match: clients3.google.com GET 0 0 - 1 204 - 172.16.1.14 Default - - - - - 18.04.15 19:10:05 - 172.16.1.14 http://clients3.google.com/generate_204 *TRUSTED* Site match: clients3.google.com GET 0 0 - 1 204 - 172.16.1.14 Default - - - - - 18.04.15 19:10:07 172.16.1.17 172.16.1.17 https://nexus.officeapps.live.com:443 - CONNECT 2904 0 - 2 200 - 172.16.1.17 Forid - - - - - 18.04.15 19:10:07 172.16.1.17 172.16.1.17 https://nexusrules.officeapps.live.com:443 - CONNECT 2904 0 - 2 200 - 172.16.1.17 Forid - - - - - 18.04.15 19:10:07 - 172.16.1.14 https://api.samsungcloud.com:443 - CONNECT 6400 0 - 1 200 - 172.16.1.14 Default - - - - - 18.04.15 19:10:08 - 172.16.1.14 https://api.samsungcloud.com:443 - CONNECT 383 0 - 1 200 - 172.16.1.14 Default - - - - - 18.04.15 19:10:12 172.16.1.17 172.16.1.17 https://v20.vortex-win.data.microsoft.com - - 4215 0 - 2 0 - 172.16.1.17 Forid - - - - -I am using IP authentication, with groups. I've got static IP for anyone who needs any "special" ACL's added. I will install the debug binaries now.
-
@Maracello, I have installed the debug binaries and it's made the mystery deeper! I have tried all the last sites in the access.log and had no crashes… I would set the debug logs to be written to a file but I can tell that they would become huge in a matter of minutes.
Have you got any other suggestions?
-
Leave it open without redirect on a ssh console.
When it crashes, latest info on ssh console will be the relevant information to send to e2guardian dev team.
-
Leave it open without redirect on a ssh console.
When it crashes, latest info on ssh console will be the relevant information to send to e2guardian dev team.
Got it, thanks bro! I'll keep a SSH session open with this on my ESXi VM :)
EDIT:
GOTCHA!! Found the problem, it's the HOSTS in the block page. Removing that broken feature right away.
hw2390: Start of request header:in hw2390: header:in before getLine - timeout:55000 Line: 1938 Function: in hw2390: firstime: header:in after getLine Line: 1942 Function: in hw2390: header:size = 7 hw2390: first line = POST /vpninfo/servers HTTP/1.1 Line: 981 Function: checkheader: Host: www.privateinternetaccess.com Line: 981 Function: checkheader: Accept-Encoding: identity,gzip,deflate Line: 981 Function: checkheader: Accept: */* Line: 981 Function: checkheader: User-Agent: Ruby hw2390: tp =Content-Length: 49 hw2390: tp =49 Contentlen.int =49 Line: 981 Function: checkheader: Content-Length: 49 hw2390: Header value from client: Content-Type: application/x-www-form-urlencode Line: 981 Function: checkheader hw2390: CheckHeader: HTTP/1.1 detected Line: 990 Function: checkheader hw2390: CheckHeader flags before normalisation: AP=1 PPC=0 1.1=1 connectionclose=1 CL=1 Line: 1051 Function: checkheader hw2390: CheckHeader flags after normalisation: AP=0 WP=0 Line: 1074 Function: checkheader hw2390: CheckHeader: Adding our own Proxy-Connection: Close Line: 1093 Function: checkheader hw2390: HTTPHeader size: 8 hw2390: from header url:http://www.privateinternetaccess.com/vpninfo/servers Line: 1235 Function: getUrl hw2390: setURL: header.front() changed from: POST http://www.privateinternetacce Line: 540 Function: setURL/1.1 Line: 548 Function: setURLw.privateinternetaccess.com/vpninfo/servers HTTP/1.1 Line: 553 Function: setURLne changed from: Host: www.privateinternetaccess.com Line: 562 Function: setURLteinternetaccess.com hw2390: isProxyRequest is 0 hw2390: firsttime =1ismitm =0 clientuser = group = 1 hw2390: Compiling ,*[a-z|A-Z].* hw2390: ...with PCRE hw2390: HTTPHeader size: 8 hw2390: from header url:http://www.privateinternetaccess.com/vpninfo/servers Line: 1235 Function: getUrl hw2390: decoding url Line: 1519 Function: decode hw2390: 38420Start URL http://www.privateinternetaccess.com/vpninfo/serversis_ssl=0ismitm=0 hw2390: inIPList no match for 172.16.1.17 hw2390: inIPList no match for 172.16.1.17 hw2390: inURLList: www.privateinternetaccess.com/vpninfo/servers hw2390: inURLList (processed): www.privateinternetaccess.com/vpninfo/servers After StoryA pre-authcheck0 mess_no 0 hw2390: isProxyRequest is 0 only_ip_auth is 1 hw2390: -Not got persistent credentials for this connection - querying auth plugins hw2390: -Querying next auth plugin... hw2390: -Auth plugin found username 172.16.1.17 (), now determining group hw2390: Matched IP 172.16.1.17 to straight IP list hw2390: Auth plugin found username & group; not querying remaining plugins hw2390: -Identity found; caching username & group Auth plugin is for a connection-based auth method - keeping credentials for entire connection hw2390: -username: 172.16.1.17 hw2390: -filtergroup: 1 hw2390: -About to check for bypass... hw2390: Found cookie: Line: 1309 Function: getCookie hw2390: No bypass cookie Line: 1337 Function: isBypassCookie hw2390: -Finished bypass checks. hw2390: inURLList: www.privateinternetaccess.com/vpninfo/servers hw2390: inURLList (processed): www.privateinternetaccess.com/vpninfo/servers hw2390: inURLList: www.privateinternetaccess.com/vpninfo/servers hw2390: inURLList (processed): www.privateinternetaccess.com/vpninfo/servers hw2390: After StoryB checkrequest isexception 0 gomitm 0 mess_no 500 hw2390: -Forwarding body to client : Upfailure is 0 hw2390: -reporting level is 3 hw2390: -Enabling filter bypass hash generation hw2390: mime type: - Line: 217 Function: isContentType hw2390: mimes result : false Line: 244 Function: isContentType hw2390: Displaying TEMPLATE hw2390: -HOST- placeholder encountered but hostname currently unknown; lookup forced. Segmentation fault -
Happened and fixed on 4.1. maybe not merged on 5.0
I'll se if I find the issue number on 4.1 branch and post on #375
-
Happened and fixed on 4.1. maybe not merged on 5.0
I'll se if I find the issue number on 4.1 branch and post on #375
Face palm even without the -HOSTS- place holder in my block page. E2 Guardian still crashes after sometime, I guess I'll have to do more debugging tomorrow.
-
Happened and fixed on 4.1. maybe not merged on 5.0
I'll se if I find the issue number on 4.1 branch and post on #375
Face palm even without the -HOSTS- place holder in my block page. E2 Guardian still crashes after sometime, I guess I'll have to do more debugging tomorrow.
SB list name banned checking type 3 Looking for banned type 3 in listmeta Found banned type 3 in listmeta list reference 196 'banned' found for banned SB list name banned checking type 2 Looking for banned type 2 in listmeta Found banned type 2 in listmeta list reference 206 'banned' found for banned SB list name banned checking type 6 Looking for banned type 6 in listmeta Found banned type 6 in listmeta list reference 223 'banned' found for banned banned matches 3 types Line 218 state is 2 actionid 5003 listname banned Compiling .*[a-z|A-Z].* ...with PCRE Setting magic key to 'PWYDSKLILPOTMCAK' read filter group: 4 /usr/local/etc/e2guardian/e2guardianf5.conf return is 1 I seem to be running already! In deleteFilterGroups loop In deleteFilterGroups loop In deleteFilterGroups loop In deleteFilterGroups loop In deleteFilterGroups loopJust got this @Marcelloc and then E2G crashed. Weird… Not sure what return is 1 means.
Also getting the below with E2G Debug binaries:
Apr 16 14:49:22 e2guardian 83603 Tunnel status not 200 ok aborting Apr 16 14:49:24 e2guardian 83603 Tunnel status not 200 ok aborting Apr 16 14:49:30 e2guardian 83603 Tunnel status not 200 ok aborting Apr 16 14:49:33 e2guardian 83603 Tunnel status not 200 ok aborting -
I seem to be running already! Is the error when there is another process running (apply whike reloading, watchdog, etc…)
-
I seem to be running already! Is the error when there is another process running (apply whike reloading, watchdog, etc…)
I think the watchdog may need to be re-designed, because that shouldn't happen. My E2G is set to kill a running instances and start new ones on config change.
-
Hi Marcelloc,
Could you please update the binaries, the E2G developers have pushed out some patches which should help with these segmentation faults. Funnily enough the debug version seems to work more stable for me than the one in the repo.
Thanks
-
hi marcello
There are 2 responsibilities
1. exe zip vs download blocking
When we try to enter some prohibited sites, such as the second picture, it reacts differently
one says that this site can not be reached and the other page comes Block.
-
Hi Marcelloc,
Could you please update the binaries, the E2G developers have pushed out some patches which should help with these segmentation faults. Funnily enough the debug version seems to work more stable for me than the one in the repo.
Thanks
Binaries updated on repo.
-
hi marcello
There are 2 responsibilities
1. exe zip vs download blocking
When we try to enter some prohibited sites, such as the second picture, it reacts differently
one says that this site can not be reached and the other page comes Block.Have you got Forge SSL certificates turned on? If you enable that and install the CA on all your devices, you will then get the block page on all banned sites.
Weird that you're getting a DNS error though. Are you sure you don't have any DNS blocking that site?
-
Hi Marcelloc,
Could you please update the binaries, the E2G developers have pushed out some patches which should help with these segmentation faults. Funnily enough the debug version seems to work more stable for me than the one in the repo.
Thanks
Binaries updated on repo.
Awesome, updating right away!
-
hi pfsensation
I'm in Turkey we have many sites forbidden we have many sites forbidden
operator bans telecom with DNS -
Last build I forgot to remove DEBUG compile option, so I've pushed binaries version e2guardian-5.0.2_5.txz without debug to the repo.
If you need current debug version, fetch from :
pkg add -f https://e-sac.websiteseguro.com/e2g/e2guardian-5.0.2_5.txz -
hi pfsensation
I'm in Turkey we have many sites forbidden we have many sites forbidden
operator bans telecom with DNSThat's your problem in the screenshot. Turkey already bans most bad sites due to religious reasons. I recently came back from Turkey (Antalya), went on holiday it was amazing! I'm guessing you're using Turkce Telekom?
I'm not too sure how it'll work in your instance as I haven't tested it. E2 Guardian can't do any phrase checking if the DNS is being null routed. However it should still be detect the URL from ShallaList. Turn on Forge SSL Certificates, and install the CA and give it a shot. Remember, on HTTPS sites you will not get a block page of you do not have MITM enabled.
-
Is anybody else receiving ERR_SSL_BAS_RECORD_MAC_ALERT on chrome and firefox after latest updated with MITM?
