Unofficial E2guardian package for pfSense



  • @kenrutt said in Unofficial E2guardian package for pfSense:

    @pfsensation Thanks for reply on the regex issue. That is a point I had not thought of, on regex using a lot of cpu power. However some sites will load fine and the regex will do fine. But whenever I go to google It will crash E2guardian immediately. I watched the cpu indicator at that point and it never even seemed to kickin much before crash. There must be something in expression that causes it. Here is a sample of what I was using.
    "<a(?:(?!.</a>).).?facebook.com.*?</a>"->"-"
    Don't know if you can see anything out of order or not.
    Thanks

    I haven't used regex myself in a while, but why not use the site list ACL to block Facebook instead? It's a much more efficient way of doing it.

    I'll have to test out regex further, just don't have much spare time at the moment. :/



  • any guides on how to make lightsquid log e2guardian network activities?



  • @sei-pine I have a blog post about Sarg to report E2guardian activities. Check the following link.

    https://lifeoverlinux.com/how-to-configure-sarg-to-use-with-e2guardian/

    For the Lightsquid, it's easy to setup. You can find how to by searching "e2guardian lightsquid" on the forum.



  • @ucribrahim i can't seem to get sargs to get report on e2guardian it shows this error
    0_1542164612328_8dee63bf-4888-4367-a35d-8157e2eec336-image.png

    i already tried to do the troubleshoot guide on the page you provided.

    edit:

    this seems to be the problem, any idea on how to fix it ?

    SARG: SARG version: 2.3.11 Jan-14-2018
    SARG: Reading access log file: /var/log/e2guardian/access.log
    SARG: Loop detected in getword_atoll after 2 bytes.
    SARG: Line="92.168.137.5 https"
    SARG: Record="92.168.137.5 https"
    SARG: searching for 'x2f'
    SARG: Invalid date in file "/var/log/e2guardian/access.log"



  • @sei-pine
    Hello
    Check log format in e2g "Report and log" configuration, it must be in "squid format"



  • @binkec said in Unofficial E2guardian package for pfSense:

    @sei-pine
    Hello
    Check log format in e2g "Report and log" configuration, it must be in "squid format"

    +1

    This should fix the error.



  • @binkec its already on squid format. Well i did manage to log e2guardian using lightsquid earlier. Gonna monitor it for now.



  • @sei-pine said in Unofficial E2guardian package for pfSense:

    @binkec its already on squid format. Well i did manage to log e2guardian using lightsquid earlier. Gonna monitor it for now.

    Manually clear your access.log visit some sites then try again. Sarg is complaining of an incorrect date and a loop.

    Glad to hear you got lightsquid working though, it's simple and it gets the job done.



  • @marcelloc are you still active with e2guardian? What is the status?



  • @marcelloc, don't we have update fix for the content scanner ?

    since the update they made for the rotate log fix, the content scanner is no longer working. it makes the pfsense gui hang and I need to restore previous config to make it work.



  • @jetberrocal said in Unofficial E2guardian package for pfSense:

    @marcelloc are you still active with e2guardian? What is the status?

    He's just very busy with work however he is slowly updating it whenever possible, last update was pretty recent.



  • @ravegen said in Unofficial E2guardian package for pfSense:

    @marcelloc, don't we have update fix for the content scanner ?

    since the update they made for the rotate log fix, the content scanner is no longer working. it makes the pfsense gui hang and I need to restore previous config to make it work.

    Are you able to provide any further info? Are you seeing any errors on logs? What's the resource usage like? Are you able to SSH into your box?

    Like I said earlier, content scanner is not a feature me or Marcelloc really use but let's try to find out why it crashes if possible.



  • Ok here's my feedback about using Lightsquid to get logs on e2guardian, after 2 days it just stop logging e2guardian. i dunno what happened lol

    edit: seems like when i turn off the transparent proxy on squid proxy server, lightsquid doesn't work.

    gonna monitor it again for a few days if it'll stop logging.

    edit: good, its now running fine. earlier was kinda slow or something

    0_1542691727081_78cfc061-63ce-4204-9531-8ceb57315c2a-image.png



  • @pfsensation, @marcelloc

    attached is the error log.

    0_1542682201493_1541384673350-error-resized.png

    I have provided that screenshot, months before.

    System resource usage is fine, load average 0.14, 0.10, 0.04, cpu & swap usage is 0%, memorage usage is 28%.

    I am able to ssh to my box.

    Like I said, the content scanner (clamdscan) was perfectly running before the rotate log fix was created and updated to the package.

    Respectfully, I think it does not matter if the content scanner is a feature for you or for Marcelloc but rather for the use of the community since it is a feature placed in the package. Such content scanner should have been already removed if it is that useless.

    What I am saying is the content scanner was broken since their was a fix on rotate log. Perhaps, the developers who modified the package can differential what changes they have done before and after the rotate log fix and made changes accordingly.



  • @ravegen said in Unofficial E2guardian package for pfSense:

    @pfsensation, @marcelloc

    attached is the error log.

    0_1542682201493_1541384673350-error-resized.png

    I have provided that screenshot, months before.

    System resource usage is fine, load average 0.14, 0.10, 0.04, cpu & swap usage is 0%, memorage usage is 28%.

    I am able to ssh to my box.

    Like I said, the content scanner (clamdscan) was perfectly running before the rotate log fix was created and updated to the package.

    Respectfully, I think it does not matter if the content scanner is a feature for you or for Marcelloc but rather for the use of the community since it is a feature placed in the package. Such content scanner should have been already removed if it is that useless.

    What I am saying is the content scanner was broken since their was a fix on rotate log. Perhaps, the developers who modified the package can differential what changes they have done before and after the rotate log fix and made changes accordingly.

    OK, those errors look config related. I'll have a chat with @marcelloc and then we'll see if we can send out another patch for it. Comparing the last build to your current, there doesn't seem to be any specific changes that would effect the content scanner.



  • i switched to sarg to log e2guardian. lightsquid is totally not working after few hours.

    0_1542761130085_26ae12be-41bc-4f08-9cee-2aa089756247-image.png



  • @sei-pine said in Unofficial E2guardian package for pfSense:

    i switched to sarg to log e2guardian. lightsquid is totally not working after few hours.

    0_1542761130085_26ae12be-41bc-4f08-9cee-2aa089756247-image.png

    How long have you set E2 Guardian to keep logs for? And how often are you log rotating? I've not come across this issue so it maybe a config issue.



  • @pfsensation i leave all lightsquid settings as default (should be fine i guess) but when i try to configure squid and turn off its transparent proxy and mitm (this is interfering with e2guardian so...) lightsquid doesn't log anything.

    i dunno, maybe i need to make lightsquid listen to e2guardian ? anyways, i set my e2guardian to keep 20 log files.



  • @pfsensation
    Hi, I was busy past two weeks and I tried to install 2.4.4 again and I can see real time traffic. E2g is working except "weighted phrases". I went trough working configuration(2.4.3) and non working(2.4.4) and I couldn't see any difference.
    The only difference is in log,
    2.4.4 I have this error:
    Nov 25 20:47:12 e2guardian 97044 I seem to be running already!
    Nov 25 20:54:23 e2guardian 75338 listen0_proxy: 1: Error 53 on accept: errorcount 0
    Nov 25 20:54:23 e2guardian 75338 listen0_proxy: 1: Error 53 on accept: errorcount 1
    Nov 25 20:57:44 e2guardian 75338 listen0_proxy: 1: Error 53 on accept: errorcount 0
    Nov 25 21:00:13 e2guardian 15579 I seem to be running already!
    Nov 25 21:04:37 e2guardian 20124 listen0_proxy: 1: Error 53 on accept: errorcount 0
    Nov 25 21:04:37 e2guardian 20124 listen0_proxy: 1: Error 53 on accept: errorcount 1
    Nov 25 21:04:37 e2guardian 20124 listen0_proxy: 1: Error 53 on accept: errorcount 2
    Nov 25 21:04:37 e2guardian 20124 listen0_proxy: 1: Error 53 on accept: errorcount 3
    Nov 25 21:04:37 e2guardian 20124 listen0_proxy: 1: Error 53 on accept: errorcount 0
    Nov 25 21:14:32 e2guardian 62358 I seem to be running already!
    Nov 25 21:17:16 e2guardian 3849 listen0_proxy: 1: Error 53 on accept: errorcount 0
    Nov 25 21:17:36 e2guardian 3849 listen0_proxy: 1: Error 53 on accept: errorcount 0

    In working 2.4.3 system there is no error.
    Regards



  • @pfsensation ,

    Is there a progress on the content scanner fix ?



  • @ravegen said in Unofficial E2guardian package for pfSense:

    @pfsensation ,

    Is there a progress on the content scanner fix ?

    I've already asked @marcelloc to have a look into it. He believes it could be an ICAP issue, I'm not sure if he's had a chance yet to try implement a fix.



  • @sei-pine said in Unofficial E2guardian package for pfSense:

    @pfsensation i leave all lightsquid settings as default (should be fine i guess) but when i try to configure squid and turn off its transparent proxy and mitm (this is interfering with e2guardian so...) lightsquid doesn't log anything.

    i dunno, maybe i need to make lightsquid listen to e2guardian ? anyways, i set my e2guardian to keep 20 log files.

    You shouldn't have transparent proxy on Squid anyways. Set E2 Guardian log format to Squid and make sure you install the custom Inc file so that light squid listens to E2 Guardian. It's higher up in this thread.



  • @binkec said in Unofficial E2guardian package for pfSense:

    @pfsensation
    Hi, I was busy past two weeks and I tried to install 2.4.4 again and I can see real time traffic. E2g is working except "weighted phrases". I went trough working configuration(2.4.3) and non working(2.4.4) and I couldn't see any difference.
    The only difference is in log,
    2.4.4 I have this error:
    Nov 25 20:47:12 e2guardian 97044 I seem to be running already!
    Nov 25 20:54:23 e2guardian 75338 listen0_proxy: 1: Error 53 on accept: errorcount 0
    Nov 25 20:54:23 e2guardian 75338 listen0_proxy: 1: Error 53 on accept: errorcount 1
    Nov 25 20:57:44 e2guardian 75338 listen0_proxy: 1: Error 53 on accept: errorcount 0
    Nov 25 21:00:13 e2guardian 15579 I seem to be running already!
    Nov 25 21:04:37 e2guardian 20124 listen0_proxy: 1: Error 53 on accept: errorcount 0
    Nov 25 21:04:37 e2guardian 20124 listen0_proxy: 1: Error 53 on accept: errorcount 1
    Nov 25 21:04:37 e2guardian 20124 listen0_proxy: 1: Error 53 on accept: errorcount 2
    Nov 25 21:04:37 e2guardian 20124 listen0_proxy: 1: Error 53 on accept: errorcount 3
    Nov 25 21:04:37 e2guardian 20124 listen0_proxy: 1: Error 53 on accept: errorcount 0
    Nov 25 21:14:32 e2guardian 62358 I seem to be running already!
    Nov 25 21:17:16 e2guardian 3849 listen0_proxy: 1: Error 53 on accept: errorcount 0
    Nov 25 21:17:36 e2guardian 3849 listen0_proxy: 1: Error 53 on accept: errorcount 0

    In working 2.4.3 system there is no error.
    Regards

    Which ports are you running E2 Guardian on? Are you using other packages like pfblockerng?



  • @pfsensation
    Hi
    I am using default port 8080 in direct connect widhout squid, no transparent mode and fresh install, only E2g and sarg on VMWARE. Like I sad I vent trough working config step by step several times and I didn't find any difference, it should be something in E2g.

    Regards



  • @binkec said in Unofficial E2guardian package for pfSense:

    @pfsensation
    Hi
    I am using default port 8080 in direct connect widhout squid, no transparent mode and fresh install, only E2g and sarg on VMWARE. Like I sad I vent trough working config step by step several times and I didn't find any difference, it should be something in E2g.

    Regards

    Which interfaces are you listening on? Make sure it's set to LAN and localhost. I run my system in VMWare ESXi without any issues.



  • @pfsensation
    Hi
    I have set to both, are you using "phase list" filtering.
    0_1543420915226_28_11.png



  • @binkec said in Unofficial E2guardian package for pfSense:

    @pfsensation
    Hi
    I have set to both, are you using "phase list" filtering.
    0_1543420915226_28_11.png

    Of course, I use black list and phrase list. Phrase based filtering is actually one of the core functions of E2 Guardian and what makes it vastly better than other systems like SquidGuard. I'm on 2.4.4 and haven't had those issues and doesn't look like anyone else has either. So I'd be interested to know why it would happen.

    Are you running just vmware work station or ESXi?

    Edit: put the HTTP workers up, I have mine at 3072.



  • 0_1543877833818_pfSense.localdomain_-Status_System_Logs_System_General-_2018-12-04_01.55.49.bmp

    System working but e2guard access.log not working. I did port 8081, 8082; result same. Why?



  • @plusbil said in Unofficial E2guardian package for pfSense:

    0_1543877833818_pfSense.localdomain_-Status_System_Logs_System_General-_2018-12-04_01.55.49.bmp

    System working but e2guard access.log not working. I did port 8081, 8082; result same. Why?

    Can you check var/log/e2guardian/access.log and see if it's updating there?



  • Not update. New access.log zero byte. But when I restard pfsense system, e2guard log working.



  • And 3-4 days later again stop.



  • 0_1543990498177_0d335c3f-0fde-4caf-bc44-4c4d9ab236d0-image.png

    so far so good, the only problem i encounter is each day when sarg stops logging, i need to change time format to American or European then force refresh for sarg to continue logging. its a little hassle but still it works.

    0_1543990693231_ecea3ffe-6613-474f-89ae-e68a0dda3bb4-image.png

    btw, till now i still can't figure out how to use the Users tab on E2Guardian or do i need to use LDAP?, need help with this one thanks.

    what I've tried so far is to use PFSENSE\(Group name)
    (Group Name)\(Account Name)
    PFSENSE\(Group Name)\(Account Name)

    ^ Doesn't work



  • @plusbil said in Unofficial E2guardian package for pfSense:

    Not update. New access.log zero byte. But when I restard pfsense system, e2guard log working.

    Clear the logs file manually, restart E2 Guardian and let me know what you've got your log rotate settings set as. It's been brilliant for me and working without any issues.



  • @sei-pine said in Unofficial E2guardian package for pfSense:

    0_1543990498177_0d335c3f-0fde-4caf-bc44-4c4d9ab236d0-image.png

    so far so good, the only problem i encounter is each day when sarg stops logging, i need to change time format to American or European then force refresh for sarg to continue logging. its a little hassle but still it works.

    0_1543990693231_ecea3ffe-6613-474f-89ae-e68a0dda3bb4-image.png

    btw, till now i still can't figure out how to use the Users tab on E2Guardian or do i need to use LDAP?, need help with this one thanks.

    what I've tried so far is to use PFSENSE\(Group name)
    (Group Name)\(Account Name)
    PFSENSE\(Group Name)\(Account Name)

    ^ Doesn't work

    Although I personally haven't used that specific configuration, and I just use groups. You may need to enable a different authentication method to be able to use the users tab. What authentication method have you currently got enabled?



  • @pfsensation ah i see! I was only using local users lmao. I'll try to configure freeradius first, thanks for the info!



  • @pfsensation said in Unofficial E2guardian package for pfSense:

    Clear the logs file manually, restart E2 Guardian and let me know what you've got your log rotate settings set as. It's been brilliant for me and working without any issues.

    It's been five day. It working for now. I'il try when there's a problem. Thanks...



  • @pfsensation

    @ravegen said in Unofficial E2guardian package for pfSense:

    @pfsensation ,

    Is there a progress on the content scanner fix ?

    What is our update on this ?



  • @ravegen said in Unofficial E2guardian package for pfSense:

    @pfsensation

    @ravegen said in Unofficial E2guardian package for pfSense:

    @pfsensation ,

    Is there a progress on the content scanner fix ?

    What is our update on this ?

    Nothing yet unfortunately. @marcelloc Have you had a chance now to take a look at this problem?



  • UPDATE

    Hello folks,
    I´ve been testing E2Guardian+SQUID with SSO NTLM and it is working REALLY fine.
    There are some small/cosmetic issues, but at all, it's working fine.
    Issues that I could get so far:
    *The LDAP search group/users mechanism has some issues. It seems the script created by the package (see crontab) doesn't have the proper env setting. I tested it assing "sh -c" in front of it, and it resolved the issue.
    *The e2guardian service sometimes got stuck.the GUI start/stop/restart icons doesn't work and you have to kill the PID from shell console. It seems it's something related to the "save/apply" process after the LDAP update script runs. I am still investigating.

    The e2guardian is "really" better than "squidguard" - There is no comparison! Forget about squidguard,

    Hope that helps,
    Fabricio.



  • @fabricioguzzy said in Unofficial E2guardian package for pfSense:

    UPDATE

    Hello folks,
    I´ve been testing E2Guardian+SQUID with SSO NTLM and it is working REALLY fine.
    There are some small/cosmetic issues, but at all, it's working fine.
    Issues that I could get so far:
    *The LDAP search group/users mechanism has some issues. It seems the script created by the package (see crontab) doesn't have the proper env setting. I tested it assing "sh -c" in front of it, and it resolved the issue.
    *The e2guardian service sometimes got stuck.the GUI start/stop/restart icons doesn't work and you have to kill the PID from shell console. It seems it's something related to the "save/apply" process after the LDAP update script runs. I am still investigating.

    The e2guardian is "really" better than "squidguard" - There is no comparison! Forget about squidguard,

    Hope that helps,
    Fabricio.

    Most of us have known or have come to know how much of a broken mess SquidGuard is. E2 Guardian filtering is much more advanced and granular.

    I suggest you report the issues on the E2 Guardian Github page for quicker response/fixes. Thank you for the update, and I'm glad you've got it working with LDAP!


Log in to reply