Sarg package for pfsense



  • Hi all,

    I've just published sarg package for pfsense with squid,squidguard and dansguardian log Analysis as well real time report tab.

    Squidguard functions are under devel yet but squid and dansguardians(as well as I tested) are working.

    After almost everything done, I found an old sarg package published on forum by joaohf and merged some function calls from this old thread.

    Another good point is that sarg is able to forward logs via email, so I'm planning to include it for nanobsd installs.

    have fun and feedback!  :)

    att,
    Marcello Coutinho









  • Excelent, thank you for your time to develop this package. I will test it today. ;D

    Cheers!!



  • @marcelloc:

    Another good point is that sarg is able to forward logs via email, so I'm planning to include it for nanobsd installs.

    Hi Marcello,

    Great job!    Just wanted to ask about sending reports via email,  you say you plan to included it in the nanobsd builds,  will it also be included in the mainline?  (or should it be already?  i didn't see it.)

    thanks and have a great day!
    -g



  • I didn't coded it yet, it's just plans for next release.



  • just some feed back,  i realize you are not finished..

    when i click on 'real time' tab and attempt to view realtime reports,  I get a 404 - Not Found error at the bottom of the page.

    -g



  • what log did you selected on sarg settings?

    squidguard features are not finished yet.  :(



  • great job marcelloc!!

    Noticed a few things:

    1: squid on my box is use path /var/squid/log, not /var/squid/logs.. Not sure why… but i corrected the path.. Think its because lightsquid was looking for /var/squid/log and the field for log location was removed from that package

    2: there is no index page for reports... I reinstalled the package and binay... Now I get "php: /pkg_edit.php: The command '/usr/local/bin/sarg ' returned exit code '1', the output was 'SARG: Cannot set the locale LC_ALL to the environment variable'"

    3: realtime doesn't work, 404 - Not Found



  • @marcelloc:

    what log did you selected on sarg settings?

    squidguard features are not finished yet.  :(

    it is squid I'm using Marcello,  here is my config should it help you out.   anything else you would like to see/try let me know.

    [2.0.1-RELEASE][root@gw-master.foobar.com]/usr/local/etc/sarg(11): cat sarg.conf | sed -e '/^#/d' -e '/^$/d'
    access_log /var/squid/logs/access.log
    graphs yes
    output_dir /usr/local/www/sarg-reports
    anonymous_output_files no
    resolve_ip no
    user_ip no
    topuser_sort_field BYTES NORMAL
    user_sort_field BYTES NORMAL
    exclude_users /usr/local/etc/sarg/exclude_users.conf
    remove_temp_files yes
    index yes
    index_tree date
    overwrite_report yes
    use_comma yes
    report_type topusers topsites sites_users users_sites date_time denied auth_failures site_user_time_date downloads
    usertab
    long_url no
    charset UTF-8
    privacy no
    dansguardian_conf
    squidguard_conf
    www_document_root /usr/local/www
    realtime_refresh_time 0
    realtime_types GET,PUT,CONNECT
    realtime_unauthenticated_records show

    -g



  • Thanks cino and gwhynott for your feedback.

    The realtime error is a missing file I've forgot to publish.

    I'll do it today and also check better squid log option.

    try to select both index options on sarg gui to see if it generate files correctly.



  • @Cino:

    1: squid on my box is use path /var/squid/log, not /var/squid/logs.. Not sure why…

    lightsquid did that.  I think the package maintainer for lightsquid has since corrected it.  I created a sym link so both locations work, try to keep everyone happy.  8)

    -g



  • @gwhynott:

    lightsquid did that.  I think the package maintainer for lightsquid has since corrected it.   I created a sym link so both locations work, try to keep everyone happy.  8)

    Thanks for the note, I'll try to read it from squid xml info.



  • @marcelloc:

    Thanks cino and gwhynott for your feedback.

    The realtime error is a missing file I've forgot to publish.

    I'll do it today and also check better squid log option.

    try to select both index options on sarg gui to see if it generate files correctly.

    thanks for quick reply!  I couldn't Force Update Now but manually running the cron job '/usr/local/bin/php /usr/local/www/sarg.php 0' seem to create the pages.. I'll wait an hour and see what happens

    Have to tweak some things now ;-) I like the idea you included commands to run after to rotate the logs… have to play around with this...



  • @gwhynott:

    @Cino:

    1: squid on my box is use path /var/squid/log, not /var/squid/logs.. Not sure why…

    lightsquid did that.  I think the package maintainer for lightsquid has since corrected it.   I created a sym link so both locations work, try to keep everyone happy.  8)

    -g

    good to know! I may create a link myself



  • I've published missing file.  :)



  • I should had just copied the file over… Some reason now, sarg.conf isn't updating. File is blank about package re-install

    I need to do more testing. remove package, reboot box and install it again

    edit: i manually created the sarg.conf file.... the realtime page doesn't return data but 'sarg -r' from the cmdline does



  • I did package reinstall.

    I'll do some tests too. :(



  • Sorry cino, I've updated an old file and now template file is missing  :P

    Just wait the 15 minutes to reinstall.



  • yeaaaaa, thanks man I will tested, I have been use this package for a while but my install is manually, love it.
      How difficult is to download the info in excel,cvs form?

    One client ask about before.

    Thanks!!!



  • @periko:

    How difficult is to download the info in excel,cvs form?

    If it's a built in sarg feature can be done but I saw only html, email or sql like reports.



  • Sarg 0.2 is out.

    • Sarg exclude lists options

    • bug fixes

    • gui improvements


  • Rebel Alliance Developer Netgate

    You should add some screenshots of sample reports to the first post on the thread. Might help people decide if this is right for them (vs lightsquid).



  • @jimp:

    You should add some screenshots of sample reports to the first post on the thread. Might help people decide if this is right for them (vs lightsquid).

    done  :) Thanks Jimp!


  • Rebel Alliance Developer Netgate

    Nice :-)

    Is that in an iframe? I had problems trying to make lightsquid do that, it was cached very aggressively and the data would never update even when I refreshed, I had to clear the cache to get some pages to reload, so I abandoned doing it that way.



  • I was having problems to view the report page. Always I clicked View report tab i got a 404 error, as you can see in the next image.

    So I decided to look in the code of "sarg_reports.php" and I found this line:

    <iframe src="/sarg-reports/" frameborder="0" width="100%" height="600"></iframe>
    

    and I changed to

    <iframe src="/sarg-reports/index.html" frameborder="0" width="100%" height="600"></iframe>
    

    And now every works fine!

    Thank you for this excellent package!!



  • @jimp:

    Is that in an iframe? I had problems trying to make lightsquid do that, it was cached very aggressively and the data would never update even when I refreshed, I had to clear the cache to get some pages to reload, so I abandoned doing it that way.

    The realtime is ajax but report is iframed. I'll check if this cache issue is happening, If so I'll do some ajax with replaces to keep it on gui.



  • @saxonbeta:

    So I decided to look in the code of "sarg_reports.php" and I found this line:

    <iframe src="/sarg-reports/" frameborder="0" width="100%" height="600"></iframe>
    

    and I changed to

    <iframe src="/sarg-reports/index.html" frameborder="0" width="100%" height="600"></iframe>
    

    And now every works fine!

    Thanks saxonbeta for your feedback, I'll test it and also the cache issue today.



  • updated sarg to 0.2.1 with report tab without cache(as well as I've tested). ;)



  • I have tested the latest version and the cache problem persists. I have to manually update the frame to get the last report. :(



  • The frame src should be a sarg_frame.php instead of sarg-reports.

    Can you check if you got these changes?

    What you mean by manual update? File edit or browser refresh?



  • @marcelloc:

    The frame src should be a sarg_frame.php instead of sarg-reports.

    Can you check if you got these changes?

    That's right, the frame source is sarg_frame.php

    @marcelloc:

    What you mean by manual update? File edit or browser refresh?

    It is a browser refresh, using the context menu>reload frame



  • On my tests everytime I click on report I get the latest index.html ???



  • just a note, with the pkg that was available this am.  glad to see this one back.

    Looks like it can create the directory etc.  This is from the system log

    php: /pkg_edit.php: The command '/usr/local/bin/sarg ' returned exit code '1', the output was 'SARG: Records in file: 174, reading: 0.00%^MSARG: cannot open /usr/local/www/sarg-reports/2012/03/31/sarg-date for writing SARG:: No such file or directory SARG: Records in file: 174, reading: 100.00%'

    Followup, access via gui give this;

    Warning: file_get_contents(/usr/local/www/sarg-reports/index.html): failed to open stream: No such file or directory in /usr/local/www/sarg_frame.php on line 41

    Sarg Realtime appears to be working ok



  • @dlawley:

    Sarg Realtime appears to be working ok

    i can't seem to get any data to show in realtime.. only when using the original sarg-realtime.php does it work for me..



  • Cino,

    What do you get on realtime? Just the headers from sarg?

    The sarg-realtime.php is just a sarg -r frontend.

    Maybe a stupid question but it shows the running info on button?



  • @dlawley:

    Warning: file_get_contents(/usr/local/www/sarg-reports/index.html): failed to open stream: No such file or directory in /usr/local/www/sarg_frame.php on line 41

    Sarg Realtime appears to be working ok

    Are you running full or nanobsd install?

    I'll check folder creation code.



  • @marcelloc:

    Cino,

    What do you get on realtime? Just the headers from sarg?

    The sarg-realtime.php is just a sarg -r frontend.

    Maybe a stupid question but it shows the running info on button?

    nothing is shown… I click on the button, it changes to running but nothing is displayed



  • @marcelloc:

    @dlawley:

    Warning: file_get_contents(/usr/local/www/sarg-reports/index.html): failed to open stream: No such file or directory in /usr/local/www/sarg_frame.php on line 41

    Sarg Realtime appears to be working ok

    Are you running full or nanobsd install?

    I'll check folder creation code.

    Sorry not to mention that , but it is a full install (not nano)



  • @marcelloc:

    @dlawley:

    Warning: file_get_contents(/usr/local/www/sarg-reports/index.html): failed to open stream: No such file or directory in /usr/local/www/sarg_frame.php on line 41

    Sarg Realtime appears to be working ok

    Are you running full or nanobsd install?

    I'll check folder creation code.

    I dunno.  Still not showing the service as "running" , but I am finally starting to see some data.  I just pulled another reinstall, not sure if that helped.



  • Sarg service will only show as running only on report updates, It's run by cron.

    Better if I remove it from service list.



  • Dlawley,

    Can you try to run sarg on console/ssh and post the response?