Sarg package for pfsense

marcelloc

Enable report overwrite to avoid multiples reports in the same day.

It looks like you didn't enabled(or disabled) squid logs after may 26.

nuphero

I tried to install SARG some times but always got problem Segmentation fault (Core dumped)
Here is log of sarg -x. Anyone experienced with this issue, please help. Thanks.

[2.0.3-RELEASE][root@pfSense.localdomain]/usr/local(34): sarg -x
SARG: Init
SARG: Loading configuration from /usr/local/etc/sarg/sarg.conf
SARG: Loading exclude host file from: /usr/local/etc/sarg/exclude_hosts.conf
SARG: Loading exclude file from: /usr/local/etc/sarg/exclude_users.conf
SARG: Reading host alias file "/usr/local/etc/sarg/hostalias"
SARG: List of host names to alias:
SARG: Deleting temporary directory "/tmp/sarg"
SARG: Parameters:
SARG: Hostname or IP address (-a) =
SARG: Useragent log (-b) =
SARG: Exclude file (-c) = /usr/local/etc/sarg/exclude_hosts. conf
SARG: Date from-until (-d) =
SARG: Email address to send reports (-e) =
SARG: Config file (-f) = /usr/local/etc/sarg/sarg.conf
SARG: Date format (-g) = Europe (dd/mm/yyyy)
SARG: IP report (-i) = No
SARG: Keep temporary files (-k) = No
SARG: Input log (-l) = /var/squid/log/access.log
SARG: Resolve IP Address (-n) = No
SARG: Output dir (-o) = /usr/local/sarg-reports/
SARG: Use Ip Address instead of userid (-p) = Yes
SARG: Accessed site (-s) =
SARG: Time (-t) =
SARG: User (-u) =
SARG: Temporary dir (-w) = /tmp/sarg
SARG: Debug messages (-x) = Yes
SARG: Process messages (-z) = No
SARG: Previous reports to keep (–lastlog) = 0
SARG:
SARG: sarg version: 2.3.6 Arp-21-2013
SARG: Reading access log file: /var/squid/log/access.log
SARG: Records in file: 874, reading: 100.00%
SARG: Records read: 874, written: 874, excluded: 0
SARG: Squid log format
SARG: Period: 30 May 2013
SARG: File /usr/local/sarg-reports/30May2013-30May2013 already exists, moved to /usr/local/sarg-reports/30May2013-30May2013.18
SARG: Sorting log /tmp/sarg/10_48_14_29.user_unsort
Segmentation fault (core dumped)

Hugovsky

Maybe you have changed date format in General tab. That was my problem. It works in default but not in European.

sully

I installed SARG on my box numerous times, following various threads on how to get it to work. I did get it to work, and wanted to start over, so deleted the files it created. Upon doing this, it would not recreate them, even with the indexing options toggled on.

A few more uninstalls and pkg_deletes and pkg_adds later, both from command and anyterm, still left it non working. Even when SARG from anyterm would return no errors, it would not create the index files.

There was however a graph error, which I turned graphing off in the config. And one other error, which was```
php: /pkg_edit.php: The command '/usr/local/bin/sarg ' returned exit code '1', the output was 'SARG: Unknown sort criterion "SITE" for parameter "topuser_sort_field"'


Inspecting the SARG config, you see this```
# TAG:  topuser_sort_field field normal/reverse
#       Sort field for the Topuser Report.
#       Allowed fields: USER CONNECT BYTES TIME
#
topuser_sort_field SITE NORMAL

The default should be```

TAG: topuser_sort_field field normal/reverse

# Sort field for the Topuser Report.
# Allowed fields: USER CONNECT BYTES TIME

#topuser_sort_field BYTES reverse


It is interesting that I uninstalled the SARG pkg, made sure the sarg.conf file was gone, installed SARG again, and prior to running it checked to make sure the sarg.conf file was still absent. Then I started SARG and clicked save using the default options. The sarg.conf file still contained that SITE NORMAL value. So my error was complaining about the SITE value evidentily not being valid, although something sets it that way even if you do nothing. Unless my install has gone flaky.

Don't know if anyone needs to know that, but once I manually changed that from site to USER or BYTES that particular error stopped and now I am getting this error

php: /pkg_edit.php: The command '/usr/local/bin/sarg ' returned exit code '1', the output was 'SARG: Records in file: 31355, reading: 0.00%^MSARG: Records in file: 5000, reading: 15.95%^MSARG: Records in file: 10000, reading: 31.89%^MSARG: Records in file: 15000, reading: 47.84%^MSARG: Records in file: 20000, reading: 63.79%^MSARG: Records in file: 25000, reading: 79.73%^MSARG: Records in file: 30000, reading: 95.68%^MSARG: cannot open /usr/local/sarg-reports/2013/06/04-09/sarg-date for writing SARG:: No such file or directory SARG: Records in file: 31355, reading: 100.00%'


Thats ok though, the more I bang away on getting this to work, the more I learn about unix based stuff in general. I've just enough geek in me to persevere lol.

sully

Finally got SARG to work again. In my case /conf/config.xml for some reason has SITE used as the <user_sort_field>although I had never messed with the users values at all. Modifying this, via the GUI back to BYTES solved that issue.

Does anyone know if SARG actually works with squidGuard logs? I don't see any denied sites there. If you log the ACL, perhaps SARG does not read those, so you must enable logging on each target category? I accessed sites that were blocked by squidguard for certain, and are in the block.log, but SARG doesn't show any menu for denied sites, nor do I see a way to tell what connections have are from squid or squidguard.

In the sarg.conf file the path to the squidguard block log is correct. Is there something that I am missing that needs to be done here?</user_sort_field>

marcelloc

@sully:

Does anyone know if SARG actually works with squidGuard logs? I don't see any denied sites there.

2.3.6 yes, there were bugs that as crashing sarg on previous versions

Lectrician

Hi.

I have been using Sarge with Squid for months, all working fine across two seperate sites. Reports created daily, just after midnight for the previous day.

On one site I have just added a second interface, and enabled this in squid as the "bind to interface" option.

I then thought perhaps I should check the sarge settings, and checked them, and clicked save, not changing a thing. Now I don't get any reports generated, just the

"Error: Could not find report index file.
Check and save sarg settings and try to force sarg schedule."

error message.

I have checked and saved the settings a few times, tried to force the update, waited 24hr etc.

Any ideas how to get this back up and running?

Thanks.

sully

@Lectrician:

Now I don't get any reports generated, just the

"Error: Could not find report index file.
Check and save sarg settings and try to force sarg schedule."

error message.

I have checked and saved the settings a few times, tried to force the update, waited 24hr etc.

Any ideas how to get this back up and running?

What I have learned is to check the sarg config file, and manually edit the values to what you want and then force the update. If this works, then inspect the GUI and see what is setting values the give conflict. In my case it was a value that I had not set, but was written to the config and caused error.

Also you can look at the system log and see different errors sarg returns. Thats how I started tracking down my errors with sarg.

HTH.

Lectrician

I recall being here before now, and to fix it, I installed CRON, and edited the sarg entry. The entry was for midnight, for the previous day, but was set for the actual day, so changing it to 23:59 gets it working again.

lou

SARG: Init
SARG: Loading configuration from /usr/local/etc/sarg/sarg.conf
SARG: Loading exclude host file from: /usr/local/etc/sarg/exclude_hosts.conf
SARG: Loading exclude file from: /usr/local/etc/sarg/exclude_users.conf
SARG: Reading host alias file "/usr/local/etc/sarg/hostalias"
SARG: List of host names to alias:
SARG: Deleting temporary directory "/tmp/sarg"
SARG: Parameters:
SARG: Hostname or IP address (-a) =
SARG: Useragent log (-b) =
SARG: Exclude file (-c) = /usr/local/etc/sarg/exclude_hosts.conf
SARG: Date from-until (-d) =
SARG: Email address to send reports (-e) =
SARG: Config file (-f) = /usr/local/etc/sarg/sarg.conf
SARG: Date format (-g) = Europe (dd/mm/yyyy)
SARG: IP report (-i) = No
SARG: Keep temporary files (-k) = No
SARG: Input log (-l) = /var/squid/logs/access.log
SARG: Resolve IP Address (-n) = No
SARG: Output dir (-o) = /usr/local/sarg-reports/
SARG: Use Ip Address instead of userid (-p) = No
SARG: Accessed site (-s) =
SARG: Time (-t) =
SARG: User (-u) =
SARG: Temporary dir (-w) = /tmp/sarg
SARG: Debug messages (-x) = Yes
SARG: Process messages (-z) = No
SARG: Previous reports to keep (–lastlog) = 0
SARG:
SARG: sarg version: 2.3.6 Arp-21-2013
SARG: Reading access log file: /var/squid/logs/access.log
SARG: Records in file: 99, reading: 100.00%
SARG: Records read: 99, written: 99, excluded: 0
SARG: Squid log format
SARG: Period: 09 Aug 2013
SARG: File /usr/local/sarg-reports/09Aug2013-09Aug2013 already exists, moved to /usr/local/sarg-reports/09Aug2013-09Aug2013.1
SARG: Sorting log /tmp/sarg/0.user_unsort
SARG: Making file: /tmp/sarg/0
SARG: Sorting log /tmp/sarg/1.user_unsort
SARG: Making file: /tmp/sarg/1
SARG: Sorting log /tmp/sarg/2.user_unsort
SARG: Making file: /tmp/sarg/2
SARG: Sorting log /tmp/sarg/3.user_unsort
SARG: Making file: /tmp/sarg/3
SARG: Sorting log /tmp/sarg/4.user_unsort
SARG: Making file: /tmp/sarg/4
SARG: Sorting log /tmp/sarg/5.user_unsort
SARG: Making file: /tmp/sarg/5
SARG: Sorting log /tmp/sarg/6.user_unsort
SARG: Making file: /tmp/sarg/6
SARG: Sorting log /tmp/sarg/7.user_unsort
SARG: Making file: /tmp/sarg/7
SARG: Sorting log /tmp/sarg/8.user_unsort
SARG: Making file: /tmp/sarg/8
SARG: Sorting log /tmp/sarg/9.user_unsort
SARG: Making file: /tmp/sarg/9
SARG: Cannot delete "/usr/local/sarg-reports/09Aug2013-09Aug2013/d8.html": No such file or directory

Hi all!! I'm trying to run SARG. But when run it. i have this error. I tried to remove all files and remove sarg-reports directory. But still having with the same problem.

do you have Any idea what happening?

Thanks a lot!

scornaky

Hi all,

I had this problem too:
" Error: Could not find report index file.
Check and save sarg settings and try to force sarg schedule."

A working solution for me was :

Report Options :

user graphics
remove temporary files
generate the main index
generate the index tree
overwrite report
use comma instead pint in reports
show de downloaded volume ond date/time reports

Report to generate:

select all

Schedule :

Sarg args: -d date +%d/%m/%Y-date +%d/%m/%Y
frequency: 4h

FORCE UPDATE NOW

and that-s it!

crashdump

got the same error as the other folks around here. The link below fixed the issue.

http://sourceforge.net/p/sarg/discussion/363374/thread/ac055758/

basically, the "date_time_by" parameter on the config file does not have the value needed. Either use "date_time_by bytes" or just comment out the line.

maverick_slo

Well I freshly installed Squid3 + HAVP and SARG reports and now I get this:

php: /pkg_edit.php: The command '/usr/pbi/sarg-i386/bin/sarg ' returned exit code '1', the output was 'SARG: Cannot set the locale LC_ALL to the environment variable'

Any idea?

Thanks!

firefox

Hello
Have you encountered this problem with sarg

I installed the package sarg
I adjusted it
It worked just fine
I do not know what I did
But somehow sarg generates only 10 reports

The package is directed to a new report every hour
And maintain 24 Recent reports
When the 25th hour report comes the report of the of the first hour deleted

Now for some reason
There are only 10 reports
Some old 5 days

What could be the problem?

http://forum.pfsense.org/index.php/topic,66031.0.html

Here is a screenshot of the same day I did the other screenshot from the first Message

Taken second screen is from today

According to settings the report should be maintained 24 hours and erased

As you can see it does not happen

First of all should be more reports

Second report Oldest supposed to be a 24 hour old

If the question disturbs thread
Please delete

MaxHeadroom

Hi,

upgrade to 2.1 and installing sarg
i can see in the system log:

php: /pkg_edit.php: Sarg: force refresh now with args, compress() and none action after sarg finish.
php: /pkg_edit.php: The command '/usr/pbi/sarg-i386/bin/sarg ' returned exit code '1', the output was 'SARG: Cannot set the locale LC_ALL to the environment variable'

But no report is generated
Does anybody know whats wrong ?
regards max

marcelloc

I've updated package today to 0.6.2.

I'll test again if I missed something.

EDIT

my /usr/pbi/sarg-amd64/etc/sarg/sarg.conf is working fine :(

MaxHeadroom

Hi,
i tried to run from ssh and i found out that (some) SARG:TAG: make troubles…
after set a comment # in front of a lot of tags it works now...

have now only this TAG's

SARG: TAG: access_log /var/squid/logs/access.log
SARG: TAG: output_dir /usr/local/sarg-reports
SARG: TAG: resolve_ip no
SARG: TAG: user_ip no
SARG: TAG: index no
SARG: TAG: overwrite_report no
SARG: TAG: privacy no
SARG: TAG: dansguardian_conf
SARG: TAG: denied_report_limit 0
SARG: TAG: sorttable /sarg_sorttable.js
SARG: Deleting temporary directory "/tmp/sarg"

(change something in the sarg config –> restore defect config )

It's realy hard to find out who is (are) the fault one.

But run from web i get still the "Cannot set the locale LC_ALL..." error

readers max

vielfede

Hi,
months ago I configured sarg on a test fw proxy server with squid+squidguard.
I remember "denied sites" were "highlighted" on userid reports by the string "DENIED" placed on right side of denied url, outside the last column(%TIME).

Now I cannot reproduce this feature on my production proxy.
I activated every log option on squid and squidguard: I can see denied sites on squidguard log, but no "DENIED" string appear on sarg report.
Moreover I can't find any "Denied sites" report despite I have enabled that option on "report to generate" section of general tab.

Do you have any idea/same problem?
Thank you in advance

marcelloc

@vielfede:

Do you have any idea/same problem?

While using squidguard, all errors pages will be logged there.
Are your report set to squid or squidguard logs?

vielfede

@marcelloc:

@vielfede:

Do you have any idea/same problem?

While using squidguard, all errors pages will be logged there.
Are your report set to squid or squidguard logs?

Squidguard.

Meanwhile I answered to my question: DENIED "message" appear on sarg report only if the blocked site is in squid blacklist (Access control tab).
I forgot/I did not notice that. Sorry….

It'd be nice if the same sarg feature was reproduced for squidguard blocked sites...
But if i do not get wrong sarg is a closed project....