Sarg package for pfsense
-
Not entirely following you by this suggestion. But I think you are wanting the output of this from?
sarg -x export LC_ALL=C &&export LC_ALL=C && sarg -x
Here we go:
$ export LC_ALL=C && sarg -x SARG: Init SARG: Loading configuration from /usr/local/etc/sarg/sarg.conf SARG: Chaining IP resolving module "dns" SARG: Loading exclude host file from: /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf SARG: Loading exclude file from: /usr/pbi/sarg-i386/etc/sarg/exclude_users.conf SARG: Reading host alias file "/usr/pbi/sarg-i386/etc/sarg/hostalias" SARG: List of host names to alias: SARG: Deleting temporary directory "/tmp/sarg" SARG: Parameters: SARG: Hostname or IP address (-a) = SARG: Useragent log (-b) = SARG: Exclude file (-c) = /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf SARG: Date from-until (-d) = SARG: Email address to send reports (-e) = SARG: Config file (-f) = /usr/local/etc/sarg/sarg.conf SARG: Date format (-g) = USA (mm/dd/yyyy) SARG: IP report (-i) = No SARG: Keep temporary files (-k) = No SARG: Input log (-l) = /var/squid/logs/access.log SARG: Resolve IP Address (-n) = Yes SARG: Output dir (-o) = /usr/local/sarg-reports/ SARG: Use Ip Address instead of userid (-p) = Yes SARG: Accessed site (-s) = SARG: Time (-t) = SARG: User (-u) = SARG: Temporary dir (-w) = /tmp/sarg SARG: Debug messages (-x) = Yes SARG: Process messages (-z) = No SARG: Previous reports to keep (--lastlog) = 0 SARG: SARG: sarg version: 2.3.9 Sep-21-2014 SARG: Loading User table: /usr/pbi/sarg-i386/etc/sarg/usertab.conf SARG: Reading access log file: /var/squid/logs/access.log SARG: Records in file: 1042, reading: 0.00% SARG: Records read: 1042, written: 1042, excluded: 0 SARG: Squid log format SARG: Period: 2015 Feb 04 SARG: File /usr/local/sarg-reports/2015Feb04-2015Feb04 already exists, moved to /usr/local/sarg-reports/2015Feb04-2015Feb04.2 SARG: Sorting log /tmp/sarg/0.user_unsort SARG: Making file: /tmp/sarg/0 SARG: Sorting log /tmp/sarg/1.user_unsort SARG: Making file: /tmp/sarg/1 SARG: Sorting log /tmp/sarg/2.user_unsort SARG: Making file: /tmp/sarg/2 SARG: Sorting log /tmp/sarg/3.user_unsort SARG: Making file: /tmp/sarg/3 SARG: Sorting log /tmp/sarg/4.user_unsort SARG: Making file: /tmp/sarg/4 SARG: Sorting log /tmp/sarg/5.user_unsort SARG: Making file: /tmp/sarg/5 SARG: Sorting log /tmp/sarg/6.user_unsort SARG: Making file: /tmp/sarg/6 SARG: (repday) Cannot open log file /usr/local/sarg-reports/2015Feb04-2015Feb04/5/d5.html SARG: Records in file: 1042, reading: 100.00%
-
OK, making progress. Sarg seems to be one of the more fragile packages. If you happen to select the wrong report options or report to generate, it won't work. Here is what I use and it seems to work OK:
-
KOM: That's pretty odd that something like the chosen report selection is causing this…but that was issue! :o
Is this a bug or is this something that is out of Sarg's control?BTW: Thank you both, KOM and marcelloc!!
-
Is this a bug or is this something that is out of Sarg's control?
Probably a bug in the pfSense Sarg package. Sarg is currently at 2.3.9 while the pfSense package is 2.3.6 so it's 1.5 years older, and as far as I know it's always acted funky like that. Pick the wrong report and the whole thing falls over.
Glad to hear you got it working.
-
looks like its using 2.3.9 on 2.2
https://github.com/pfsense/pfsense-packages/blob/master/pkg_config.10.xml
<depends_on_package_pbi>sarg-2.3.9-##ARCH##.pbi</depends_on_package_pbi>
I haven't downloaded the package myself yet on 2.2
-
I must have been looking at my 2.1.5 box.
-
The only manual fix I had to do on my 2.2 labs was the manual symlink to fix pbi mess.
-
@KOM:
OK, making progress. Sarg seems to be one of the more fragile packages. If you happen to select the wrong report options or report to generate, it won't work. Here is what I use and it seems to work OK:
Hi, I can't get Sarg to produce any report, not even with these settings. Any idea ?
Realtime works fine thoughtI got one Report in the list, which is broken, from last year when I tried it once. How can I delete this report and strat over new ?
-
@Satras:
I got one Report in the list, which is broken, from last year when I tried it once. How can I delete this report and strat over new ?
Did you tried to remove old reports via console/ssh ?
-
I've been digging sarg codes these past days. I tried hacking the template which overrides any changes on the sarg.conf file. I hope we can point the directory of squid rather than have it fixed directory
-
@tux:
rather than have it fixed directory
Do you mean /usr/local/sarg-reports ?
Sarg package needs this to "jail" report access permissions on pfsense gui.
-
No, I mean the sarg.template file. Since whenever there is a change in the config(on the webconfig) it is overridden by the template. So I changed the access log path. Then I also tried creating a folder on a separate drive and symlinked to the default sarg-reports folder but it was a fail. Hope we can configure the path of the access log and same for where to store sarg reports.
-
Hi marcelloc,
i figured out that my sarg dont rotate access.log file. I got a 21GB logfile and wonder why my reports takes so long ;D.
Log: php: /pkg_edit.php: Sarg: force refresh now with -d
date +%d/%m/%Y
args, compress(on) and rotate action after sarg finish.But i saw only cache.log seams to be rotated.
If i use the rotation settings on proxy server tab it works…..but then i have no sarg reports over long period ()eg. 30 days).
Versions:
PfSense 2.1.5 (i386)
squid3 3.1.20 pkg 2.1.2
squidguard 1.4_4 pkg v.1.9.6
havp 0.91_1 pkg v1.05
sarg 2.3.6_2 pkg v.0.6.3
Lightsquid 1.8.2 pkg v.2.33thanks
PS: maybe lightsquid prevent sarg from rotating...so i temporary disabled automatic reports in lightsquid.
-
thanks for the feedback, I'll take a look on rotate call done by sarg.
what pfsense version are you using?
-
hi,
what pfsense version are you using?
PfSense 2.1.5 (i386)
-
Hi I am new to pfsense and i was able to figure out IP sec vpn but i can not get the reports to work nor can I get the realtime to show any dans names.
Can any one help me?
I am running the latest version of pfsense. -
Hi I am new to pfsense and i was able to figure out IP sec vpn but i can not get the reports to work nor can I get the realtime to show any dans names.
Can any one help me?
I am running the latest version of pfsense.Maybe you forgot to enable logging on squid settings.
-
Hi,
My goal with Sarg is to create Squid's daily reports. I have logging and logging rotation both enabled on Squid. My question is why Sarg doesn't show any reports for the previous day (with -d
date -v-1d +%d/%m/%Y
extra args) even though I definitely have yesterday's reports in /var/squid/logs. Is it because of Squid's own logging rotation feature? Do I need to turn it off? Or is it because of 60 minutes restriction on "Find Limit" option? I'm asking this because Sarg easily creates daily reports with -ddate +%d/%m/%Y
extra args.I'm using pfSense 2.2-RELEASE (amd64) with Sarg package 2.3.9 pkg v.0.6.4. Enabled Sarg report options are: 1, 5-10, 13-16.
Thank you!
-
I had a look in the sarg.conf file when I was looking for somthing other and I found that the configuration there only points to the actual access.log file.
access_log /var/log/squid/access.log
So I assume when squid rotates your logs then sarg cannot analyze the logs form access.log.1.
I did some research and found out that since some newer version of sarg (2.3?) - which is installed on pfsense - there is the possibility to set a "*" so sarg analyzes more logfiles.access_log /var/log/squid/access.log*
Source:
http://sourceforge.net/p/sarg/discussion/363374/thread/e2e10ffb/ -
Thanks for your reply!
I modified access_log string in sarg.conf and after that I got "file not found" error (Cannot get the modification time of input log file /var/squid/logs/access.log* (No such file or directory)). So the current version doesn't support this feature.
-
@worldfirst You can not directly edit the sarg.conf since it will be changed backed to its original configuration using the template which is the sarg.template file
-
Yes, I noticed that.
-
pfSense 2.2 (i386)
Squid 3 + SargCan't get report, error message:
Error: Could not find report index file. Check and save sarg settings and try to force sarg schedule.
Interesting thing with 2 almost same commands in console:
1. Don't work. No report in destination directory after run, i.e. dir is empty.sarg -x -f /usr/pbi/sarg-i386/etc/sarg/sarg.conf -o /usr/local/sarg-reports/
2. Works fine. Generates report in destination directory and after linking this dir to /usr/local/sarg-reports i can see reports via pfsense's GUI.
sarg -x -f /usr/pbi/sarg-i386/etc/sarg/sarg.conf -o /sarg-reports/
So, sarg works for 1st level directory but don't work for 3rd level subdir.
Any ideas how to solve problem?
-
I confirm the problem ! :(
-
Try this from the shell:
rm -r /usr/local/sarg-reports
ln -s /usr/pbi/sarg-i386/local/sarg-reports /usr/local/sarg-reportsUse ln -s /usr/pbi/sarg-amd64/local/sarg-reports /usr/local/sarg-reports if you have 64-bit build.
-
@KOM:
Try this from the shell:
rm -r /usr/local/sarg-reports
ln -s /usr/pbi/sarg-i386/local/sarg-reports /usr/local/sarg-reportsThanks! Fixes problem :)
-
My Hardware: APU1C4
only this solved the Problem:
in console
rm -rf /usr/local/sarg-reports
ln -s /usr/pbi/sarg-amd64/local/sarg-reports /usr/local/sarg-reports -
Was having a similar issue glad i found this post fixed the issue for me thanks
-
I ran the above commands to delete the directory and create the symlink to /usr/pbi/sarg-amd64/local/sarg-reports. I can see sarg generated logs under here. However on the 'View Report' tab I still get
Error: Could not find report index file. Check and save sarg settings and try to force sarg schedule.
This is on a fresh pfSense install running 2.2-RELEASE with Sarg 2.3.9 pkg v.0.6.4, Squid3 4.3.10_2 pkg 0.2.6
-
Check your other options. Sarg seems to be finicky and will not work right with the wrong combination of report options. Do you have Generate main index.html and Generate the index tree by file set to Yes, for instance?
-
Thanks KOM - checked the two options you suggested about index.html and they were selected. I then selected all and thought to try that and reduce it to my original set.
After selecting every option, I ran```
sarg -xThanks for the help.
-
for who have this error even after creating symlink /usr/local/sarg-reports to /usr/pbi/sarg-i386/local/sarg-reports /usr/local/sarg-reports or /usr/pbi/sarg-amd64/local/sarg-reports /usr/local/sarg-reports depending on the version you have installed
Error: Could not find report index file. Check and save sarg settings and try to force sarg schedule.
you certainly chosen SquidGuard as proxy server in Sarg general report setting (with Squid it's work fine).
after some debug,i noticed that Sarg is looking for SquidGuard config file in the below location
/usr/pbi/squidguard-i386/etc/squidguard/squidGuard.conf
but the conf file is located in an other folder
/usr/pbi/squidguard-i386/local/etc/squid/squidGuard.conf
for this Srag break generating index report, even if you try to change the folder in the Sarg configuration file, it will always set to the wrong one after each saving Sarg configuration.
i wil tryed to fixe that with symlink but it's dont work,
-
I too am having problems with sarg reports, but having looked at this thread I can't see an identical issue - realtime report is OK but normal reports aren't generated, in syslog I get
php-fpm[68742]: /pkg_edit.php: The command 'export LC_ALL=C && /usr/pbi/sarg-amd64/bin/sarg -d `date +%d/%m/%Y' returned exit code '2', the output was ''
In SSH I get
export: Command not found.
Sarg is 2.3.9 v0.6.4 and pfsense is 2.2.1 (amd64) ie. both the latest.
Any ideas?
thanks
-
Forget that. It was the wrong close quote mark ie should have been:
date +%d/%m/%Y
more dumb questions coming soon.
Oh and add me to the "me too" people who had to do the link command ln -s to get this to work
-
I can't seem to get the report feature to work. Error:
Error: Could not find report index file.
Check and save sarg settings and try to force sarg schedule.
have forced updates etc… but no help. I know it can generate them in the cli by running sarg -x and then if I go into the tmp directory I can see them and read them, however the index file is not being created.
I believe the conf file is correct butTAG: index yes|no|only
# Generate the main index.html.
# only - generate only the main index.htmlindex yes
TAG: index_tree date|file
# How to generate the index.
index_tree file
TAG: output_dir
# The reports will be saved in that directory
# sarg -o diroutput_dir /usr/local/sarg-reports
any useful info would be appreciated
System
2.2.1-RELEASE (amd64)
built on Fri Mar 13 08:16:49 CDT 2015
FreeBSD 10.1-RELEASE-p6
Squid3 -3.4.10_2 pkg 0.2.7
Sarg 2.3.9 pkg v.0.6.4 -
Did you try the symlink fix shown abpve?
-
I have the same problem
"Error: Could not find report index file.
Check and save sarg settings and try to force sarg schedule."I can view the realtime with no problem though.
It appears the reports are being generated but to:
"/usr/pbi/sarg-amd64/local/sarg-reports"I checked the conf file and the report path is: /usr/local/sarg-reports
Tried changing it to:"/usr/pbi/sarg-amd64/local/sarg-reports" but no difference.
Tried creating symb link from "/usr/pbi/sarg-amd64/local/sarg-reports" to /usr/local/sarg-reports with the direction from another post in this thread and yes the reports were there but still the the same error message.I can see I am not the only one with this issue but I have not seen a real solution yet. Is this a bug with squid3 dev or a problem with sarg?
Do I need to manually put in some type of integration parameters as squidguard puts in for itself?
Or if someone has the info on how to configure sarg to send the reports via email that would be good also.
Jabo
-
@KOM:
Did you try the symlink fix shown abpve?
Yes I did, see here
https://forum.pfsense.org/index.php?topic=47765.msg511144#msg511144Thanks
Jabo -
Here is my Sarg in action!! In system log no sarg error and date is correct when i force update sarg reports but In Sarg View reports and Sarg real time are showing old reports!!
2.1.5-RELEASE (i386)
built on Mon Aug 25 07:44:26 EDT 2014
FreeBSD 8.3-RELEASE-p16Intel(R) Atom(TM) CPU N280 @ 1.66GHz
2 CPUs: 1 package(s) x 1 core(s) x 2 HTT threads$ squid -v
Squid Cache: Version 2.7.STABLE9Sarg 2.3.6_2 pkg v.0.6.3
SYSTEM LOG:
Apr 18 02:00:00 php: sarg.php: Sarg: force refresh now with -d
date +%d/%m/%Y
-date +%d/%m/%Y
args, compress(on) and none action after sarg finish.Apr 18 01:17:47 php: /pkg_edit.php: Sarg: force refresh now with -d
date +%d/%m/%Y
-date +%d/%m/%Y
args, compress(on) and none action after sarg finish.
Apr 18 01:17:47 check_reload_status: Syncing firewall
Apr 18 01:17:40 php: /pkg_edit.php: Sarg: force refresh now with -ddate +%d/%m/%Y
-date +%d/%m/%Y
args, compress(on) and none action after sarg finish.SARG VIEW REPORT:
Squid User Access Report
FILE/PERIOD CREATION DATE USERS BYTES AVERAGE
2015Mar23-2015Mar23 Mon Mar 23 23:00:24 2015 3 58,608 19,536
2015Mar22-2015Mar22 Sun Mar 22 23:00:27 2015 18 4,130,379,537 229,465,529
2015Mar21-2015Mar21 Sat Mar 21 23:00:33 2015 37 3,529,371,665 95,388,423
2015Mar20-2015Mar20 Fri Mar 20 23:00:28 2015 32 3,079,436,066 96,232,377
2015Mar19-2015Mar19 Thu Mar 19 23:00:29 2015 19 4,513,906,771 237,574,040
2015Mar18-2015Mar18 Wed Mar 18 23:00:26 2015 17 2,775,934,852 163,290,285
2015Mar17-2015Mar17 Tue Mar 17 23:00:24 2015 19 1,386,602,417 72,979,074
2015Mar16-2015Mar16 Mon Mar 16 23:00:23 2015 15 1,118,870,024 74,591,33SARG REALTIME:
DATE/TIME IP/NAME USERID TYPE ACCESSED SITE
2015-03-23 00:13 10.10.10.171 - GET captive.apple.com
2015-03-23 00:09 10.10.10.33 - GET i.ytimg.com
2015-03-22 23:59 10.10.10.171 - GET captive.apple.com
2015-03-22 23:59 10.10.10.33 - GET safesecuredownload.com
2015-03-22 23:59 10.10.10.33 - GET gomotrak.com -
@KOM:
Try this from the shell:
rm -r /usr/local/sarg-reports
ln -s /usr/pbi/sarg-i386/local/sarg-reports /usr/local/sarg-reportsUse ln -s /usr/pbi/sarg-amd64/local/sarg-reports /usr/local/sarg-reports if you have 64-bit build.
This fix the problem… try it