Sarg package for pfsense

marcelloc

looks like you have no logs on squid file

sarg -x also need the export LC_ALL=C &&

ck42

looks like you have no logs on squid file

sarg -x also need the export LC_ALL=C &&

Not entirely following you by this suggestion. But I think you are wanting the output of this from?
sarg -x export LC_ALL=C &&

If so, the output just shows the same command entered (this is from the GUI). No error messages or anything else.

In looking at the log files in /var/squid/log, I noticed that all of the access.log files are EMPTY (0 byte files). The cache logs look normal though. Also, all the dates on the log files in that dir are current from the last few days.
I'm not sure how to proceed next though to troubleshoot this. Seems to me that this may be the issue (or at least part of it). Why are the access log files not accumulating data?

[EDIT] Might be on to something. I forced an update in a schedule and then noticed that the .0 access log is accumulating. The View Report tab also now no longer gives me the error about the index file.
So what I've done for a test is to disable the log rotation in the report settings. Log rotation is already set for 30 days in the Squid setup.

Still seeing " Cannot set the locale LC_ALL to the environment variable" when I try to run sarg -x though.

Something else that might be helpful from my system.

# LC_ALL=C sarg -x
SARG: Init
SARG: Loading configuration from /usr/local/etc/sarg/sarg.conf
SARG: Chaining IP resolving module "dns"
SARG: Loading exclude host file from: /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
SARG: Loading exclude file from: /usr/pbi/sarg-i386/etc/sarg/exclude_users.conf
SARG: Reading host alias file "/usr/pbi/sarg-i386/etc/sarg/hostalias"
SARG: List of host names to alias:
SARG: Parameters:
SARG:           Hostname or IP address (-a) =
SARG:                    Useragent log (-b) =
SARG:                     Exclude file (-c) = /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
SARG:                  Date from-until (-d) =
SARG:    Email address to send reports (-e) =
SARG:                      Config file (-f) = /usr/local/etc/sarg/sarg.conf
SARG:                      Date format (-g) = Sites & Users (yyyy/ww)
SARG:                        IP report (-i) = No
SARG:             Keep temporary files (-k) = No
SARG:                        Input log (-l) = /var/squid/logs/access.log
SARG:               Resolve IP Address (-n) = Yes
SARG:                       Output dir (-o) = /usr/local/sarg-reports/
SARG: Use Ip Address instead of userid (-p) = Yes
SARG:                    Accessed site (-s) =
SARG:                             Time (-t) =
SARG:                             User (-u) =
SARG:                    Temporary dir (-w) = /tmp/sarg
SARG:                   Debug messages (-x) = Yes
SARG:                 Process messages (-z) = No
SARG:  Previous reports to keep (--lastlog) = 0
SARG:
SARG: sarg version: 2.3.9 Sep-21-2014
SARG: Loading User table: /usr/pbi/sarg-i386/etc/sarg/usertab.conf
SARG: Reading access log file: /var/squid/logs/access.log
SARG: Records in file: 174, reading: 100.00%
SARG:    Records read: 174, written: 174, excluded: 0
SARG: Squid log format
SARG: Period: 2015.05
SARG: Sorting log /tmp/sarg/0.user_unsort
SARG: Sorting log /tmp/sarg/1.user_unsort
SARG: Sorting log /tmp/sarg/2.user_unsort
SARG: (repday) Cannot open log file /usr/local/sarg-reports/2015.05/0/d0.html

Regarding that very last line of output, here's what in the 2015.05 directory:

# ls -la /usr/local/sarg-reports/2015.05
total 18
drwxr-xr-x  2 root  wheel   512 Feb  4 11:54 .
drwxr-xr-x  5 root  wheel   512 Feb  4 11:54 ..
-rw-r--r--  1 root  wheel  4437 Feb  4 11:54 index.html
-rw-r--r--  1 root  wheel    22 Feb  4 11:54 sarg-date
-rw-r--r--  1 root  wheel  1398 Feb  4 11:54 sarg-general
-rw-r--r--  1 root  wheel     2 Feb  4 11:54 sarg-users
-rw-r--r--  1 root  wheel   116 Feb  4 11:54 top

So it's correct in that there's no "0" directory in which to find the d0.html file it's looking for.
A system wide search for this file DOES show that a copy exist here though.
/usr/pbi/sarg-i386/local/sarg-reports/2015.01.1/0/d0.html

And lastly, the Realtime logging appears to be working correctly.

marcelloc

@ck42:

Not entirely following you by this suggestion. But I think you are wanting the output of this from?
sarg -x export LC_ALL=C &&

export LC_ALL=C && sarg -x

ck42

@marcelloc:

@ck42:

Not entirely following you by this suggestion. But I think you are wanting the output of this from?
sarg -x export LC_ALL=C &&

export LC_ALL=C && sarg -x

Here we go:

$ export LC_ALL=C && sarg -x
SARG: Init
SARG: Loading configuration from /usr/local/etc/sarg/sarg.conf
SARG: Chaining IP resolving module "dns"
SARG: Loading exclude host file from: /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
SARG: Loading exclude file from: /usr/pbi/sarg-i386/etc/sarg/exclude_users.conf
SARG: Reading host alias file "/usr/pbi/sarg-i386/etc/sarg/hostalias"
SARG: List of host names to alias:
SARG: Deleting temporary directory "/tmp/sarg"
SARG: Parameters:
SARG:           Hostname or IP address (-a) = 
SARG:                    Useragent log (-b) = 
SARG:                     Exclude file (-c) = /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
SARG:                  Date from-until (-d) = 
SARG:    Email address to send reports (-e) = 
SARG:                      Config file (-f) = /usr/local/etc/sarg/sarg.conf
SARG:                      Date format (-g) = USA (mm/dd/yyyy)
SARG:                        IP report (-i) = No
SARG:             Keep temporary files (-k) = No
SARG:                        Input log (-l) = /var/squid/logs/access.log
SARG:               Resolve IP Address (-n) = Yes
SARG:                       Output dir (-o) = /usr/local/sarg-reports/
SARG: Use Ip Address instead of userid (-p) = Yes
SARG:                    Accessed site (-s) = 
SARG:                             Time (-t) = 
SARG:                             User (-u) = 
SARG:                    Temporary dir (-w) = /tmp/sarg
SARG:                   Debug messages (-x) = Yes
SARG:                 Process messages (-z) = No
SARG:  Previous reports to keep (--lastlog) = 0
SARG: 
SARG: sarg version: 2.3.9 Sep-21-2014
SARG: Loading User table: /usr/pbi/sarg-i386/etc/sarg/usertab.conf
SARG: Reading access log file: /var/squid/logs/access.log
SARG: Records in file: 1042, reading: 0.00%
SARG:    Records read: 1042, written: 1042, excluded: 0
SARG: Squid log format
SARG: Period: 2015 Feb 04
SARG: File /usr/local/sarg-reports/2015Feb04-2015Feb04 already exists, moved to /usr/local/sarg-reports/2015Feb04-2015Feb04.2
SARG: Sorting log /tmp/sarg/0.user_unsort
SARG: Making file: /tmp/sarg/0
SARG: Sorting log /tmp/sarg/1.user_unsort
SARG: Making file: /tmp/sarg/1
SARG: Sorting log /tmp/sarg/2.user_unsort
SARG: Making file: /tmp/sarg/2
SARG: Sorting log /tmp/sarg/3.user_unsort
SARG: Making file: /tmp/sarg/3
SARG: Sorting log /tmp/sarg/4.user_unsort
SARG: Making file: /tmp/sarg/4
SARG: Sorting log /tmp/sarg/5.user_unsort
SARG: Making file: /tmp/sarg/5
SARG: Sorting log /tmp/sarg/6.user_unsort
SARG: Making file: /tmp/sarg/6
SARG: (repday) Cannot open log file /usr/local/sarg-reports/2015Feb04-2015Feb04/5/d5.html
SARG: Records in file: 1042, reading: 100.00%

KOM

OK, making progress. Sarg seems to be one of the more fragile packages. If you happen to select the wrong report options or report to generate, it won't work. Here is what I use and it seems to work OK:

sarg.png_thumb

ck42

KOM: That's pretty odd that something like the chosen report selection is causing this…but that was issue! :o
Is this a bug or is this something that is out of Sarg's control?

BTW: Thank you both, KOM and marcelloc!!

KOM

Is this a bug or is this something that is out of Sarg's control?

Probably a bug in the pfSense Sarg package. Sarg is currently at 2.3.9 while the pfSense package is 2.3.6 so it's 1.5 years older, and as far as I know it's always acted funky like that. Pick the wrong report and the whole thing falls over.

Glad to hear you got it working.

Cino

looks like its using 2.3.9 on 2.2

https://github.com/pfsense/pfsense-packages/blob/master/pkg_config.10.xml

<depends_on_package_pbi>sarg-2.3.9-##ARCH##.pbi</depends_on_package_pbi>

I haven't downloaded the package myself yet on 2.2

KOM

I must have been looking at my 2.1.5 box.

marcelloc

The only manual fix I had to do on my 2.2 labs was the manual symlink to fix pbi mess.

A Former User

@KOM:

OK, making progress. Sarg seems to be one of the more fragile packages. If you happen to select the wrong report options or report to generate, it won't work. Here is what I use and it seems to work OK:

Hi, I can't get Sarg to produce any report, not even with these settings. Any idea ?
Realtime works fine thought

I got one Report in the list, which is broken, from last year when I tried it once. How can I delete this report and strat over new ?

marcelloc

@Satras:

I got one Report in the list, which is broken, from last year when I tried it once. How can I delete this report and strat over new ?

Did you tried to remove old reports via console/ssh ?

tux

I've been digging sarg codes these past days. I tried hacking the template which overrides any changes on the sarg.conf file. I hope we can point the directory of squid rather than have it fixed directory

marcelloc

@tux:

rather than have it fixed directory

Do you mean /usr/local/sarg-reports ?

Sarg package needs this to "jail" report access permissions on pfsense gui.

tux

No, I mean the sarg.template file. Since whenever there is a change in the config(on the webconfig) it is overridden by the template. So I changed the access log path. Then I also tried creating a folder on a separate drive and symlinked to the default sarg-reports folder but it was a fail. Hope we can configure the path of the access log and same for where to store sarg reports.

spiritbreaker

Hi marcelloc,

i figured out that my sarg dont rotate access.log file. I got a 21GB logfile and wonder why my reports takes so long ;D.

Log: php: /pkg_edit.php: Sarg: force refresh now with -d date +%d/%m/%Y args, compress(on) and rotate action after sarg finish.

But i saw only cache.log seams to be rotated.

If i use the rotation settings on proxy server tab it works…..but then i have no sarg reports over long period ()eg. 30 days).

Versions:

PfSense 2.1.5 (i386)

squid3 3.1.20 pkg 2.1.2
squidguard 1.4_4 pkg v.1.9.6
havp 0.91_1 pkg v1.05
sarg 2.3.6_2 pkg v.0.6.3
Lightsquid 1.8.2 pkg v.2.33

thanks

PS: maybe lightsquid prevent sarg from rotating...so i temporary disabled automatic reports in lightsquid.

marcelloc

thanks for the feedback, I'll take a look on rotate call done by sarg.

what pfsense version are you using?

spiritbreaker

hi,

what pfsense version are you using?

PfSense 2.1.5 (i386)

djpeach310

Hi I am new to pfsense and i was able to figure out IP sec vpn but i can not get the reports to work nor can I get the realtime to show any dans names.
Can any one help me?
I am running the latest version of pfsense.

marcelloc

@djpeach310:

Hi I am new to pfsense and i was able to figure out IP sec vpn but i can not get the reports to work nor can I get the realtime to show any dans names.
Can any one help me?
I am running the latest version of pfsense.

Maybe you forgot to enable logging on squid settings.