Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Sarg package for pfsense

    Scheduled Pinned Locked Moved pfSense Packages
    467 Posts 99 Posters 501.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N
      Nachtfalke
      last edited by

      Did you enable logging on squid and/or dansguardian?
      Did you create a shedule on sarg?
      Only enable logrotation on sarg and not on squid.

      Depending on what reports sarg should create and how big the log files are it could take some time to generate a report.

      1 Reply Last reply Reply Quote 0
      • J
        jpmunroe
        last edited by

        Seems as if it was a time thing. After making no changes and just checking the reporting now, it is working.

        1 Reply Last reply Reply Quote 0
        • N
          Nachtfalke
          last edited by

          @jpmunroe:

          Seems as if it was a time thing. After making no changes and just checking the reporting now, it is working.

          I did have this sometimes when creating a shedule for the first time. Doing a force update now does not work or it seems as it does not but later everything is running.

          1 Reply Last reply Reply Quote 0
          • D
            Daksh
            last edited by

            I have installed squid(logging enabled) and i have set transparent mode off(if i switch it on i cannot access most of the websites)
            after that if i try sarg, in report: it doesnt show anything(it says you have not configured….) and if i go in realtime and check then it shows only 1 entry of txt/...

            i have tried to install this soo many times earlier once it showed websites in realtime but only once upon a time that too of only 15 minutes(approx.)

            Please help...


            I am a newbie

            1 Reply Last reply Reply Quote 0
            • C
              ck42
              last edited by

              Getting the same message:
              rror: Could not find report index file.
              Check and save sarg settings and try to force sarg schedule.

              But I can see the index.html file and other related files in /usr/local/sarg-reports/2014Jun04-2014Jun05

              So…I simply copied the index.html up one level and into /usr/local/sarg-reports
              the .conf file has the output_dir set to /usr/local/sarg-reports
              But...how does this work if files are being placed in these dated sub directories??

              This seems to have fixed things.
              Bug?  A configuration setting I need to change somewhere?

              [EDIT]
              Fiddled around with some settings and now it's apparently working?
              I'm wondering if disabling the logging function and then re-enabling did something?

              1 Reply Last reply Reply Quote 0
              • P
                peruvichito2014
                last edited by

                Hi Gurus
                After following the recommendation that I was learning in this page:
                I copy the "index.html" from /usr/local/sarg-reports/2014/06/30/ to /usr/local/sarg-reports/

                But I Can not obtain any value:

                Review in the log file, I observed
                ![](http://Log Sarg.jpg)

                Any suggestion why I can not obtain any report:

                In the real Time report I can observe the pages that my customer access:

                Any suggestion

                Proxy-Report.jpg
                Proxy-Report.jpg_thumb
                ![Log Sarg.jpg](/public/imported_attachments/1/Log Sarg.jpg)
                ![Log Sarg.jpg_thumb](/public/imported_attachments/1/Log Sarg.jpg_thumb)
                Satus_Sarg_Realtime.jpg
                Satus_Sarg_Realtime.jpg_thumb

                1 Reply Last reply Reply Quote 0
                • P
                  peruvichito2014
                  last edited by

                  Gurus
                  After running this command:

                  [2.1.3-RELEASE][admin@firewall.hejcu.gob.pe]/root(4): sarg -x
                  SARG: Init
                  SARG: Loading configuration from /usr/pbi/sarg-i386/etc/sarg/sarg.conf
                  SARG: Chaining IP resolving module "dns"
                  SARG: Loading exclude host file from: /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
                  SARG: Loading exclude file from: /usr/pbi/sarg-i386/etc/sarg/exclude_users.conf
                  SARG: Reading host alias file "/usr/pbi/sarg-i386/etc/sarg/hostalias"
                  SARG: List of host names to alias:
                  SARG: Deleting temporary directory "/tmp/sarg"
                  SARG: Parameters:
                  SARG:          Hostname or IP address (-a) =
                  SARG:                    Useragent log (-b) =
                  SARG:                    Exclude file (-c) = /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
                  SARG:                  Date from-until (-d) =
                  SARG:    Email address to send reports (-e) =
                  SARG:                      Config file (-f) = /usr/pbi/sarg-i386/etc/sarg/sarg.conf
                  SARG:                      Date format (-g) = USA (mm/dd/yyyy)
                  SARG:                        IP report (-i) = No
                  SARG:            Keep temporary files (-k) = No
                  SARG:                        Input log (-l) = /var/squid/logs/access.log
                  SARG:              Resolve IP Address (-n) = Yes
                  SARG:                      Output dir (-o) = /usr/local/sarg-reports/
                  SARG: Use Ip Address instead of userid (-p) = No
                  SARG:                    Accessed site (-s) =
                  SARG:                            Time (-t) =
                  SARG:                            User (-u) =
                  SARG:                    Temporary dir (-w) = /tmp/sarg
                  SARG:                  Debug messages (-x) = Yes
                  SARG:                Process messages (-z) = No
                  SARG:  Previous reports to keep (–lastlog) = 0
                  SARG:
                  SARG: sarg version: 2.3.6 Arp-21-2013
                  SARG: Reading access log file: /var/squid/logs/access.log
                  SARG: Records in file: 970, reading: 100.00%
                  SARG:    Records read: 970, written: 970, excluded: 0
                  SARG: Squid log format
                  SARG: Period: 2014 Jun 30
                  SARG: Sorting log /tmp/sarg/192_168_1_49.user_unsort
                  SARG: Sorting log /tmp/sarg/192_168_1_10.user_unsort
                  SARG: Sorting log /tmp/sarg/user2.user_unsort
                  SARG: Sorting log /tmp/sarg/192_168_1_71.user_unsort
                  SARG: Sorting log /tmp/sarg/192_168_1_9.user_unsort
                  SARG: (repday) Cannot open log file /usr/local/sarg-reports/2014/06/30/192_168_1_49/d192_168_1_49.html

                  [2.1.3-RELEASE][admin@firewall.x.x.x.x]/root(1): sarg -x
                  SARG: Init
                  SARG: Loading configuration from /usr/pbi/sarg-i386/etc/sarg/sarg.conf
                  SARG: Chaining IP resolving module "dns"
                  SARG: Loading exclude host file from: /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
                  SARG: Loading exclude file from: /usr/pbi/sarg-i386/etc/sarg/exclude_users.conf
                  SARG: Reading host alias file "/usr/pbi/sarg-i386/etc/sarg/hostalias"
                  SARG: List of host names to alias:
                  SARG: Deleting temporary directory "/tmp/sarg"
                  SARG: Parameters:
                  SARG:          Hostname or IP address (-a) =
                  SARG:                    Useragent log (-b) =
                  SARG:                    Exclude file (-c) = /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
                  SARG:                  Date from-until (-d) =
                  SARG:    Email address to send reports (-e) =
                  SARG:                      Config file (-f) = /usr/pbi/sarg-i386/etc/sarg/sarg.conf
                  SARG:                      Date format (-g) = USA (mm/dd/yyyy)
                  SARG:                        IP report (-i) = No
                  SARG:            Keep temporary files (-k) = No
                  SARG:                        Input log (-l) = /var/squid/logs/access.log
                  SARG:              Resolve IP Address (-n) = Yes
                  SARG:                      Output dir (-o) = /usr/local/sarg-reports/
                  SARG: Use Ip Address instead of userid (-p) = No
                  SARG:                    Accessed site (-s) =
                  SARG:                            Time (-t) =
                  SARG:                            User (-u) =
                  SARG:                    Temporary dir (-w) = /tmp/sarg
                  SARG:                  Debug messages (-x) = Yes
                  SARG:                Process messages (-z) = No
                  SARG:  Previous reports to keep (–lastlog) = 0
                  SARG:
                  SARG: sarg version: 2.3.6 Arp-21-2013
                  SARG: Reading access log file: /var/squid/logs/access.log
                  SARG: Records in file: 970, reading: 100.00%
                  SARG:    Records read: 970, written: 970, excluded: 0
                  SARG: Squid log format
                  SARG: Period: 2014 Jun 30
                  SARG: Sorting log /tmp/sarg/192_168_1_49.user_unsort
                  SARG: Sorting log /tmp/sarg/192_168_1_10.user_unsort
                  SARG: Sorting log /tmp/sarg/user2.user_unsort
                  SARG: Sorting log /tmp/sarg/192_168_1_71.user_unsort
                  SARG: Sorting log /tmp/sarg/192_168_1_9.user_unsort
                  SARG: (repday) Cannot open log file /usr/local/sarg-reports/2014/06/30/192_168_1_49/d192_168_1_49.html

                  [2.1.3-RELEASE][admin@firewall.x.x.x.x]/root(4): sarg -x
                  SARG: Init
                  SARG: Loading configuration from /usr/pbi/sarg-i386/etc/sarg/sarg.conf
                  SARG: Chaining IP resolving module "dns"
                  SARG: Loading exclude host file from: /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
                  SARG: Loading exclude file from: /usr/pbi/sarg-i386/etc/sarg/exclude_users.conf
                  SARG: Reading host alias file "/usr/pbi/sarg-i386/etc/sarg/hostalias"
                  SARG: List of host names to alias:
                  SARG: Deleting temporary directory "/tmp/sarg"
                  SARG: Parameters:
                  SARG:          Hostname or IP address (-a) =
                  SARG:                    Useragent log (-b) =
                  SARG:                    Exclude file (-c) = /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
                  SARG:                  Date from-until (-d) =
                  SARG:    Email address to send reports (-e) =
                  SARG:                      Config file (-f) = /usr/pbi/sarg-i386/etc/sarg/sarg.conf
                  SARG:                      Date format (-g) = USA (mm/dd/yyyy)
                  SARG:                        IP report (-i) = No
                  SARG:            Keep temporary files (-k) = No
                  SARG:                        Input log (-l) = /var/squid/logs/access.log
                  SARG:              Resolve IP Address (-n) = Yes
                  SARG:                      Output dir (-o) = /usr/local/sarg-reports/
                  SARG: Use Ip Address instead of userid (-p) = No
                  SARG:                    Accessed site (-s) =
                  SARG:                            Time (-t) =
                  SARG:                            User (-u) =
                  SARG:                    Temporary dir (-w) = /tmp/sarg
                  SARG:                  Debug messages (-x) = Yes
                  SARG:                Process messages (-z) = No
                  SARG:  Previous reports to keep (–lastlog) = 0
                  SARG:
                  SARG: sarg version: 2.3.6 Arp-21-2013
                  SARG: Reading access log file: /var/squid/logs/access.log
                  SARG: Records in file: 970, reading: 100.00%
                  SARG:    Records read: 970, written: 970, excluded: 0
                  SARG: Squid log format
                  SARG: Period: 2014 Jun 30
                  SARG: Sorting log /tmp/sarg/192_168_1_49.user_unsort
                  SARG: Sorting log /tmp/sarg/192_168_1_10.user_unsort
                  SARG: Sorting log /tmp/sarg/user2.user_unsort
                  SARG: Sorting log /tmp/sarg/192_168_1_71.user_unsort
                  SARG: Sorting log /tmp/sarg/192_168_1_9.user_unsort
                  SARG: (repday) Cannot open log file /usr/local/sarg-reports/2014/06/30/192_168_1_49/d192_168_1_49.html
                  [2.1.3-RELEASE][admin@firewall.x.x.x.x]/root(6): cat /usr/pbi/sarg-i386/etc/sarg/sarg.conf | more

                  sarg.conf

                  TAG:  access_log file

                  #      Where is the access.log
                  #      sarg -l file

                  access_log /var/squid/logs/access.log

                  TAG: graphs yes|no

                  #      Use graphics where is possible.
                  #          graph_days_bytes_bar_color blue|green|yellow|orange|brown|red

                  graphs yes
                  #graph_days_bytes_bar_color orange

                  TAG:  graph_font

                  #      The full path to the TTF font file to use to create the graphs. It is required
                  #      if graphs is set to yes.

                  #graph_font /usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf

                  TAG:  title

                  #      Especify the title for html page.

                  #title "Squid User Access Reports"

                  TAG:  font_face

                  #      Especify the font for html page.

                  #font_face Tahoma,Verdana,Arial

                  TAG:  header_color

                  #      Especify the header color

                  #header_color darkblue

                  TAG:  header_bgcolor

                  #      Especify the header bgcolor

                  #header_bgcolor blanchedalmond

                  TAG:  font_size

                  #      Especify the text font size

                  #font_size 9px

                  TAG:  header_font_size

                  #      Especify the header font size

                  #header_font_size 9px

                  TAG:  title_font_size

                  #      Especify the title font size

                  #title_font_size 11px

                  TAG:  background_color

                  TAG:  background_color

                  #      Html page background color

                  background_color white

                  TAG:  text_color

                  #      Html page text color

                  #text_color #000000

                  TAG:  text_bgcolor

                  #      Html page text background color

                  #text_bgcolor lavender

                  TAG:  title_color

                  #      Html page title color

                  #title_color green

                  TAG:  logo_image

                  #      Html page logo.

                  #logo_image none

                  TAG:  logo_text

                  #      Html page logo text.

                  #logo_text ""

                  TAG:  logo_text_color

                  #      Html page logo texti color.

                  #logo_text_color #000000

                  TAG:  logo_image_size

                  #      Html page logo image size.
                  #      width height

                  #image_size 80 45

                  TAG:  background_image

                  #      Html page background image

                  #background_image none

                  TAG:  password

                  #      User password file used by Squid authentication scheme
                  #      If used, generate reports just for that users.

                  #password none

                  TAG:  temporary_dir

                  #      Temporary directory name for work files
                  #      sarg -w dir

                  #temporary_dir /tmp

                  TAG:  output_dir

                  #      The reports will be saved in that directory
                  #      sarg -o dir

                  output_dir /usr/local/sarg-reports

                  TAG:  anonymous_output_files yes/no

                  #      Use anonymous file and directory names in the report. If it is set to
                  #      no (the default), the user id/ip/name is slightly mangled to create a
                  #      suitable file name to store the report of the user but the user's
                  #      identity can easily be guessed from the mangled name. If this option is
                  #      set, any file or directory belonging to the user is replaced by a short
                  #      number.  The purpose is to hide the identity of the user when looking
                  #      at the report file names but it may serve to shorten the path too.

                  anonymous_output_files no

                  TAG:  output_email

                  #      Email address to send the reports. If you use this tag, no html reports will be generated.
                  #      sarg -e email

                  #output_email none

                  TAG:  resolve_ip yes/no

                  #      Convert ip address to dns name
                  #      sarg -n
                  resolve_ip yes

                  TAG:  user_ip yes/no

                  #      Use Ip Address instead userid in reports.
                  #      sarg -p
                  user_ip no

                  TAG:  topuser_sort_field field normal/reverse

                  #      Sort field for the Topuser Report.
                  #      Allowed fields: USER CONNECT BYTES TIME

                  topuser_sort_field TIME normal

                  TAG:  user_sort_field field normal/reverse

                  #      Sort field for the User Report.
                  #      Allowed fields: SITE CONNECT BYTES TIME

                  user_sort_field TIME normal

                  TAG:  exclude_users file

                  #      users within the file will be excluded from reports.
                  #      you can use indexonly to have only index.html file.

                  exclude_users /usr/pbi/sarg-i386/etc/sarg/exclude_users.conf

                  TAG:  exclude_hosts file

                  #      Hosts, domains or subnets will be excluded from reports.

                  #      Eg.: 192.168.10.10  - exclude ip address only
                  #            192.168.10.0/24 - exclude full C class
                  #            s1.acme.foo    - exclude hostname only
                  #            *.acme.foo      - exclude full domain name

                  exclude_hosts /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf

                  TAG:  useragent_log file

                  #      useragent.log file patch to generate useragent report.

                  #useragent_log none

                  TAG:  date_format

                  #      Date format in reports: e (European=dd/mm/yy), u (American=mm/dd/yy), w (Weekly=yy.ww)

                  #date_format u
                  date_format u

                  TAG:  per_user_limit file MB

                  #      Saves userid on file if download exceed n MB.
                  #      This option allow you to disable user access if user exceed a download limit.

                  #per_user_limit none

                  TAG: lastlog n

                  #      How many reports files must be kept in reports directory.
                  #      The oldest report file will be automatically removed.
                  #      0 - no limit.

                  #lastlog 0
                  lastlog 0

                  TAG: remove_temp_files yes

                  #      Remove temporary files: geral, usuarios, top, periodo from root report directory.

                  remove_temp_files yes

                  TAG: index yes|no|only

                  #      Generate the main index.html.
                  #      only - generate only the main index.html

                  index yes

                  TAG: index_tree date|file

                  #      How to generate the index.

                  index_tree date

                  TAG: index_fields

                  #      The columns to show in the index of the reports
                  #      Columns are: dirsize

                  #index_fields dirsize

                  TAG: overwrite_report yes|no

                  #      yes - if report date already exist then will be overwrited.
                  #      no - if report date already exist then will be renamed to filename.n, filename.n+1

                  overwrite_report yes

                  TAG: records_without_userid ignore|ip|everybody

                  #      What can I do with records without user id (no authentication) in access.log file ?

                  #      ignore - This record will be ignored.
                  #          ip - Use ip address instead. (default)
                  #  everybody - Use "everybody" instead.

                  #records_without_userid ip

                  TAG: use_comma no|yes

                  #      Use comma instead point in reports.
                  #      Eg.: use_comma yes => 23,450,110
                  #          use_comma no  => 23.450.110

                  use_comma yes

                  TAG: mail_utility

                  #      Mail command to use to send reports via SMTP. Sarg calls it like this:
                  #        mail_utility -s "SARG report, date" "output_email" <"mail_content"

                  #      Therefore, it is possible to add more arguments to the command by specifying them
                  #      here.

                  #      If you need too, you can use a shell script to process the content of /dev/stdin
                  #      (/dev/stdin is the mail_content passed by sarg to the script) and call whatever
                  #      command you like. It is not limited to mailing the report via SMTP.

                  #      Don't forget to quote the command if necessary (i.e. if the path contains
                  #      characters that must be quoted).

                  #mail_utility mailx

                  TAG: topsites_num n

                  #      How many sites in topsites report.

                  #topsites_num 100

                  TAG: topsites_sort_order CONNECT|BYTES|TIME A|D

                  #      Sort for topsites report, where A=Ascendent, D=Descendent

                  #topsites_sort_order CONNECT D

                  TAG: index_sort_order A/D

                  #      Sort for index.html, where A=Ascendent, D=Descendent

                  #index_sort_order D

                  TAG: exclude_codes file

                  #      Ignore records with these codes. Eg.: NONE/400
                  #      Write one code per line. Lines starting with a # are ignored.
                  #      Only codes matching exactly one of the line is rejected. The
                  #      comparison is not case sensitive.

                  exclude_codes /usr/pbi/sarg-i386/etc/sarg/exclude_codes

                  TAG: replace_index string

                  #      Replace "index.html" in the main index file with this string
                  #      If null "index.html" is used

                  #replace_index

                  TAG: max_elapsed milliseconds

                  #      If elapsed time is recorded in log is greater than max_elapsed use 0 for elapsed time.
                  #      Use 0 for no checking

                  #max_elapsed 28800000

                  8 Hours

                  max_elapsed 0

                  TAG: report_type type

                  #      What kind of reports to generate.
                  #      topusers            - users, sites, times, bytes, connects, links to accessed sites, etc
                  #      topsites            - site, connect and bytes report
                  #      sites_users        - users and sites report
                  #      users_sites        - accessed sites by the user report
                  #      date_time          - bytes used per day and hour report
                  #      denied              - denied sites with full URL report
                  #      auth_failures      - autentication failures report
                  #      site_user_time_date - sites, dates, times and bytes report
                  #      downloads          - downloads per user report

                  #      Eg.: report_type topsites denied

                  #report_type topusers topsites sites_users users_sites date_time denied auth_failures site_user_time_date downloads
                  report_type topusers

                  TAG: usertab filename

                  #      You can change the "userid" or the "ip address" to be a real user name on the reports.
                  #      If resolve_ip is active, the ip address is resolved before being looked up into this
                  #      file. That is, if you want to map the ip address, be sure to set resolv_ip to no or
                  #      the resolved name will be looked into the file instead of the ip address. Note that
                  #      it can be used to resolve any ip address known to the dns and then map the unresolved
                  #      ip addresses to a name found in the usertab file.
                  #      Table syntax:
                  #              userid name  or  ip address name
                  #      Eg:
                  #              SirIsaac Isaac Newton
                  #              vinci Leonardo da Vinci
                  #              192.168.10.1 Karol Wojtyla

                  #      Each line must be terminated with '\ n'
                  #      If usertab have value "ldap" (case ignoring), user names
                  #      will be taken from LDAP server. This method as approaches for reception
                  #      of usernames from Active Didectory

                  #usertab none
                  usertab none

                  TAG: LDAPHost hostname

                  #      FQDN or IP address of host with LDAP service or AD DC
                  #      default is '127.0.0.1'
                  #LDAPHost 127.0.0.1

                  TAG: LDAPPort port

                  #      LDAP service port number
                  #      default is '389'
                  #LDAPPort 389

                  TAG: LDAPBindDN CN=username,OU=group,DC=mydomain,DC=com

                  #      DN of LDAP user, who is authorized to read user's names from LDAP base
                  #      default is empty line
                  #LDAPBindDN cn=proxy,dc=mydomain,dc=local

                  TAG: LDAPBindPW secret

                  #      Password of DN, who is authorized to read user's names from LDAP base
                  #      default is empty line
                  #LDAPBindPW secret

                  TAG: LDAPBaseSearch OU=users,DC=mydomain,DC=com

                  #      LDAP search base
                  #      default is empty line
                  #LDAPBaseSearch ou=users,dc=mydomain,dc=local

                  TAG: LDAPFilterSearch (uid=%s)

                  #      User search filter by user's logins in LDAP
                  #      First founded record will be used
                  #      %s - will be changed to userlogins from access.log file
                  #      filter string can have up to 5 '%s' tags
                  #      default value is '(uid=%s)'
                  #LDAPFilterSearch (uid=%s)

                  TAG: LDAPTargetAttr attributename

                  #      Name of the attribute containing a name of the user
                  #      default value is 'cn'
                  #LDAPTargetAttr cn

                  TAG: long_url yes|no

                  #      If yes, the full url is showed in report.
                  #      If no, only the site will be showed

                  #      YES option generate very big sort files and reports.

                  long_url no

                  TAG: date_time_by bytes|elap

                  #      Date/Time reports show the downloaded volume or the elapsed time or both.

                  #date_time_by bytes
                  date_time_by bytes

                  TAG: charset name

                  #      ISO 8859 is a full series of 10 standardized multilingual single-byte coded (8bit)
                  #      graphic character sets for writing in alphabetic languages
                  #      You can use the following charsets:
                  #              Latin1          - West European
                  #              Latin2          - East European
                  #              Latin3          - South European
                  #              Latin4          - North European
                  #              Cyrillic
                  #              Arabic
                  #              Greek
                  #              Hebrew
                  #              Latin5          - Turkish
                  #              Latin6
                  #              Windows-1251
                  #              Japan
                  #              Koi8-r
                  #              UTF-8

                  #charset Latin1
                  charset UTF-8

                  TAG: user_invalid_char "&/"

                  #      Records that contain invalid characters in userid will be ignored by Sarg.

                  #user_invalid_char "&/"

                  TAG: privacy yes|no

                  #      privacy_string "..."
                  #      privacy_string_color blue
                  #      In some countries the sysadm cannot see the visited sites by a restrictive law.
                  #      Using privacy yes the visited url will be changes by privacy_string and the link
                  #      will be removed from reports.

                  privacy no
                  #privacy_string "..."
                  #privacy_string_color blue

                  TAG: include_users "user1:user2:…:usern"

                  #      Reports will be generated only for listed users.

                  #include_users none

                  TAG: exclude_string "string1:string2:...:stringn"

                  #      Records from access.log file that contain one of listed strings will be ignored.

                  #exclude_string none

                  TAG: show_successful_message yes|no

                  #      Shows "Successful report generated on dir" at end of process.

                  #show_successful_message yes

                  TAG: show_read_statistics yes|no

                  #      Shows some reading statistics.

                  #show_read_statistics yes

                  TAG: topuser_fields

                  #      Which fields must be in Topuser report.

                  #topuser_fields NUM DATE_TIME USERID CONNECT BYTES %BYTES IN-CACHE-OUT USED_TIME MILISEC %TIME TOTAL AVERAGE

                  TAG: user_report_fields

                  #      Which fields must be in User report.

                  #user_report_fields CONNECT BYTES %BYTES IN-CACHE-OUT USED_TIME MILISEC %TIME TOTAL AVERAGE

                  TAG: bytes_in_sites_users_report yes|no

                  #      Bytes field must be in Site & Users Report ?

                  #bytes_in_sites_users_report no
                  bytes_in_sites_users_report no

                  TAG: topuser_num n

                  #      How many users in topsites report. 0 = no limit

                  #topuser_num 0
                  topuser_num 0

                  TAG: datafile file

                  #      Save the report results in a file to populate some database

                  #datafile none

                  TAG: datafile_delimiter "    "

                  #      ascii character to use as a field separator in datafile

                  #datafile_delimiter ""

                  TAG: datafile_fields all

                  #      Which data fields must be in datafile
                  #      user;date;time;url;connect;bytes;in_cache;out_cache;elapsed

                  #datafile_fields user;date;time;url;connect;bytes;in_cache;out_cache;elapsed

                  TAG: datafile_url ip|name

                  #      Saves the URL as ip or name in datafile

                  #datafile_url ip

                  TAG: weekdays

                  #      The weekdays to take into account ( Sunday->0, Saturday->6 )

                  Example:

                  #weekdays 1-3,5

                  Default:

                  #weekdays 0-6

                  TAG: hours

                  #      The hours to take into account

                  Example:

                  #hours 7-12,14,16,18-20

                  Default:

                  #hours 0-23

                  TAG: dansguardian_conf file

                  #      DansGuardian.conf file path
                  #      Generate reports from DansGuardian logs.
                  #      Use 'none' to disable it.
                  #      dansguardian_conf /usr/dansguardian/dansguardian.conf

                  dansguardian_conf

                  TAG: dansguardian_filter_out_date on|off

                  #      This option replaces dansguardian_ignore_date whose name was not appropriate with respect to its action.
                  #      Note the change of parameter value compared with the old option.
                  #      'off' use the record even if its date is outside of the range found in the input log file.
                  #      'on'  use the record only if its date is in the range found in the input log file.

                  TAG: squidguard_conf file

                  #      path to squidGuard.conf file
                  #      Generate reports from SquidGuard logs.
                  #      Use 'none' to disable.
                  #      You can use sarg -L filename to use an alternate squidGuard log.
                  #      squidguard_conf /usr/local/squidGuard/squidGuard.conf

                  TAG: redirector_log file

                  #      the location of the web proxy redirector log such as one created by squidGuard or Rejik. The option
                  #      may be repeated up to 64 times to read multiple files.
                  #      If this option is specified, it takes precedence over squidguard_conf.
                  #      The command line option -L override this option.

                  #redirector_log /usr/local/squidGuard/var/logs/urls.log

                  TAG: redirector_filter_out_date on|off

                  #      This option replaces squidguard_ignore_date and redirector_ignore_date whose names were not
                  #      appropriate with respect to their action.
                  #      Note the change of parameter value compared with the old options.
                  #      'off' use the record even if its date is outside of the range found in the input log file.
                  #      'on'  use the record only if its date is in the range found in the input log file.

                  #redirector_filter_out_date on

                  TAG: redirector_log_format

                  #      Format string for web proxy redirector logs.
                  #      This option was named squidguard_log_format before sarg 2.3.
                  #      REJIK      #year#-#mon#-#day# #hour# #list#:#tmp# #ip# #user# #tmp#/#tmp#/#url#/#end#
                  #      SQUIDGUARD  #year#-#mon#-#day# #hour# #tmp#/#list#/#tmp#/#tmp#/#url#/#tmp# #ip#/#tmp# #user# #end#
                  #redirector_log_format #year#-#mon#-#day# #hour# #tmp#/#list#/#tmp#/#tmp#/#url#/#tmp# #ip#/#tmp# #user# #end#

                  TAG: show_sarg_info yes|no

                  #      shows sarg information and site path on each report bottom

                  show_sarg_info no

                  TAG: show_sarg_logo yes|no

                  #      shows sarg logo

                  show_sarg_logo no

                  TAG: parsed_output_log directory

                  #      Saves the processed log in a sarg format after parsing the squid log file.
                  #      This is a way to dump all of the data structures out, after parsing from
                  #      the logs (presumably this data will be much smaller than the log files themselves),
                  #      and pull them back in for later processing and merging with data from previous logs.

                  #parsed_output_log none

                  TAG: parsed_output_log_compress /bin/gzip|/usr/bin/bzip2|nocompress

                  #      Command to run to compress sarg parsed output log. It may contain
                  #      options (such as -f to overwrite existing target file). The name of
                  #      the file to compresse is provided at the end of this
                  #      command line. Don't forget to quote things appropriately.

                  #parsed_output_log_compress /bin/gzip

                  TAG: displayed_values bytes|abbreviation

                  #      how the values will be displayed in reports.
                  #      eg. bytes        -  209.526
                  #          abbreviation -  210K

                  #displayed_values bytes
                  displayed_values bytes

                  Report limits

                  TAG: authfail_report_limit n

                  TAG: denied_report_limit n

                  TAG: siteusers_report_limit n

                  TAG: squidguard_report_limit n

                  TAG: user_report_limit n

                  TAG: dansguardian_report_limit n

                  TAG: download_report_limit n

                  #      report limits (lines).
                  #      '0' no limit

                  #authfail_report_limit 10
                  authfail_report_limit 0
                  #denied_report_limit 10
                  denied_report_limit 0
                  #siteusers_report_limit 0
                  #squidguard_report_limit 10
                  #dansguardian_report_limit 10
                  #user_report_limit 10
                  #user_report_limit 50
                  siteusers_report_limit 0
                  user_report_limit 0

                  TAG: www_document_root dir

                  #    Where is your Web DocumentRoot
                  #    Sarg will create sarg-php directory with some PHP modules:
                  #    - sarg-squidguard-block.php - add urls from user reports to squidGuard DB

                  #www_document_root /var/www/html
                  www_document_root /usr/local/www

                  TAG: block_it module_url

                  #    This tag allow you to pass urls from user reports to a cgi or php module,
                  #    to be blocked by some Squid acl

                  #    Eg.: block_it /sarg-php/sarg-block-it.php
                  #    sarg-block-it is a php that will append a url to a flat file.
                  #    You must change /var/www/html/sarg-php/sarg-block-it to point to your file
                  #    in  variable, and chown to a httpd owner.

                  #    sarg will pass http://module_url?url=url

                  #block_it none

                  TAG: external_css_file path

                  #    Provide the path to an external css file to link into the HTML reports instead of
                  #    the inline css written by sarg when this option is not set.

                  #    In versions prior to 2.3, this used to be an absolute file name to
                  #    a file to include verbatim in each HTML page but, as it takes a lot of
                  #    space, version 2.3 switched to a link to an external css file.
                  #    Therefore, this option must contain the HTTP server path on which a client
                  #    browser may find the css file.

                  #    Sarg use theses style classes:
                  #      .logo          logo class
                  #      .info          sarg information class, align=center
                  #      .title_c        title class, align=center
                  #      .header_c      header class, align:center
                  #      .header_l      header class, align:left
                  #      .header_r      header class, align:right
                  #      .text          text class, align:right
                  #      .data          table text class, align:right
                  #      .data2          table text class, align:left
                  #      .data3  &nb

                  1 Reply Last reply Reply Quote 0
                  • L
                    lucapsg
                    last edited by

                    Hi guys, two questions and a suggestion.
                    I configured Sarg in order to analyze the logs of DansGuardian.

                    1. How many days keeps the log rotation made ​​by Sarg?
                    2. Where do you configure this period, both for Sarg and DansGuardian?

                    As Marcelo (https://forum.pfsense.org/index.php?topic=50239.msg270375#msg270375), at this time my scheduled tasks are only two:

                    Status	Update	Aditional Args						Post		Description
                    	Frequency							Action
                    on	1h 	-d `date +%d/%m/%Y`					none		Today
                    on	1d 	-d `date +01/%m/%Y`-`date +31/%m/%Y`			none		This month
                    

                    Since these operations produce reports that do not include the data generated in the last part of the period (respectively, from 23:00 to 00:00 and in the last day of the month), it would be a useful planning like this:

                    Status	Update	Aditional Args						Post		Description
                    	Frequency							Action
                    on	1h 	-d `date +%d/%m/%Y`					none		Today
                    on	1d	-d `date -v-1d +%d/%m/%Y`				none		Consolidate yesterday
                    on	1d 	-d `date +01/%m/%Y`-`date +31/%m/%Y`			none		This month
                    	30d	-d `date -v-1m +01/%m/%Y`-`date -v-1m +31/%m/%Y`	rotate		Consolidate last month
                    

                    But this is not possible because planning more operations with the same frequency produces an error:

                    Jun 27 00:00:01 	php: sarg.php: The command 'export LC_ALL=C && /usr/pbi/sarg-amd64/bin/sarg -d `date -v-1d +%d/%m/%Y`' returned exit code '1', the output was 'SARG: Cannot create directory /tmp/sarg - File exists'
                    Jun 27 00:00:00 	php: sarg.php: Sarg: force refresh now with -d `date +01/%m/%Y`-`date +31/%m/%Y` args, compress() and none action after sarg finish.
                    Jun 27 00:00:00 	php: sarg.php: Sarg: force refresh now with -d `date -v-1d +%d/%m/%Y` args, compress() and none action after sarg finish.
                    

                    To solve this problem it would seem sufficient to create the directory /tmp/sarg only if it does not exist and every time create a subdirectory with the name of the report to be generated. In any case, any other solution that would enable the simultaneous execution of multiple operations is welcome  :D

                    Obviously, the last operation specified above can not be performed every 30 days, but the 1st day of each month: this is not possible to specify it in the GUI, although CRON could do it.  ;)

                    Thank you.

                    Cattura.JPG
                    Cattura.JPG_thumb

                    1 Reply Last reply Reply Quote 0
                    • A
                      Averenix
                      last edited by

                      Potential fix for those experiencing the:

                      Error: Could not find report index file.
                      Check and save sarg settings and try to force sarg schedule.

                      Seems there is a bug with Sarg 2.3.6 (which the pfSense package uses) which means you MUST select sites_users and users_sites in the report types, otherwise Sarg fails to run.
                      See bug here: http://sourceforge.net/p/sarg/bugs/154/

                      This has been resolved in Sarg 2.4.

                      1 Reply Last reply Reply Quote 0
                      • M
                        MarkVLK
                        last edited by

                        Hey guys, I just installed pfSense today and have been trying to get everything set up and am running into some issues with Sarg (and Lightsquid for that matter).

                        I'm trying to get Sarg and Lightsquid to generate reports from Squid proxy, but for some reason they're both having trouble.

                        Sarg gives me an error:

                        [ Sarg config error: squid log file (/var/squid/logs/access.log) does not exists]

                        and Lightsquid gives me the error:

                        Error : report folder '/var/lightsquid/report' not contain any valid data! Please run lightparser.pl (and check 'report' folder content)

                        For Lightsquid, I tried clicking both "Refresh now" and "Refresh full" but still got the same error. For Sarg I checked /var/squid/logs/ and there is only one file - cache.log

                        Is there something I haven't configured correctly with Squid, or anyone experience and solve these issues in the past? Any help would be greatly appreciated!

                        1 Reply Last reply Reply Quote 0
                        • KOMK
                          KOM
                          last edited by

                          Please start a new thread for your problem and I'll be happy to take a look.  This thread is for a particular Sarge issue.

                          1 Reply Last reply Reply Quote 0
                          • M
                            MarkVLK
                            last edited by

                            @KOM:

                            Please start a new thread for your problem and I'll be happy to take a look.  This thread is for a particular Sarge issue.

                            Sorry, didn't mean to hijack this thread! I posted a new thread here:
                            https://forum.pfsense.org/index.php?topic=79140.0

                            1 Reply Last reply Reply Quote 0
                            • P
                              PokerMunkee
                              last edited by

                              I'm a newb at this.  Forgive me.

                              Is there a pretty way of exporting logs?  If a manager asks for an Internet usage report for certain computer with a date range, how would I do this?

                              I take it I create a custom Schedule.  I want the report to show the Date/Time and website.

                              1 Reply Last reply Reply Quote 0
                              • KOMK
                                KOM
                                last edited by

                                You should probably start a new thread instead of hijacking this one that's already 27 pages long.

                                1 Reply Last reply Reply Quote 0
                                • P
                                  peruvichito2014
                                  last edited by

                                  Hi Gurus
                                  After some investigation I obtaind some type of report

                                  By command line I changed some value and obtain these parameter in the sarg.conf file:

                                  [2.1.3-RELEASE][admin@pfSense.localdomain]/usr/pbi/sarg-amd64/etc/sarg(28): cat sarg.conf | sed -e '/^#/d' -e '/^$/d'
                                  access_log /var/squid/logs/access.log
                                  graphs yes
                                  output_dir /usr/local/sarg-reports
                                  anonymous_output_files yes
                                  resolve_ip yes
                                  user_ip no
                                  topuser_sort_field BYTES normal
                                  user_sort_field BYTES normal
                                  exclude_users /usr/pbi/sarg-amd64/etc/sarg/exclude_users.conf
                                  exclude_hosts /usr/pbi/sarg-amd64/etc/sarg/exclude_hosts.conf
                                  date_format e
                                  lastlog 0
                                  remove_temp_files yes
                                  index yes
                                  index_tree file
                                  overwrite_report yes
                                  use_comma yes
                                  exclude_codes /usr/pbi/sarg-amd64/etc/sarg/exclude_codes
                                  max_elapsed 0
                                  report_type sites_users users_sites
                                  usertab none
                                  long_url yes
                                  date_time_by bytes
                                  charset Latin3
                                  privacy no
                                  bytes_in_sites_users_report no
                                  topuser_num 0
                                  dansguardian_conf
                                  show_sarg_info no
                                  show_sarg_logo no
                                  displayed_values bytes
                                  authfail_report_limit 0
                                  denied_report_limit 0
                                  siteusers_report_limit 0
                                  user_report_limit 0
                                  www_document_root /usr/local/www
                                  ntlm_user_format domainname+username
                                  realtime_refresh_time 0
                                  realtime_types GET,PUT,CONNECT
                                  realtime_unauthenticated_records show
                                  sorttable /sarg_sorttable.js
                                  hostalias /usr/pbi/sarg-amd64/etc/sarg/hostalias
                                  [2.1.3-RELEASE][admin@pfSense.localdomain]/usr/pbi/sarg-amd64/etc/sarg(29):

                                  And after run this executable command:

                                  [2.1.3-RELEASE][admin@pfSense.localdomain]/usr/pbi/sarg-amd64/etc/sarg(30): sarg -x

                                  Appear a succesfull response:

                                  ….....
                                  ...........
                                  ..............
                                  SARG: Sorting log /tmp/sarg/141.user_unsort
                                  SARG: Making index.html
                                  SARG: Successful report generated on /usr/local/sarg-reports/06Aug2014-20Aug2014
                                  SARG: Purging temporary file sarg-general
                                  SARG: End
                                  [2.1.3-RELEASE][admin@pfSense.localdomain]/usr/pbi/sarg-amd64/etc/sarg(31):

                                  See the picture below

                                  But this report appear another problem

                                  Fatal error: Allowed memory size of 262144000 bytes exhausted (tried to allocate 103903809 bytes) in /usr/local/www/sarg_frame.php on line 77

                                  This line means in this file means:

                                  [2.1.3-RELEASE][admin@pfSense.localdomain]/usr/local/www(43): sed '77q;d' sarg_frame.php
                                          print preg_replace($pattern,$replace,$report);
                                  [2.1.3-RELEASE][admin@pfSense.localdomain]/usr/local/www(44):

                                  The question is how to resolv this event???

                                  I hope your suggestion / Recomendation

                                  Regard

                                  Reportes.jpg
                                  Reportes.jpg_thumb

                                  1 Reply Last reply Reply Quote 0
                                  • M
                                    MarkVLK
                                    last edited by

                                    My Sarg doesn't seem to be generating reports anymore since I updated to pfSense 2.1.5, is this expected or do I need to change something? I already changed the conf so that it was pointing to what I think is the correct access.log (or something like that, I forget if that was the exact file name now) and that seems to have fixed an error I was seeing in my system logs, but no reports are being generated anymore, I just see this in the system logs when it is meant to create a report

                                    php: sarg.php: Sarg: force refresh now with args, compress() and none action after sarg finish.

                                    Any ideas what might be wrong?

                                    1 Reply Last reply Reply Quote 0
                                    • S
                                      sujyo1
                                      last edited by

                                      my sarg problem start after update to 2.1.5.

                                      sarg view report showing:

                                      Error: Could not find report index file.
                                      Check and save sarg settings and try to force sarg schedule.

                                      sys log showing:

                                      Sep 16 00:00:01 php: sarg.php: Sarg: force refresh now with -d date +%d/%m/%Y-date +%d/%m/%Y args, compress(on) and none action after sarg finish.
                                      Sep 16 00:00:01 php: sarg.php: Sarg: force refresh now with -d date +%d/%m/%Y-date +%d/%m/%Y args, compress(on) and none action after sarg finish.

                                      [2.1.5-RELEASE][admin@xxxxxx.localdomain]/root(1): sarg
                                      SARG: Records in file: 50, reading: 100.00%
                                      SARG: No records found
                                      SARG: End

                                      [2.1.5-RELEASE][admin@xxxxxx.localdomain]/root(5): pkg_info
                                      bsdinstaller-2.0.2014.0410 BSD Installer mega-package
                                      gettext-0.18.3.1    GNU gettext package
                                      libiconv-1.14_1    A character set conversion library

                                      [2.1.5-RELEASE][admin@xxxxx.localdomain]/root(6): sarg -x
                                      SARG: Init
                                      SARG: Loading configuration from /usr/pbi/sarg-i386/etc/sarg/sarg.conf
                                      SARG: Loading exclude host file from: /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
                                      SARG: Loading exclude file from: /usr/pbi/sarg-i386/etc/sarg/exclude_users.conf
                                      SARG: Reading host alias file "/usr/pbi/sarg-i386/etc/sarg/hostalias"
                                      SARG: List of host names to alias:
                                      SARG: Deleting temporary directory "/tmp/sarg"
                                      SARG: Parameters:
                                      SARG:          Hostname or IP address (-a) =
                                      SARG:                    Useragent log (-b) =
                                      SARG:                    Exclude file (-c) = /usr/pbi/sarg-i386/etc/sarg/exclude_hosts.conf
                                      SARG:                  Date from-until (-d) =
                                      SARG:    Email address to send reports (-e) =
                                      SARG:                      Config file (-f) = /usr/pbi/sarg-i386/etc/sarg/sarg.conf
                                      SARG:                      Date format (-g) = USA (mm/dd/yyyy)
                                      SARG:                        IP report (-i) = No
                                      SARG:            Keep temporary files (-k) = No
                                      SARG:                        Input log (-l) = /var/squid/logs/access.log
                                      SARG:              Resolve IP Address (-n) = No
                                      SARG:                      Output dir (-o) = /usr/local/sarg-reports/
                                      SARG: Use Ip Address instead of userid (-p) = No
                                      SARG:                    Accessed site (-s) =
                                      SARG:                            Time (-t) =
                                      SARG:                            User (-u) =
                                      SARG:                    Temporary dir (-w) = /tmp/sarg
                                      SARG:                  Debug messages (-x) = Yes
                                      SARG:                Process messages (-z) = No
                                      SARG:  Previous reports to keep (–lastlog) = 0
                                      SARG:
                                      SARG: sarg version: 2.3.6 Arp-21-2013
                                      SARG: Reading access log file: /var/squid/logs/access.log
                                      SARG: Records in file: 50, reading: 100.00%
                                      SARG:    Records read: 50, written: 0, excluded: 0
                                      SARG: Squid log format
                                      SARG: No records found
                                      SARG: End

                                      cat /usr/pbi/sarg-i386/etc/sarg/sarg.conf | more

                                      sarg.conf

                                      TAG:  access_log file

                                      #      Where is the access.log
                                      #      sarg -l file

                                      access_log /var/squid/logs/access.log

                                      TAG: graphs yes|no

                                      #      Use graphics where is possible.
                                      #          graph_days_bytes_bar_color blue|green|yellow|orange|brown|red

                                      graphs yes
                                      #graph_days_bytes_bar_color orange

                                      TAG:  graph_font

                                      #      The full path to the TTF font file to use to create the graphs. It is required
                                      #      if graphs is set to yes.

                                      #graph_font /usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf

                                      TAG:  title

                                      --More--(byte 529)# sarg.conf

                                      1 Reply Last reply Reply Quote 0
                                      • C
                                        ck42
                                        last edited by

                                        Looks like Sarg is not working again.
                                        Updated to 2.2 the other day.  Sarg is the latest version available in packages (2.3.9 pkg v.0.6.4) and running squid (2.7.9 pkg v.4.3.6)

                                        Attempting to run the Reports gives 'Could not find report index file.' message.
                                        Trying to run Sarg from the CLI gives me 'Cannot set the locale LC_ALL to the environment variable'

                                        Anyone have Sarg working with pf 2.2?

                                        1 Reply Last reply Reply Quote 0
                                        • KOMK
                                          KOM
                                          last edited by

                                          Doing a quick search for Sarg and picking a result a little more current would have led you to this thread here with a solution.

                                          1 Reply Last reply Reply Quote 0
                                          • C
                                            ck42
                                            last edited by

                                            Funny that you refer to that thread, KOM.  I had actually read through that prior to posting.
                                            I had already tried creating the symlink on my system with no luck.  That's why I wanted to know if anyone actually has this working with 2.2…hoping that they would confirm whether or not any special steps needed to be taken to make it work.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.