Want package for squidanalyzer. INSTRUCTIONS AND FILES TO INSTALL

KOM

OK, I was doing this on a test server so I rolled it back to default to try again. I got everything in place, but when I installed Perl I got this error:

[1/1] Installing perl5.16-5.16.3_18…
[1/1] Extracting perl5.16-5.16.3_18: 100%
makewhatis: not found
makewhatis: not found
pkg: POST-INSTALL script failed

This seems to be a known problem with FreeBSD 10 with some people saying that Perl is a mess in early 10 release builds, but has been subsequently fixed.

Now perl -v doesn't do anything. I don't know how I had it working before.

Update: It still works via /usr/local/bin/perl. I still get the fatal error though. The command:

cd /temp/squidanalyzer/ && perl squid-analyzer -r -c /etc/squidanalyzer/squidanalyzer.conf /var/squid/logs/access.log

can't possibly work since there is no /etc/squidanalyzer/squidanalyzer.conf file. The folder /etc/squidanalyzer only gets created if you run the install.sh installer, and at no point does this tutorial mention that.

sacaman

Dear KOM

i get the same problem with other that i can't get the updated report chowing on the man page

and i post my method to get it working on my system

@sacaman:

hi ,

first thank you for this amazing reporting program

i get a problem that the report is not updated the script is well executed but the report is not showing the daily update

i try it and make it working i a different way

i download your temp file only no need for the www or perl

inside the temp file i execute the file

sh install_all.sh

it will create all the files and path needed

install -m 755 squid-analyzer /usr/local/bin/
install -m 644 resources/sorttable.js /var/www/squidanalyzer/
install -m 644 resources/squidanalyzer.css /var/www/squidanalyzer/
install -m 644 resources/flotr2.js /var/www/squidanalyzer/
install -m 644 resources/images/logo-squidanalyzer.png /var/www/squidanalyzer/images/
install -m 644 resources/images/cursor.png /var/www/squidanalyzer/images/
install -m 644 resources/images/domain.png /var/www/squidanalyzer/images/
install -m 644 resources/images/back-arrow.png /var/www/squidanalyzer/images/
install -m 644 resources/images/info.png /var/www/squidanalyzer/images/
install -m 644 resources/images/network.png /var/www/squidanalyzer/images/
install -m 644 resources/images/user.png /var/www/squidanalyzer/images/
install -m 644 lang/* /etc/squidanalyzer/lang/
pod2text doc/SquidAnalyzer.pod README
pod2man doc/SquidAnalyzer.pod squid-analyzer.3

i make a symlink /var/www/squidanalyzer/ to /usr/local/www/squidanalyzer/

ln /var/www/squidanalyzer/ /usr/local/www/squidanalyzer/

i modify the file /etc/squidanalyzer/squidanalyzer.conf

to meet the real path on pfsense

change this area

Output /usr/local/www/squidanalyzer
WebUrl /squidanalyzer
LogFile /var/squid/logs/access.log

on the cron i need now only

cd /temp/squidanalyzer/ && perl squid-analyzer

and it create a daily report at 9:01 morning

i hope that it help someone to get it working

sacaman

please try it as bellow and let me know if it work with you
also you don't need to install perl if you follow my how to

Br

sacaman

KOM

perl: Command not found.

which is to be expected since Perl isn't installed. You must have installed it at some point in the past.

KOM

After manually installing perl and clearing up stupid typos that I insist on making, I got it working on my 2.2-x64 test box. I'm trying to make it work on a 2.1.5-i386 box and it does everything except write the report into the output folder. NO errors, but no output. Hmmm.

Does anyone know if this script will handle archived logs, like access.log.0, access.log.1 etc?

sacaman

try only this command in terminal

cd /temp/squidanalyzer/ && perl squid-analyzer

KOM

The script runs, but it doesn't create any files and there is no error log or anything. Here is the output:

[2.1.5-RELEASE][admin@pfsense.MyDomain.local]/etc/squidanalyzer(137): perl /usr/local/bin/squidanalyzer -d -r -c /etc/squidanalyzer/squidanalyzer.conf /var/squid/logs/access.log
No new log registered…
DEBUG: the log statistics gathering took: 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)
Building HTML output into /usr/local/www/squidanalyzer
DEBUG: generating HTML output took: 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)
DEBUG: total execution time: 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)

KOM

OK, I managed to get it working by running just this command:

perl /usr/local/bin/squidanalyzer /var/squid/logs/access.log

Just that by itself generated the reports.

Topper727

Sorry been busy and not around to help

KOM

No problem. I've long since given up on it and moved on.

acc4ever

Guys, thank you very much for this How-To … I was able to install SquidAnalyzer v6.1, It works really good ...

Topper727, can you compile/port the new version ?

Thanx in advance

Topper727

Yes, I will try make one soon for you. sorry I been away for bit..

Topper727

I looked up the new version.. seems nothing new yet or I missed it. mine is still working quite well. love to see how many hits in percents and all that per day and all that stuff..shows me squid proxy is working and if I make a change I can see maybe a increase or decrease in stuff proxy caches.. Nice to have

Topper727

Wish someone with package skills could turn this into package for all of us. Nice reporting tool

killmasta93

awsome stuff :)

But cant seem to install perl it says that its not in the repository, I also tried looking for perl it did show up on some files.

any idea?

Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
All repositories are up-to-date.
pkg: No packages available to install matching 'perl5.16-5.16.3_18' have been fo

then tried installing perl

pkg install perl5

then ran your command

cd /temp/squidanalyzer/ && perl squid-analyzer -r -c /etc/squidanalyzer/squidanalyzer.conf /var/squid/logs/access.log
perl: Command not found.

Thank you

Clipboarder.2016.01.02-002.png

Clipboarder.2016.01.02-005.png

Topper727

I will get my notes tonight and see what can help. Just wanted drop a line sayjng i relead this and will help later today

killmasta93

Thanks mate really good stuff :)

Topper727

When I ran pkg install perl5 I got this

FreeBSD repository update completed. 24761 packages processed.
New version of pkg detected; it needs to be installed first.
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
pkg: 1.5.6 -> 1.6.2

The process will require 148 KiB more space.
2 MiB to be downloaded.

Proceed with this action? [y/N]:

I wonder if there is something with your repository lists.

Topper727

what happens if you type

perl -v

I get

[2.2.6-RELEASE][admin@firewall.wholesale-florida.com]/temp/squidanalyzer: perl -v

This is perl 5, version 20, subversion 2 (v5.20.2) built for amd64-freebsd-thread-multi

Copyright 1987-2015, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

maybe type

rehash

Then your perl -v again if it not works

killmasta93

Hi Topper,
Thank you for the reply,
I seems that perl is installed and working i guess but when i run your command

cd /temp/squidanalyzer/ && perl squid-analyzer -r -c /etc/squidanalyzer/squidanalyzer.conf /var/squid/logs/access.log

I get Fatal no configuration file

But i do have the squid analyzer webui

See pictures

Thank you

Clipboarder.2016.01.18-002.png

Clipboarder.2016.01.18-003.png

Clipboarder.2016.01.18-004.png

Clipboarder.2016.01.18-005.png

Topper727

Ah .. make sure you have this file

/etc/squidanalyzer/squidanalyzer.conf

It should have this inside

This file is the default configuration file for SquidAnalyzer

Edit it to match your needs and copy it under /etc/squidanalyzer/squidanalyzer.conf

Path where SquidAnalyzer should dump all HTML and images files.

Choose a path that can be read by a Web browser

Output /usr/local/www/squidanalyzer

The URL of the SquidAnalyzer javascript, HTML and images files.

WebUrl /squidanalyzer

Set the path to the Squid log file

LogFile /var/squid/logs/access.log

If you want to use DNS name instead of client Ip address as username enable

this directive. When you don't have authentication, the username is set to

the client ip address, this allow you to use the DNS name instead.

Note that you must have a working DNS resolution and that it can really slow

down the generation of reports.

UseClientDNSName 1

If you have enabled UseClientDNSName and have lot of ip addresses that do

not resolve you may want to increase the DNS lookup timeout. By default

SquidAnalyzer will stop to lookup a DNS name after 0.0001 second (100 ms).

DNSLookupTimeout 0.001

Set the file containing network alias name. Network are

show as Ip addresses so if you want to display name instead

create a file with this format :

LOCATION_NAME IP_NETWORK_ADDRESS

Separator must be a tabulation

NetworkAlias /etc/squidanalyzer/network-aliases

Set the file containing user alias name. If you don't have auth_proxy

enable user are seen as Ip addresses, or if you want to replace login

name by full user name, create a file with this format :

FULL_USERNAME IP_ADDRESS || LOGIN_NAME

Separator must be a tabulation

UserAlias /etc/squidanalyzer/user-aliases

How do we sort Network, User and Url report screen

Value can be: bytes, hits or duration. Default is bytes.

OrderNetwork bytes
OrderUser bytes
OrderUrl bytes

How do we sort Mime types report screen

Value can be: bytes or hits. Default is bytes.

OrderMime bytes

Should we display user URL details. This will show all URL read

by user. Take care to have enougth space disk for large user.

UrlReport 1

Should we display user details. This will show statistics

per user.

UserReport 1

Run in quiet mode or print debug information

QuietMode 1

Cost of the bandwith per Mb. If you want to generate invoice per Mb

for bandwith traffic this can help you. Value 0 mean no cost.

CostPrice 0

Currency of the bandwith cost

Currency €

Top number of url to show

TopNumber 100

Path to the file containing client ip addresses, network ip address,

and/or auth login to exclude from report

Exclude /etc/squidanalyzer/excluded

Path to the file containing client ip addresses, network ip address,

and/or auth login to include into the report. Other entries will be

excluded by default.

Include /etc/squidanalyzer/included

Translation Lang /etc/squidanalyzer/lang/en_US.txt,

en_US.txt, ru_RU.txt, uk_UA.txt, cs_CZ.txt, pl_PL.txt and de_DE.txt).

Default to:

Lang /etc/squidanalyzer/lang/en_US.txt

Date format used to display date (year = %y, month = %m and day = %d)

You can also use %M to replace month by its 3 letters abbreviation.

DateFormat %y-%m-%d

Set this to 1 if you want to anonymize all user login. The username

will be replaced by an unique id that change at each squid-analyzer

run. Default disable.

AnonymizeLogin 0

Adds peer cache hit (CD_SIBLING_HIT) to be taken has local cache hit.

Enabled by default, you must disabled it if you don't want to report

peer cache hit onto your stats.

SiblingHit 1

Set the default unit for transfert size. Default is BYTES, other possible

values are KB, MB and GB

TransfertUnit BYTES

Minimum percentage of data in pie's graphs to not be placed in the others item.

MinPie 2

Set this to your locale to display generated date in your language. Default

is to use strftime. If you want date in German for example, set it to de_DE.

For french, fr_FR should do the work.

Locale en_US

By default SquidAnalyzer is saving current collected statistics each time

a new hour is found in log file. Most of the time this is enough but if

you have huge log file and don't have enough memory this will slow down the

parser by forcing Perl to use temporaries files. Use lower value following

your memory and the size of your log file, on very huge log file with lot of

requests/seconde a value of 30 minutes (1800) or less should help.

WriteDelay 3600

Use this directive to show the top N users that look at an URL or a domain.

Set it to 0 to disable this feature.

TopUrlUser 10

This directive allow you to replace the SquidAnalyze logo by your custom

logo. The default value is defined as follow:

[#

#]($self->{WebUrl}) SquidAnalyzer

Feel free to define your own header but take care to not break current design.

#CustomHeader My ISP Company

This directive allow exclusion of some unwanted methods in report statistics

like HEAD, POST, CONNECT, etc. Can be a comma separated list of methods.

#ExcludedMethods HEAD

This directive allow exclusion of some unwanted mimetypes in report statistics

like text/html, text/plain, or more generally text/*, etc. Can be a comma separated

list of perl regular expression.

#ExcludedMimes text/.,image/.

This directive allow exclusion of some unwanted codes in report statistics

like TCP_DENIED/403 which are generated when a user accesses a page the first

time without authentication. Can be a comma separated list of methods.

#ExcludedCodes TCP_DENIED/403

https://www.dropbox.com/s/vtf7loy2x9lnhl3/squidanalyzer.conf?dl=0

download this file, run winscp and put in the folder on your firewall of
/etc/squidanalyzer/

This should of been created by unzipping the files from my original zip. Not sure why it failed there. If file exists maybe permission?

squidanalyzer.png_thumb