How can I view the Squid access.log in realtime from the GUI

phil.davis · Dec 28, 2011, 8:26 AM

I have an embedded system, so LightSquid is not available. I would like to be able to view the activity that goes into /var/squid/logs/access/log from the pFsense GUI in some way (or some similar data from somewhere). Preferably that it shows recent entries that scroll/update in almost realtime. I don't need to have the data written anywhere for posterity. The purpose is to be able to quickly see who is accessing what URL's, so as to be able to pinpoint those computers that are causing issues. Other tools can show the bandwidth usage by IP address in realtime, but they don't give easy clues about the site names or URLs that are being accessed.

At the moment I use PuTTY to access the console and "tail -f /var/squid/logs/access.log" - so any ideas for a way to do something similar from the GUI are welcome.

Thanks

dvserg · Dec 28, 2011, 8:53 AM

Type you command from Diagnostics > Command Prompt

marcelloc · Dec 28, 2011, 12:21 PM

At the moment I use PuTTY to access the console and "tail -f /var/squid/logs/access.log" - so any ideas for a way to do something similar from the GUI are welcome.

Without sarg or lightsquid, that's the best way to do this.

But if you need or prefer using gui, I would change tail -f to tail -200 and repeat command after reading logs.

dvserg · Dec 28, 2011, 1:51 PM

Current connections also can be viewed with SQStat

phil.davis · Dec 29, 2011, 4:54 AM

For the benefit of future readers;

Using "tail -f" from the Diagnostics->Command Prompt GUI page causes things to stop responding.
I suppose that the "Execute Command Shell" section expects the command given to exit nicely, returning some output, which it would happily display. By using "-f" the command runs "forever" producing output along the way but does not exit. So the result is not shown on the GUI output.

This works:

tail -n 100 /var/squid/logs/access.log

Change the "100" to however many lines you would like to view.
Repeat the command as needed.

phil.davis · Dec 29, 2011, 5:13 AM

For the benefit of future readers in English…
I found various posts in Russian about how to install SqStat. The commands worked fine, but it was a little more difficult for me to read the text of the instructions. :) So here is what I did:

Logon as admin at the console by whatever method you like (I used PuTTY to the IP address of the pFsense router)
Select option 8, Shell.
Execute these commands:

pkg_add -r http://diskatel.narod.ru/files/sqstat-1.20_2.tbz
Fetching http://diskatel.narod.ru/files/sqstat-1.20_2.tbz... Done.
pkg_add: cannot open /var/db/pkg/sqstat-1.20_2/+DISPLAY as display file

ln -s /usr/local/share/sqstat /usr/local/www/sqstat
cd /usr/local/share/sqstat/
cp config.inc.php.defaults config.inc.php
chmod 0744 /usr/local/share/sqstat

(You can ignore the "pkg_add: cannot open..." message above)
(I didn't need to change anything in the config file)

From a browser, go to http://router-name-or-ip/sqstat/sqstat.php
Enter an Auto Refresh interval and click "Update".

From what I could make out, version 1.20_2 is still current.

It doesn't ask for any password, so this is making the data available to anyone on your network that knows how to look.

dvserg · Dec 29, 2011, 5:26 AM

It doesn't ask for any password, so this is making the data available to anyone on your network that knows how to look.

SQStat is password protected by pfSense GUI. Try logoff from pfsense and check new _http://router-name-or-ip/sqstat/sqstat.php .

phil.davis · Dec 29, 2011, 11:01 AM

I logged off from the pFsense router, then also cleared everything in my Firefox caches/history, then went to another computer that had never accessed the pFsense router before. In all 3 cases I was able to operate the SqStat php page without providing any authentication.
Maybe the "chmod 0744 /usr/local/share/sqstat" has something to do with it?
Maybe I need to setup a cachemgr_passwd in squid.conf and config.inc.php? But I think that only controls how SqStat is able to get data from Squid, not the interaction of SqStat to the browser front end.
In my installation, SqStat is displaying its data to anyone without authentication.

dvserg · Dec 29, 2011, 11:14 AM

I still believe that the authentication is present. My system is 2.0, browser Chrome.

Cino · Dec 29, 2011, 3:06 PM

@dvserg The original SqStat package you created does by-pass pfsense security while your latest one does prompt me to you sign in. Hope this helps.

Latest: from post http://forum.pfsense.org/index.php/topic,38820.0.html (http://diskatel.narod.ru/pfsense/sqstat_pf.rar)
Original: from post http://forum.pfsense.org/index.php/topic,24362.0.html (http://diskatel.narod.ru/files/sqstat-1.20_2.tbz)

dvserg · Dec 29, 2011, 4:56 PM

@Cino:

@dvserg The original SqStat package you created does by-pass pfsense security while your latest one does prompt me to you sign in. Hope this helps.

Latest: from post http://forum.pfsense.org/index.php/topic,38820.0.html (http://diskatel.narod.ru/pfsense/sqstat_pf.rar)
Original: from post http://forum.pfsense.org/index.php/topic,24362.0.html (http://diskatel.narod.ru/files/sqstat-1.20_2.tbz)

:o Thanks, I saw an error. I'll try to fix it. :o

Cino · Dec 29, 2011, 5:15 PM

anytime.. I like the new interface since it integrates into pfsense but i find myself using the old one since it resolves IPs to hostnames on the fly…

marcelloc · Dec 29, 2011, 5:21 PM

@dvserg:

:o Thanks, I saw an error. I'll try to fix it. :o

maybe is just missing one include on your php

@require_once("guiconfig.inc");

phil.davis · Dec 29, 2011, 5:37 PM

I just copied the files from http://diskatel.narod.ru/pfsense/sqstat_pf.rar onto my system in /usr/local/share/sqstat
Yes, now it makes me login first.
But I get this error displayed on the SqStat GUI page:

Error (1): Cannot get data. Server answered: HTTP/1.0 404 not found

This is mentioned in the forum post at http://forum.pfsense.org/index.php/topic,38820.msg210182.html#msg210182
But the post is in Russian - I can read English, Nepali and Hindi, but not Russian! - so I can't understand the solution.

Also, if you are making changes/fixes then it would be really nice to have the latest version available in a standard package like the older file http://diskatel.narod.ru/files/sqstat-1.20_2.tbz - that way anyone can easily install it in a more automated way.

phil.davis · Dec 29, 2011, 6:08 PM

I remembered to try Google Translate. I tried a couple of things that I saw in the translation; setting the permission of the folder/s to 0755 and adding entries to the Squid Access Control for External Cache-Managers. But those changes made no difference. I couldn't see a direct solution to the "404 not found" error in any of the Google translations of the Russian posts.

Anyway, bedtime for me in Nepal!

@dvserg - If you are able to make a new version with it all working with pFsense 2.0.n that would be great.

Cino · Dec 29, 2011, 6:17 PM

I use Google Translate all the time for non-english boards. Works great.

did you link it to /usr/local/www/sqstat? Or what I did, I just copied the sqstat dir to /usr/local/www

phil.davis · Dec 30, 2011, 4:22 AM

I had sqstat "ln"ed from /usr/local/share into /usr/local/www
Now I have deleted the link, checked that the sqstat GUI can't even find itself (thus confirming that it was really looking in /usr/local/www/sqstat), copied the whole directory /user/local/share/sqstat to /usr/local/www/sqstat and made sure that the permissions are set to 0755.
The SqStat GUI does its thing, but the symptoms are the same as before - the GUI is displayed and the time updates (so it is trying to refresh) but the bottom box says;
"Error (1): Cannot get data. Server answered: HTTP/1.0 404 not found"
just like before.