$100 bounty - List sites visited by ip
-
Apologies for the huge delay, Scott.
I looked at DarkStat some time back, but it shows very very little info, and the little that's there is not intuitive, IMO, (those few graphs are just a few vertical and horizontal lines ;) ), and what I've seen is nowhere close to the "spec" mentioned earlier in this thread.
Best regards,
- Mike -
I believe that lightsquid is what you are looking for. This is already working without modification from the repositories. There is no gui configuration for it however, so you have to do all the conf files by hand at current. This would be easy money for someone with more time on their hands than myself. pkg_add -r lightsquid should do the trick. Fix the confs, add the cron job and have fun. This will keep track of sites visited by ip, and associate them with a calendar date. You can also see other stats like most visited site for the month etc. Just a suggestion!
-
Thank you, JustinW.
LightSquid looks VERY appropriate indeed.
A few further comments:
- "My" pfSense is now "live", so I cannot easily mess with it. I'd need to build a similar box for experiments.- In other long threads here, the installation of LightSquid was discussed "a lot", with much of it over my head. Perhaps, sometimes, installation can be a bit complex ? And there was mention of having an integrated LightSquid installation option available in the next release fo pfSense - which would be real nice.
- In other threads here also, I note that other apps are required, and I don't know if some/all of these are in the standard pfSense build (Squid, etc).
- At my site, DHCP is running, so the IP addresses assigned to users are transient. Again, I've seen references to LightSquid scripts which require users to log-in/log-out, and which will then update a database to relate Usernames to IP addresses (based on time). This might add another layer of complexity!
- Dang - "nothing's easy"!
Thank you again,
- Mike -
I've just installed lightsquid on my pfsense and wow, it's wonderfully easy and useful! Now I know where the good pr0n are all at!
-
Look this
http://forum.pfsense.org/index.php/topic,4314.msg30643.html#msg30643I make two themes for LightSquid Anybody test this :D
I use novosea theme
All themes tested on IE6 and FFox. -
SARG is the solution, IF you are forcing users to use squid via ACL's
Transparent proxying may be a bit more hairy, but still do-able (the advantage with the ACL solution is that you can get squid to authenticate agains a winderz AD server, so you get reports on actual users and not just IP's).If you wan a SARG solution, say so (and maybe an e-mail to remind me), and I'll take a looky.
Regards
The Sproggg -
Thank you. SARG looks superb!
From the info I read on it, I could not verify that it can cope in a DHCP environment, but I'm hoping it can.
At the time that any traffic occurs through pfSense (with DHCP), pfSense knows the IP and a "Name" associated with that IP - I don't recall if it's a "Computer-Name" (which would be very good!), or a "User-Name" (which would be only GREAT!!). So, hopefully someone (SQUID, or SARG, or…) can associate these names at the time the traffic occurs, and log them, without any separate logins, etc...
I would be most interested in pursuing this option for pfSense, though, clearly, I can help only at a basic technical level, as well as pay for all assistance. I notice a few other threads here discussing the installation of SARG, so, obviously, there's interest in such tools.
Thank you again,
- Mike -
Okeydokey.
I've downloaded the developer iso, and will start looking this evening.
In interest of full disclosure :) .. there IS a freebsd 6.2 package available, so all I have to figure out is how to build a pfsense addon package, and make the menu structure pfsense style pretty.Milestone 1:
Incorporate the package with an installer (This will probably give you the option to see SARG via an unlinked, custum URL)Milestone 2:
Integrate package with pfsense…1 Should be done in a few days (read, about a week).
2 ... I dunno, the package templates don't look TOO nasty, I just don't want to break anything else :)
If anyone has already developed packages, or is great at PhP, I'm more than happy to share the bounty if this will speed up a solution
SARG has no problems with a DHCP environment, just take a look at the squid access.log file (tail -f it to see it in motion). SARG uses the data in this file to compile reports.
AGAIN, if we want to do transparent squid, we need to get X-Forwarded for switched on, else SARG thinks all traffic comes from the pfsense box itsef!
Also, I will take a look at the NMB authentication (or RADIUS, or any other PAM) available in psense squid for USER as opposed to IP logging
This may need a tweak, and I'm loathe to mess with someone elses code.regards
The Sproggg -
Thank you again,
In case you end up re-inventing "The Wheel", run a search here on SARG, and you'll see that some others put in some serious work on pfSense install procedures for SARG, early this year. In one thread, Scott indicated that the forthcoming pfSense 1.2 might be needed, because of some very recent tweaks to the CRON features.
- Mike
-
@MJK:
In one thread, Scott indicated that the forthcoming pfSense 1.2 might be needed, because of some very recent tweaks to the CRON features.
- Mike
If you need - can use my code for define cron task pfSense 1.2
// setup cron tasks // original source from '/etc/inc/pfsense-utils.inc' function 'tdr_install_cron' // this function safe for other tasks // ***************************************************************************** // - $task_name: cron task name (for config identification) /for searching my cron tasks/ // - $options: array=[0:minute][1:hour][2:mday][3:month][4:wday][5:who][6:cmd] // - $task_key: cron command key for searching // - $on_off: true-'on task', false-'off' task // required: $task_nameand $on_off // ***************************************************************************** define('FIELD_TASKNAME', 'task_name'); function ls_setup_cron($task_name, $options, $task_key, $on_off) { global $config; update_log("ls_setup_cron: start task_name=$task_name, task_key=$task_key, on_off=$on_off"); // check input params if(!$task_name) { update_log("ls_setup_cron: exit - uncomplete input params."); return; } // search cron config settings if(!$config['cron']['item']) { update_log("ls_setup_cron: exit - 'config.xml'->[cron]->[items] not found."); return; } // searching task $x_name=''; $x=0; foreach($config['cron']['item'] as $item) { if($item[FIELD_TASKNAME] and $task_name and ($item[FIELD_TASKNAME]==$task_name)) { update_log("ls_setup_cron: found cron task with name=$task_name on [$x_name]."); $x_name = $x; } $x++; } unset($x); // install cron: // - if not found with such name and not found 'task_key', when install task // - if found task with such name, when renew this item (delete and add new with all check's) // deinstall cron: // - deinstall only, if found such name switch($on_off) { case true: if($task_key) { // searching task $x=0; $x_task=''; foreach($config['cron']['item'] as $item) { if(strstr($item['command'], $task_key)) { $x_task = $x; update_log("ls_setup_cron: found cron task with key=$task_key on [$x]."); } $x++; } unset($x); if($x_task and (!$x_name or ($x_task != $x_name))) { // other task with $task_key alredy installed update_log("ls_setup_cron: can't add cron task, while such task exists $task_key"); break; } else { if(is_array($options)) { // delete this item (by name) if($x_name > 0) unset($config['cron']['item'][$x_name]); // and add new $cron_item = array(); $cron_item[FIELD_TASKNAME] = $task_name; $cron_item['minute'] = $options[0]; $cron_item['hour'] = $options[1]; $cron_item['mday'] = $options[2]; $cron_item['month'] = $options[3]; $cron_item['wday'] = $options[4]; $cron_item['who'] = $options[5]; $cron_item['command'] = $options[6]; // check options if(!$cron_item['who']) $cron_item['who'] = "nobody"; $config['cron']['item'][] = $cron_item; write_config("Installed cron task '$task_name' for 'lightsquid' package"); configure_cron(); // log update_log("ls_setup_cron: add cron task '$task_name'='" . $cron_item['command'] . "'"); } } } else // log update_log("ls_setup_cron: input prm 'task_key' not defined"); break; case false: // delete cron task only with name $task_name if($x_name > 0) { unset($config['cron']['item'][$x_name]); write_config(); // log update_log("ls_setup_cron: delete cron task '$task_name'"); } break; } configure_cron(); update_log("ls_setup_cron: end"); }