Tuning Guide for collection traffic statistics using ipcad and lightsquid

xpaco

:) Hi! Sorry, for my bad english :-X
There is insruction for collection traffic statistics in pfSense using ipcad and lightsquid (without Squid)

0. Who may benefit from this instruction?
There is a known issue with Squid MultiWANs; When you run a proxy server, Squid let all traffic by 'Default Gateway'; It may be solved, but if you have a lot of rules written in 'Firewall' - tuning can be very confusing, or maybe you just don't want to resolve this conflicts and then this guide is for you.

1. Version of pfSense
I have pfSense 2.3.2-RELEASE (amd64), but you can select a different version and capacity.
2. Installing Squid and Lightsquid It's simple: go to the "System" -> "Package Manager" -> tab "Available Packages" -> find the "squid" -> and "+ Install";
Similarly, for "Lightsquid", "Cron" and "Shellcmd".

3. RSH for export ipcad statistics
Download RSH (amd64, i386) and place it in the directory "/usr/bin/". Assign file permissions 0555.

4. ipcad installing
Maybe there are other ways, but for me it came this:
· Download "ipcad"
For amd64: http://pkg.freebsd.org/freebsd:10:x86:64/latest/All/ipcad-3.7.3_2.txz
For i386: http://pkg.freebsd.org/freebsd:10:x86:32/latest/All/ipcad-3.7.3_2.txz
and put it in "/home/" (If you are working with a newer / older FreeBSD-pfSense assembly, then you should know your version of the FreeBSD build, you can do this in the "Status" -> "DashBoard" in the "System Information" widget in the "Version" line, for example '2.4. 2-RELEASE (amd64) FreeBSD 11.1-RELEASE-p4 '; Therefore, you need to download the ipcad package from the corresponding ftp site folder http://pkg.freebsd.org/; Fortunately, everything is "on the shelves" is located and all other paths remain the same /latest/All/ipcad-3.7.3_2.txz)

· Next, run in 'Shell' the command: pkg_add /home/ipcad-3.7.3_2.txz

· And: pkg install /home/ipcad-3.7.3_2.txz

5. ipcad Configuration
Find "/usr/local/etc/ipcad.conf" file (or create if it does not exist), and replace the contents of the following:


capture-ports enable;

## buffers = 64k;

interface alc0 filter "ip and dst net 192.168.5.0/24 and not src net 10.0.0.0/8 and not 172.16.0.0/12 and not 192.168.0.0/16";

aggregate 10.0.0.0/8 strip 32; 	/* Don't aggregate internal range */
aggregate 172.16.0.0/12 strip 32;	/* Don't aggregate internal range */
aggregate 192.168.0.0/16 strip 32; /* Don't aggregate internal range */
aggregate 0.0.0.0/0 strip 32;	/* Don't aggregate external networks */

#aggregate 1024-65535	into 65535;	/* Aggregate wildly */
#aggregate 3128-3128	into 3128;	/* Protect these ports */
#aggregate 150-1023	into 1023;	/* General low range */

# netflow export destination 127.0.0.1 9996;
netflow export version 5;	# NetFlow export format version {1|5}
netflow timeout active 30;	# Timeout when flow is active, in minutes
netflow timeout inactive 15;	# Flow inactivity timeout, in seconds
netflow engine-type 73;		# v5 engine_type; 73='I' for "IPCAD"
netflow engine-id 1;		# Useful to differentiate multiple ipcads.

netflow ifclass eth mapto 0-99;		# i.e., "eth1"->1, "eth3"->3
netflow ifclass fxp mapto 0-99;		# i.e., "fxp4"->4, "fxp0"->0
netflow ifclass ppp mapto 100-199;	# i.e., "ppp32"->532, "ppp7"->507
netflow ifclass gre mapto 200-299;
netflow ifclass tun mapto 300-399;	# i.e., "tun0"->300

rsh enable at 127.0.0.1;

rsh root@127.0.0.1 admin;	/* Can shutdown ipcad */
rsh root@127.0.0.1 backup;	/* Can dump/restore/import accounting table */
rsh root@127.0.0.1;		/* Can view and modify accounting tables */
rsh 127.0.0.1 view-only;	/* Other users can view current tables */

rsh ttl = 3;

rsh timeout = 30;

dumpfile = ipcad.dump;	# The file is inside chroot(), see below...

chroot = /var/ipcad;
pidfile = ipcad.pid;

# uid = 65534;
# gid = 65534;

memory_limit = 10m;

You need to replace the data in the line interface alc0 filter "ip and dst net 192.168.5.0/24 and not src net 10.0.0.0/8 and not 172.16.0.0/12 and not 192.168.0.0/16";
Where alc0 - the name of your LAN interface (can be found in the "Status" -> "Interfaces") and 192.168.5.0/24 - your LAN network through which users have access to the Internet. -> Save, close.

6. Create the directory "/var/ipcad/" and give him permissions 0755

7. Create a file in the directory "/usr/local/sbin/tolog.sh", with permissions 0755, as follows:


#!/bin/sh
rez2=""
#REZ="q"
iptoname ()
{
rez2=$1
REZ=$(/usr/local/bin/dig @[color]192.168.5.1[/color] -x $1 +short)
# dig @ use your DNS server 
# echo "REZ - $REZ - $1"
if [ -z $REZ]
then
{
# echo
}
else
{
rez2=$REZ
}
fi
}

[color]net="192.168.5"[/color]
#net = your local net 
ttime=`/usr/bin/rsh 127.0.0.1 sh ip acco | /usr/bin/grep 'Accounting data saved' | /usr/bin/awk '{print ($4)}'`
/usr/bin/rsh 127.0.0.1 clear ip accounting
/usr/bin/rsh 127.0.0.1 show ip accounting checkpoint | /usr/bin/grep $net > /root/tolog.txt

while read p1 p2 p3 p4 p5 p6 p7 p8
do
#
if [ "$p5" != "0" ]
then
rez2=""
iptoname "$p1"
echo "$ttime.000" 1 $p2 "TCP_MISS/200" $p4 "CONNECT" $rez2:"$p5 "-" "DIRECT/"$p1" -"" >>/var/squid/logs/access.log
# echo "$vtime.000" 1 $p2 "TCP_MISS/200" $p4 "CONNECT" $rez2:"$p5 "-" "DIRECT/"$p1" -"" >>/var/squid/logs/access.log
else
fi
done < /root/tolog.txt
chown proxy:proxy /var/squid/logs/access.log

exit 0

replace "192.168.5.1" on the appropriate DNS server you; net="192.168.5" - on your LAN network;

Also check exist the file "/usr/local/bin/dig" if it is not, run the following command: pkg install bind-tools

8. In the "Services" -> "Shellcmd" create a new command in type shellcmd:
/usr/local/bin/ipcad -rds
It ipcad run at system startup. Also, do it in a shell, to see if there are no errors.

9. In the "Services" -> "Cron" create a new task to run per minute tolog.sh:
*/1 * * * * root /usr/local/sbin/tolog.sh

10. Go to the menu "Services -> Squid Proxy Server", go to the "Local Cache" tab, fill in: "Hard Disk Cache Size" - "1024" and click "Save";
Next, on the "General" tab, fill in: "Proxy Interface (s) - Select LAN and Loopback", also "Enable Access Logging - Check";
Next, the menu "Status -> Squid Proxy Reports": "Refresh Scheduler" - 10min "," Save "

11. Reboot pfSense, wait for 10 minutes (logs are processed every 10 minutes)

12. Here, by clicking on "Open Lightsquid", you can begin to see the statistics. If it does not yet have, you may need to wait a bit. Check file "/var/squid/logs/access.log" on entries marked "CONNECT", if not, then once again go through this instruction step by step (check permissions to files that i have pointed).

13. Go to the directory "/usr/local/etc/lightsquid/"
here we have two interesting file "skipuser.cfg" and "realname.cfg"; At the first file, you can specify which users IP-address does not need to be in statistics, and in the second file you can be assign names corresponding to each IP on your network.

14. Traffic statistics setup is completed :) I hope that you will succeed.

AR15USR

Would you mind posting some screenshots of what the results of all this looks like please?

xpaco

@AR15USR:

Would you mind posting some screenshots of what the results of all this looks like please?

Sure!
(please login to view the images)

1.PNG_thumb

2.PNG_thumb

3.PNG_thumb

serge2pf

I see in the last image it showing some links go thru HTTPS - is it out of ipcad or thru Squid SSL MITM?

pablot

This is great!!!

If I have more interfaces to monitor I just have to duplicate the interface line changing the actual interface and ip address?

Is this OK for adding 3 VLANS ?

interface vtnet0 filter "ip and dst net 192.168.2.0/24 and not src net 10.0.0.0/
8 and not 172.16.0.0/12 and not 192.168.0.0/16";
interface vtnet0.3 filter "ip and dst net 192.168.3.0/24 and not src net 10.0.0.
0/8 and not 172.16.0.0/12 and not 192.168.0.0/16";
interface vtnet3.4 filter "ip and dst net 192.168.4.0/24 and not src net 10.0.0.
0/8 and not 172.16.0.0/12 and not 192.168.0.0/16";
interface vtnet3.5 filter "ip and dst net 192.168.5.0/24 and not src net 10.0.0.
0/8 and not 172.16.0.0/12 and not 192.168.0.0/16";