Bandwidth Usage/Statistics Question
-
Hi,
I am probably not the expert for that but I know that SQUID proxy can log and Lightsquid or Sarg can analyze the squid log for http and https access. And you can see how much traffic was transferred by URL/IP.
When squid is running in transparent mode it just filters http access but no need to configure anything on the clients. Perhaps you can start with that a see if the log analyzer do what you want. If it is ok and you need https then you need to configure squid in non-transparent mode and you need to configure the hosts to use squid as your proxy.
PS: Try with squid2 first - this is probably more stable than squid3 at the moment.
-
Setting up squid as a transparent proxy will monitor both incoming and outgoing http traffic?
-
As a hosting provider, you should be using Netflow. I like nfsen as a free option for a netflow collector which has all the reporting capabilities you mentioned.
-
Thank you Netflow seems like a good option. What is the recommended way of exporting the netflow info from pfsense? I've seen pfflowd and softflowd but I am unsure of the differences
-
So i tried pfflowd with Manageengines Netflow Analyzer. This did not work as it displayed thousands of ifindex#### interfaces, and showed multiple terabytes of traffic in less then 10 minutes.
I uninstalled pfflowd and installed softflowd which seems to be working. Started it with this command:
./softflowd -i em1 -v 9 -n 192.168.2.10:9996em1 is my LAN interface, em0 is the wan interface
however in ManageEngines Analyzer it only shows inbound traffic
I tried starting a second copy with ./softflowd -i em0 -v 9 -n 192.168.2.10:9996
but that seemed to just add the wan traffic to the incoming traffic section in manage engine. it shows all traffic ifindex-1Any ideas?
-
I gave up on managed engines and went with PRTG network monitor and softflowd .
I remember having issues with PFflowd. -
I've installed PRTG and it seems to be working, but It also seems to only show total bandwidth for the interface.
I added pfsense and setup a custom netflow v9 sensor.
I still can't figure out how to break down traffic by URL -
In prtg you have to add a sensor for each IP
so one device ( the PF box
and sensors for this box
Ie in the Include filter you put IP[192.168.25.41] The Ip you want.Or let it auto create sensors
-
Pfsense is 192.168.2.1
Ok, so i have my webservers, internal ip of 192.168.2.40 and .46
both webservers serve several domains on each IP address, and I need to find out how much traffic each site is using, is what you are suggesting going to give me that kind of detail? -
Yes
You can let it auto create sensors ant it should probe and find the servers and make snmp sensors . I think the newest version lets you add credentials so it can probe deeper and bring back more info such as cpu load disk status as well as traffic.Or you can manually add a sensor too a device (pf box) Need to add the device first. and add a filter so so each sensor watches for only 1 IP
A simple filter is IP[192.168.2.40]
set flow time out too 6 or 10 minutes.But try the auto create wizard first. If you put in the user/pw for the servers you might get all the info you need and more from that.