Pfsense and ISP billing (data monitoring)
-
See new version in second post, first version/post for reference only
I wasnt sure where to post it, but since others might see some benifit in it, I though I would post up. Hopefully my ramblings prove usefull to someone!
First a little history. My ISP monitors how much data I transfer each billing period, so I have to watch myself. Before pfsense I was internet sharing through a windows PC, and use Bitmeter2 to montior my usage. I have been using pfsense for a little over a year, as marvelous as it is, I couldnt find a function to duplicate what Bitmeter2 was doing on my pc. When I set it up, I tried a variety of plugins, none doing exactly what I wanted. So I modified an old yahoo widget I wrote years ago to use SNMP to track my usage. That had a downfalls though, as the pc hosting the widget had to be always on and I had to run yahoo widgets for only the one widget. Alot of overhead for something so simple. Regardless, I used that function for most of the time until now. As a result my wife and I having a child, the rooms in the house being rearranged, and the computer with the widget will no longer have the uptime it use to and will be missing data that the other computers use, so I was again on the hunt for a solution that was on the router.
I went though the plugins again, but nothing exactly fit my bill, then I wrote some scripts to use the RRD backend to generate what I needed, but the implementation was awkward. So I ended up just writing my own php script to do what I needed. It is extremely simplistic, and defenatly not the most elequant code, but I haven't done php in years, and I'm a windows boy, and clueless when it comes to FreeBSD.
Currently I am running 1.2.3RELEASE installed on a hard drive.
see posts below for the install files.
-
for what its worth, a bit of an overhaul, but ill leave the old one up so you can remember how to uninstall it or continue to use it if you like as it does work just fine. if you install the new version, and want to keep your stats, just rename your old /tmp/wandata.data to /tmp/wanstats.data i had to many weird filename going on through the rewrite and clean them up, and that one one that was renamed.
this one is a bit easier to install too, just make sure you complete remove the old version (above) prior to installing the new version though (including all the cron entries, rc entries, and php files).
-download all the attached .txt files
-rename all the files to remove the .txt extention
-upload all the files to pfsense through the webgui (diagnotics/command). this will put all the files in the /tmp directory
-through the webgui (again through the diagnostics/command menu) run the following commands:
cp /tmp/wanstats_install.php /usr/local/www/wanstats_install.php
-now point your browser to http://pfsense.local/wanstats_install.php
-all done!should you want to uninstall point your browser to http://pfsense.local/wanstats_uninstall.php
a couple screen shots of the new version included
let me know if there are any bugs/errors/nuances/etc as i doubt the code is perfect.
again, it is for the standard install of 1.2.3RELEASE, it 'should' run on 1.2.3 release nanobsd as well, but is untested on the nano platform.
-
That is a very nice package! I wish someone would integrate this in the packagelist…..So many could use this for monitoring the amount of traffic.
-
This looks great, would a per ip breakdown be possible sometime in the future? Maybe you'd have to pull data from bandwidthd rather than RRD to achieve that?
-
That is a very nice package! I wish someone would integrate this in the packagelist…..So many could use this for monitoring the amount of traffic.
thanks! if anyone wanted to put it in, i would help where i could, but setting up a test server to build the package was a bit more than i wanted to get into
This looks great, would a per ip breakdown be possible sometime in the future? Maybe you'd have to pull data from bandwidthd rather than RRD to achieve that?
nope, that would be a completely different script/package, alot more involved that what I have written. the current pacakge is independate of RRD as well, using netstat to pull the byte info from the wan interface. in a previous iteration i had used RRD data, but it wasnt very clean and i was never happy with it.
-
minor bug update, mostly for allowing for what happens if for whatever reason your pfsense box crashes/freezes.
otherwise exactly the same as described in my second post
wanstats_maint.php.txt
wanstats.php.txt
wanstats_uninstall.php.txt
wanstats.inc.txt
wanstats.xml.txt
wanstats_install.php.txt
wanstats_rec.php.txt -
This is SUPER FANTASTIC. Quarter of a million dollar script! Definitely should come with pfSense by DEFAULT!!!
Thank you for your great work!!! -
Somehow I missed this thread before, but caught it when it was linked from another thread.
It shouldn't be too hard to turn this into a proper package. I thought about making one like this myself, even. I'll see if I can get it in the repo.
EDIT: A couple suggestions: The name should be more general, and you should be able to choose the interface. Otherwise it's not much use to those of us with multi-wan or those who want to use it on internal interfaces. Even something like "Bandwidth Totals" would be good.
-
After looking deeper at the files you have, it does look like it would be quite a bit of work to make that into a package.
Much of the things you have done could be automated or handled for you by using the package system's functionality, though.
You might want to download the package repository and look at how some other packages work.
-
Try the new RRD Summary package I just put up, it should produce similar output using only the data which exists in the RRDs.
-
After looking deeper at the files you have, it does look like it would be quite a bit of work to make that into a package.
Much of the things you have done could be automated or handled for you by using the package system's functionality, though.
You might want to download the package repository and look at how some other packages work.
easy to change the interface, but your right, a fair amount of coding woud be involved to monitor multiple interfaces.
i looked at the package repository, but i didnt want to have to set up a server to get it all going, maybe ill give it another try, although with your simple rrd script, not sure if thats necessary anymore!
i did run into problems in the past with rrd though, where it didnt match what my ip was saying i was doing, out by about 10%, but using scripting using ifstats matched fine, which was one reason i went the way i did
-
This is exactly the type of package I'm after. The RRD Summary package only shows you usage from the current month - eg, my usage period starts on the 15th of each month, so if I wanted to view July 15th - August 15th I can't do it :) ISP Billing looks exactly perfect.
Any chance we can see this functionality appear in pfSense anytime soon, either as the ISP Billing package or get RRD Summary fleshed out a bit more? :)
-
RRD Summary lets you pick the start date as well as shows you both the current and prior months. You should be able to do exactly what you want with it.
It's already in 2.0.
-
Is there anything new with this package? Saw it this days and would like it as a package!
-
Zorac, I just installed your package, and it works very well! Just a suggestion: maybe some graphing and the possibility to setup a periodic (monthly, weekly, etc) cap and send a notification (email, network message, etc) if the cap is almost reached/has been reached?
What do you think?
Also, I was setting up the package before use, and I noticed this:
Data Backup: Data is saved to the /tmp directory, which means in an improper shutdown, data can be lost, this option will use cron to back up the data to the phyiscal media on a daily basis so no more than 1 day of data can ever be lost due to improper shutdown.
What does physical media means? Can we change the location where it stores the backup? I'd rather have this on the pfsense box harddrive… Also, can we change the backup frequency? Instead of daily, maybe ervery 4 hours or so? My bandwidth can change very rapidly. Within a few hours, it can go up by several GB's...
Thanks!
-
The application failed. After several weeks working properly it stopped recording several days ago and I've lost track of my bandwidth for good.
Any idea why suddenly it stopped working?
