Interface Total Period Throughput on v2.3



  • Hi,

    How does one view, via the new monitoring system, the total amount of traffic transferred (MB/GB/TB) over a period of time on a specific interface? It was obvious under the "old" RRD graphs, but I can't seem to find it on the new system.

    TIA



  • I also understand your pain…

    When I was using pfSense on a wireless hotspot before pfSense 2.3, I would constantly monitor the RRD totals from the 23rd day of each month because that's when my data allowances begin and end as I started my service on the 23rd many years ago. The overages would've "KILLED" me  >:(

    Now, Uverse DSL/ADSL2+/VDSL2 have 100/250/300GB caps in various area and plans with EXPENSIVE consequences for overages.  The cable giant Comcast also has various caps as well.

    They sometimes will warn you when it is almost too late.

    The monitoring packages that pfSense had were unstable as heck... So I REALLY liked the Total for a Period of time.

    I'm currently looking at dd-wrt & tomato as a stop-gap solution until the developers re-add this feature back into pfSense.

    Good luck..


  • Rebel Alliance Developer Netgate

    The numbers from the old graph were not all that accurate. It was handled via rrdtool's graph function using an internal algorithm it had to help make it closer to reality, but it's not. Any way we try to massage the data on the graphs it can't be all that accurate by the nature of how rrdtool averages/collapses data over time and loses resolution. We no longer have access to "rrdtool graph" because including it means including a few hundred MB more of dependencies, including some x11 libraries, which we aren't fond of doing.

    The RRD Summary package attempts to get close to what you want but also tends to be inaccurate. We're working on a more reliable way of tracking totals but we don't have a solid ETA yet.


  • Administrator

    The path of least resistance looks like putting an interface on the vnstat package. I have vnstat installed and running and will play with options. Still no ETA.

    If you want you can do a pkg update and pkg install vnstat and use it via CLI for now, but that is at your own risk.


  • Administrator

    I've made some progress on the vnstat GUI: http://i.imgur.com/A0t4G0G.gifv

    Still no ETA.



  • VERY cool to see this!!

    I suppose since you already have D3 in pfSense for the monitoring graphs, you may as well use that… I'm assuming that's what you're doing here. It looks good, and while patience can be hard to come by sometimes, I'll definitely be patient for this to be done, and done right. :)

    Thanks for the sneak peek!


  • Administrator

    yea, there is a lot of code reuse now that we have d3 in place. Here is an update: http://i.imgur.com/UDvzN0c.gifv

    Also, you can easily get png output with vnstat so bandwidth reports could be added back to the email package when vnstat is added as a dependancy.

    Someone has reached out to help with the back end and is getting up to speed. Hopefully we'll have a package in the repo soon that people can test out.



  • @jdillard:

    The path of least resistance looks like putting an interface on the vnstat package. I have vnstat installed and running and will play with options. Still no ETA.

    If you want you can do a pkg update and pkg install vnstat and use it via CLI for now, but that is at your own risk.

    Hi jdillard,

    In the past I've installed vnstat, BandwidthD and Darkstat, following the recommendations at: https://doc.pfsense.org/index.php/How_can_I_monitor_bandwidth_usage

    They caused pfSense to become unstable, so I had to do a fresh install and stayed away from those packages on my Alix2D3, as they were conflicting with other parts of pfSense and often corrupting the data that was supposed to be graphed. In addition they didn't quite match the bandwidth data from my dd-wrt or Tomato routers.

    Will you be making it a built-in feature, that can be enabled/disabled without "hurting" the system & seamlessly integrated into pfSense?

    Thanks…


  • Administrator

    did you have them installed all at once? Apparently BandwidthD was a problem package, that's why it isn't back in 2.3.

    I would't be surprised if dd-wrt and Tomato uses vnStat to drive those metrics based on the information presented, but someone would have to confirm that. This will be an optional package that basically installs vnStat and drops a couple GUI pages in. I don't expect vnStat to cause any stability issues, but if you have evidence of the contrary let me know.

    edit: The package is also making progress. I have built a test package locally (my first, so I had to learn) and I am working on the vnStat integration bits. the front-end is largely done.



  • Hi jdillard,

    I was playing around with the "Status / Monitoring / Settings / Interactive Graph" and the way it is setup just doesn't seem to be very "intuitive" to use.  I realize it is still mostly incomplete… :)

    When I was browsing DSLReports forums, some who were using vnStat said it tends to under report usage.

    I believe Tomato uses Ajax & SVG for its bandwidth monitoring, and offloading most of the work to the browsing computer's browser.  I guess that is why it is fast and snappy even on a lowly 200Mhz 4MB flash/16MB ram WRT54GL router.  It seems to use very little resources when drawing the graphs, compared to say, vnstat.

    Is it possible to use the graphing / bandwidth monitoring technology in Tomato for the Monitoring portion of pfSense?  If you do, please take a look at AdvancedTomato and TomatoShibby. The graphs are arranged slightly differently, but looks great and very self explanatory to use.

    Thanks...


  • Administrator

    Ajax & SVG is the front end, vnStat is the back end (just like RRD is the back-end of Status > Monitoring). As I said, Tomato could very well be using vnStat to drive the front-end. If someone has hard data on vnStats inaccuracy I'll take a look at it, but the RRD totals from before are even more inaccurate at they average out the data.

    The only thing happening with this gif, http://i.imgur.com/A0t4G0G.gifv, involving vnStat is a 'vnstat –json' call, which is a straight database dump into a JSON object, you can't get anymore efficient on the server-side than that.



  • Are you able to re-use and/or transplant some of the Tomato front-end code for use by pfSense Monitoring?

    The graphs and UI is very slick and easy to use.  The best that I've seen…


  • Administrator

    No, I think they use a different framework. That gif was from basically the first time there was anything worth taking a screencast of, it has come a long way since.



  • The graphs looks nice…

    I wonder if you'll have a way to specify the "First Day Of The Month" for calculations to begin?  That would be a VERY important feature for people who needs to monitor their monthly caps.

    And backup/restore of the data.