New Traffic Totals package available for testing
-
It uses the vnstat database (just takes the JSON output to make the graphs).
go forth and break things. (or just play around with it and have fun)
Known issues:
#1) There is an issue with the stacked bar graph. Documented here: https://github.com/novus/nvd3/issues/341#issuecomment-222370991
#2) If you have vnstat already running it will most likely override your options and also install the new database files in a different location than where yours currently are. This scenario is likely to produce the most edge cases.
#3) The start day defaults to 1 and is stored in the vnstat config file at the time of saving, but I don't have a way to get it back out of the config currently, so it shows up as blank.[fixed]#4) Which ties into number four, there isn't a way to save the graph options at this time.[fixed]#5) The cron job that grabs the data runs every 5 minutes and the interval isn't documented at this time. [fix pending]
#6a) units should be in KiB/MiB/etc and the Y axis label needs to be changed to reflect totals not speed.[fixed]#6b) y axis tickmarks need custom formatting.[fixed]#7) lists PHP as a dependency when it doesn't need to.[fixed]#8) Top 10 Days only works when one interface is selected. I need to throw an error if multiple interfaces are selected.[fixed]#9a) NanoBSD image is untested at this time, but should work. Data loss after a reboot needs to also be tested (actually, I may be able to tie the data into backups).
#9b) vnStat databases aren't being backed up.
#10a) Some interfaces are showing up blank.[fixed]#10b) It looks like vnStat doesn't support virtual and aliased interfaces, so not all interfaces will show up.[not entirely true]#11) match graph legend to interface names[fixed]#12) the sortable table doesn't work after you update the graph or change the time period.
#13) looks like crap on mobile
#14) package needs to remove cron/config on uninstall
-
Is there any way it can save the last graph settings for each of the 4 output options?
Thanks
Myk -
Ah no, I forgot about that. I haven't figured out a good way to save config options for this package just yet. I'll take another look at that as well.
-
also
On the daily stats, can we have to option to reverse the order, newest on top - so you dont have to scroll to the bottom of the list. Thats the way the old vnstat2 did it.
Thanks again for this package!
Myk
-
Ah no, I forgot about that. I haven't figured out a good way to save config options for this package just yet. I'll take another look at that as well.
one of my docker containers, rutorrent front end, saves selected columns, width, columns sortings etc via a json file that is save - maybe that is a simple option
or maybe something on the user/browser side?
Thanks
Myk -
On the daily stats, can we have to option to reverse the order, newest on top - so you dont have to scroll to the bottom of the list.
I'll see about reversing the table order for all of them. Also, possibly making it a sortable table.
-
Installed 2.3.2 just for this… was going to try and wait until release, but now you did this. :)
So far in favor of all mentioned items... save default graph settings, sortable table, and most recent period at the top.
My comment is that it seems to be showing an average speed for the period rather than the actual amount of data. Maybe make that an option as well? Some people might care about the average speed for the period... but most are more interested in the actual amount of data transferred instead.
Additionally, it would be good to see vnstatd added to the Services widget on the dashboard, so we can make sure it's running and properly tracking traffic.
-
@virgiliomi:
My comment is that it seems to be showing an average speed for the period rather than the actual amount of data. Maybe make that an option as well? Some people might care about the average speed for the period… but most are more interested in the actual amount of data transferred instead.
What leads you to believe it is average? It should be totals.
@virgiliomi:
Additionally, it would be good to see vnstatd added to the Services widget on the dashboard, so we can make sure it's running and properly tracking traffic.
It uses a cron job not vnstatd, I thought it would be more reliable. You can see the cron job if you install the cron package. I also haven't added the functionality that lets you change how often the cron job runs yet.
-
@virgiliomi:
My comment is that it seems to be showing an average speed for the period rather than the actual amount of data. Maybe make that an option as well? Some people might care about the average speed for the period… but most are more interested in the actual amount of data transferred instead.
What leads you to believe it is average? It should be totals.
I've been watching a 5Mbps+ video stream for the past 45 minutes… yet for this hour (25 minutes so far), it's not showing anywhere near the amount of traffic showing it should be showing... Unless the numbers are really megabytes? The unit on the graph says "bits per second"... that's clearly not right.
-
I believe I have the units represented wrong, it looks like it should be KiB/MiB/etc. Thanks for catching that!
-
ya, mine shows today aprox 10mb, old graph shows right at 9.xg
Myk
-
Ok, that's what screwed me up then. :)
As for cron versus service… It's not a big deal to me either way... I just wasn't sure how it was running. I did quickly pick up that it was updating every 5 minutes though.
-
Also on the side of the graphs - Traffic (Bits per Second) - does not work - its total traffic - no speed
Myk
-
Yea, I think that was an artifact of using the Status > Monitoring as a starting point. I'll try and get some of these fixes in tomorrow.
-
Just wondering about why PHP is listed as a dependency for the package. Isn't PHP part of the whole pfSense ecosystem anyway? Or is that for future planning if it goes away? Does this potentially cause problems if PHP is upgraded by pfSense core, but the package isn't updated to reflect the new version of PHP, when someone goes to install the package?
-
It uses a cron job not vnstatd, I thought it would be more reliable.
Why is the vnstatd daemon unreliable on FreeBSD? Had it running for years on a Linux box with no problems at all.
-
It uses a cron job not vnstatd, I thought it would be more reliable.
Why is the vnstatd daemon unreliable on FreeBSD? Had it running for years on a Linux box with no problems at all.
He didn't say that it was unreliable on FreeBSD… just that he thought it would be more reliable as a cron job. Using the cron job, it's only running every 5 minutes rather than a constantly running process that could be stopped at any time leaving holes in the data.
I see pluses and minuses to both ways... I personally have no preference. I just want it to work and accurately track my usage. It doesn't need to be minute-by-minute... updating every 5 minutes is fine by me. Most of what I care about (and probably most people) is monthly usage more than anything.
-
Hmmm…
Is this topic about the new graphing system on 2.3 which replaced RRD? Or is it an addition to that? Or is it just a similar graphing to what's already built-in?
Why as a separate package? -
@virgiliomi:
He didn't say that it was unreliable on FreeBSD… just that he thought it would be more reliable as a cron job. Using the cron job, it's only running every 5 minutes rather than a constantly running process that could be stopped at any time leaving holes in the data.
Cron jobs are run by a daemon that could also be stopped at any time ;) So the question remains, why not use the vnstat daemon? Don't get me wrong, I just want to understand it, don't want to offend anyone :)
-
@virgiliomi:
Just wondering about why PHP is listed as a dependency for the package. Isn't PHP part of the whole pfSense ecosystem anyway? Or is that for future planning if it goes away? Does this potentially cause problems if PHP is upgraded by pfSense core, but the package isn't updated to reflect the new version of PHP, when someone goes to install the package?
I had a line in the Makefile that didn't need to be there. I'll be taking it out and it should remove the PHP dependancy.
-
It uses a cron job not vnstatd, I thought it would be more reliable.
Why is the vnstatd daemon unreliable on FreeBSD? Had it running for years on a Linux box with no problems at all.
It's not unreliable on FreeBSD, just harder to tie into all the necessary GUI bits last time I looked. Maybe I was looking at it wrong.
-
Hmmm…
Is this topic about the new graphing system on 2.3 which replaced RRD? Or is it an addition to that? Or is it just a similar graphing to what's already built-in?
Why as a separate package?TL;DR: RRDTool graphing dependancies got too large so we cut them out. Part of that was the function that provided the period totals. Period totals using RRD data aren't that accurate to begin with since they get averaged out over time. vnStat handles the collection of historical traffic totals better and is more accurate, hence the addition. Also, not everyone cares about traffic totals so no need to bundle it in core and having it as a package allows it to be revved separate from core.
-
Period totals using RRD data aren't that accurate to begin with since they get averaged out over time. vnStat handles the collection of historical traffic totals better and is more accurate, hence the addition. Also, not everyone cares about traffic totals so no need to bundle it in core and having it as a package allows it to be revved separate from core.
In that case, replacing Period totals with vnStat in the future would make many people happy.
-
under interfaces 2 of my 4 interfaces dont show but if i click the blank area datas there
-
I don't see anything in the interface selection.
I can click through the blank space and it does follow
I'm using VLANS on a single LAGG of two physical interfaces.
The totals do show up for my physical and ovpn interfaces, but none of those are assigned labels. -
Ok I'll have to look into the code that generates the interface names, I have a feeling it is making the option element, but not populating it with content. Grandrivers, do you happen to know what makes those 2 interfaces that aren't showing up different from the others (based on type, etc)?
-
one is a pppoe WAN the other is a non assigned opt interface but had not noticed the neither of my HE ipv6 tunnels show up at all might be nice to have those as well
-
I wonder if there's limits to what vnstat can track usage for… because interfaces that seem to be showing up are physical interfaces, while non-physical ones - VLANs, tunnels, etc. - don't seem to be showing up.
Also, the interfaces selected show their pfSense names (WAN, LAN, GUEST), but in the graph and table they're identified as the actual physical interface (igb0, igb1, igb2). I also have a blank entry, which is my unassigned igb3 interface. This is part of what leads me to the above suggestion that vnstat may only be able to track physical interfaces.
-
Just bumped the version to 0.2. It should include misc fixes as well as fixes for:
#6) The correct units and axis label. Although, I have to figure out how to make a custom axis format as it uses binary prefixes and not the SI prefixes.
#7) PHP shouldn't be listed as a dependency anymore. This may take some more kicking to get to work.
#8) Don't allow TOP 10 when multiple interfaces are selected.
-
@virgiliomi:
I wonder if there's limits to what vnstat can track usage for… because interfaces that seem to be showing up are physical interfaces, while non-physical ones - VLANs, tunnels, etc. - don't seem to be showing up.
Also, the interfaces selected show their pfSense names (WAN, LAN, GUEST), but in the graph and table they're identified as the actual physical interface (igb0, igb1, igb2). I also have a blank entry, which is my unassigned igb3 interface. This is part of what leads me to the above suggestion that vnstat may only be able to track physical interfaces.
It could also have to do with the get_interface_list() function I am using. I might need to use a different function or a combination of them, I am in the process of testing get_configured_interface_with_descr().
Edit: Bad news bears, this is what the vnStat man page says:
Virtual and aliased interfaces cannot be monitored because the kernel doesn't provide traffic information for that type of interfaces. Such interfaces are usually named eth0:0, eth0:1, eth0:2 etc. where eth0 is the actual interface being aliased.
-
For those that have blank interface issues can you view the page source (or inspect the element if you know how) and grab the HTML for the select element with id="interfaces".
It should look something like:
<select class="form-control" id="interfaces" name="interfaces" multiple=""><option value="em0" selected="">wan</option></select> -
As requested:
<select class="form-control" id="interfaces" name="interfaces" multiple=""><option value="igb3" selected="">lan</option></select>
-
<select class="form-control" id="interfaces" name="interfaces" multiple=""><option value="igb1" selected="">opt1</option><option value="igb2" selected="">lan</option></select>
-
<select class="form-control" id="interfaces" name="interfaces" multiple=""><option value="igb0" selected="">wan</option><option value="igb1" selected="">opt3</option></select>
-
Thanks! I was finally able to replicate the blank interface issue, looking into it now.
-
Will the vnstat database be included in the XML backup similar to RRD?
-
In theory yes, but I haven't messed with that code yet.
-
I just pushed version 0.4 and it should fix:
#6) Units are more accurate and consistent. it was mostly fixed before, but there were a couple small errors.
#10a) Interfaces shouldn't show up blank anymore, but it is recommended to reset the graphing data after updating the package.
#10b) not really a fix, but it doesn't look as bad as previously thought. I can pull in VLANs, IPSec, and PPPOE. More interface types will need to be tested. If you want to see which interfaces vnStat can capture data for you can run the command 'vnstat –iflist'.
-
still have a blank line - it used to show as my LAN interface
Myk
-
I don't think that the update was released last night yet… I had updated my pfSense build maybe 15-20 minutes before your post time and the package manager still showed the .3 version of this package.
