Bandwidthd graphs no longer displaying after updating to the latest version
-
Hi All
I recently update bandwidthd to the latest available version. After the update I was no longer able to view the graphs.
Findings:
1. The settings are configured to enable showing graphs. (I have checked in the bandwidthd.conf file, /usr/local/bandwidthd/etc/)
2. All the .png graph files located in (/usr/local/bandwidthd/htdocs/) have a zero file size.
3. Running the "/usr/local/bandwidthd/bandwidthd" process application does not yield any different results or produce any errors.
4. Uninstalling and Re-installing bandwidthd does not resolve the issue.
5. All other graphs (RRD, ntop, lightsquid) are working/displaying correctly.???
Any suggestions would be very welcome.
My System:
2.0.2-RELEASE (i386)
built on Fri Dec 7 16:30:38 EST 2012
FreeBSD 8.1-RELEASE-p13Bandwidthd version 2.0.1.4
Thanks
-
I am experiencing the same issues.
2.0.2-RELEASE (i386)
built on Fri Dec 7 16:30:38 EST 2012
FreeBSD 8.1-RELEASE-p13Bandwidthd version 2.0.1.4
-
same here.
im using:
2.0.2-RELEASE (i386)
built on Fri Dec 7 16:30:38 EST 2012
FreeBSD 8.1-RELEASE-p13bandwidthd 2.0.1.4
-
From a non-nanoBSD system, can someone post a copy of /usr/local/etc/rc.d/bandwidthd.sh
and the output ofls -l /usr/local/bandwidthd/bandwidthd
The code now makes a custom bandwidthd.sh for nanoBSD that puts the data files into /var/bandwidthd (memory disk)
But for systems with a real disk:
bandwidthd.sh should be the same as before;
/usr/local/bandwidthd/bandwidthd should be a sym link to where the program is (different for pfSense 2.0.n and 2.1 due to pbi on 2.1); and
the data files on the real disk.
Obviously I broke something for the non-nanoBSD case, but I can't see what and don't have access to a non-nanoBSD system right now. -
Hi Phil
Thanks for the fast response.
Here are the outputs you requested:cat /usr/local/etc/rc.d/bandwidthd.sh
Output
#!/bin/sh # This file was automatically generated # by the pfSense service handler. rc_start() { /usr/local/bandwidthd/bandwidthd } rc_stop() { /usr/bin/killall bandwidthd } case $1 in start) rc_start ;; stop) rc_stop ;; restart) rc_stop rc_start ;; esac
ls -l /usr/local/bandwidthd/bandwidthd
Output
-rwxr-xr-x 1 root wheel 59776 Jun 11 2012 /usr/local/bandwidthd/bandwidthd
Thanks
-
Your bandwidthd.sh and /usr/local/bandwidthd/bandwidthd program look fine.
I installed a VM running:
2.0.2-RELEASE (i386)
built on Fri Dec 7 16:30:14 EST 2012
FreeBSD 8.1-RELEASE-p13Installed bandwidthd package (2.0.1.4) and put another VM behind it to generate some traffic.
It runs fine for me:[2.0.2-RELEASE][admin@pfSense.localdomain]/usr/local/bandwidthd(79): ls -l total 64 -rwxr-xr-x 1 root wheel 59776 Jun 11 2012 bandwidthd drwxr-xr-x 2 root wheel 512 Mar 6 06:31 etc drwxr-xr-x 2 root wheel 512 Mar 6 11:17 htdocs [2.0.2-RELEASE][admin@pfSense.localdomain]/usr/local/bandwidthd(80): ls -l htdocs/ total 30 -rw-r--r-- 1 root wheel 2175 Mar 6 11:20 192.168.42.208-1-R.png -rw-r--r-- 1 root wheel 2164 Mar 6 11:20 192.168.42.208-1-S.png -rw-r--r-- 1 root wheel 2264 Mar 6 11:20 Subnet-1-192.168.42.0.html -rw-r--r-- 1 root wheel 2175 Mar 6 11:20 Total-1-R.png -rw-r--r-- 1 root wheel 2164 Mar 6 11:20 Total-1-S.png -rw-r--r-- 1 root wheel 2949 Mar 6 11:20 index.html -rw-r--r-- 1 root wheel 653 Mar 6 11:14 index2.html -rw-r--r-- 1 root wheel 653 Mar 6 11:14 index3.html -rw-r--r-- 1 root wheel 653 Mar 6 11:14 index4.html
When I click the "Access Bandwidthd" tab, it opens in a new browser tab. I need to refresh that browser tab once bandwidthd has started making graphs, then it displays them.
I tried the default settings (LAN and Draw Graphs on, other settings blank/off), and with output_cdf and recover_cdf selected. Both worked.
Did you update the whole package, or just the XML?
Can you give any clues about what is interesting/different about your setup?
Then I can try and replicate it. -
I'm not sure it's helpful or not, but I don't think it can hurt
I just switched to pfSense over the weekend. Running the 2.0.2 vmware image I noticed the graphs were not displaying sometimes. It took me a few minutes, but I did realize that for me, it seemed to be if I accessed it with internet explorer 9. Chrome and Opera seem to work fine for me.
-
Hi Phil
There is nothing special about my configurations. I am currently running two firewalls both having the same issue.
One is a simple single WAN, single LAN with packages:
squid3, lightsquid, bandwidthd, ntop, and backupThe second is also very basic but has a single LAN and multiple WANs. With traffic shaping and routing over the multiple WANs.
This configuration has the same packages installed.I have done an uninstall and re-install of bandwidthd on both servers.
It appears though that the graph .png images are not being generated when I run /usr/local/bandwidthd/bandwidthd.
So using different browsers does not resolve the issue./usr/local/bandwidthd(13): ls -lst htdocs/ | less total 234 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.1-1-R.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.1-1-S.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.1-2-R.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.1-2-S.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.1-3-R.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.1-3-S.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.10-1-R.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.10-1-S.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.10-2-R.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.10-2-S.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.10-3-R.png 0 -rw-r--r-- 1 root wheel 0 Mar 7 08:43 192.168.99.10-3-S.png
Bandwidth Config:
/usr/local/bandwidthd(15): cat etc/bandwidthd.conf # # This file was automatically generated by the pfSense # package management system. Changing this file # will lead to it being overwritten again when # the package manager resyncs. # #################################################### # Bandwidthd.conf # # Commented out options are here to provide # documentation and represent defaults # Subnets to collect statistics on. Traffic that # matches none of these subnets will be ignored. # Syntax is either IP Subnet Mask or CIDR subnet 192.168.99.0/24 # Device to listen on # Bandwidthd listens on the first device it detects # by default. Run "bandwidthd -l" for a list of # devices. dev "re0" ################################################### # Options that don't usually get changed # An interval is 2.5 minutes, this is how many # intervals to skip before doing a graphing run # Graph cutoff is how many k must be transfered by an # ip before we bother to graph it #Put interface in promiscuous mode to score to traffic #that may not be routing through the host machine. promiscuous false #Log data to cdf file htdocs/log.cdf output_cdf true #Read back the cdf file on startup recover_cdf true #Libpcap format filter string used to control what bandwidthd sees #Please always include "ip" in the string to avoid strange problems #Draw Graphs - This default to true to graph the traffic bandwidthd is recording #Usually set this to false if you only want cdf output or #you are using the database output option. Bandwidthd will use very little #ram and cpu if this is set to false. graph true #Set META REFRESH seconds (default 150, use 0 to disable).
I wish I could provide you with more information but I am not sure where else to look that may be causing the issue. There are not error messages appearing in my log files either.
-
I did another uninstall and re-install of bandwidthd. After the uninstall I noticed the file "bandwidthd.core" remains in the /usr/local/bandwidthd directory along with the cdf files.
I moved the bandwidthd.core file to another location and ran the install. After the install there is no new bandwidthd.core file created in the /usr/local/bandwidthd folder.
Is this file no longer required or has it been moved to another location?After uninstall "/usr/local/bandwidthd"
/usr/local/bandwidthd(97): ls -lst total 189190 2 drwxr-xr-x 2 root wheel 512 Mar 7 09:01 etc 63424 -rw-r--r-- 1 root wheel 64893216 Mar 7 09:00 log.1.0.cdf 58080 -rw-r--r-- 1 root wheel 59413778 Mar 7 08:58 log.2.0.cdf 10960 -rw-r--r-- 1 root wheel 11191933 Mar 7 08:40 log.3.0.cdf 1376 -rw-r--r-- 1 root wheel 1386281 Mar 7 08:39 log.4.0.cdf 0 -rw-r--r-- 1 root wheel 0 Jan 13 19:44 log.1.1.cdf.bak 45760 -rw-r--r-- 1 root wheel 46807500 Jan 13 19:44 log.1.2.cdf 2 -rw-r--r-- 1 root wheel 660 Nov 19 07:51 log.1.3.cdf 112 -rw-r--r-- 1 root wheel 113848 Nov 19 07:51 log.1.4.cdf 2 -rw-r--r-- 1 root wheel 432 Nov 18 22:56 log.1.5.cdf 9472 -rw------- 1 root wheel 9678848 Oct 18 16:16 bandwidthd.core
After Re-install having moved the bandwidthd.core file before the install.
/usr/local/bandwidthd(103): ls -lst total 179790 12 drwxr-xr-x 2 root wheel 11264 Mar 7 09:03 htdocs 2 drwxr-xr-x 2 root wheel 512 Mar 7 09:02 etc 63424 -rw-r--r-- 1 root wheel 64893216 Mar 7 09:00 log.1.0.cdf 58080 -rw-r--r-- 1 root wheel 59413778 Mar 7 08:58 log.2.0.cdf 10960 -rw-r--r-- 1 root wheel 11191933 Mar 7 08:40 log.3.0.cdf 1376 -rw-r--r-- 1 root wheel 1386281 Mar 7 08:39 log.4.0.cdf 0 -rw-r--r-- 1 root wheel 0 Jan 13 19:44 log.1.1.cdf.bak 45760 -rw-r--r-- 1 root wheel 46807500 Jan 13 19:44 log.1.2.cdf 2 -rw-r--r-- 1 root wheel 660 Nov 19 07:51 log.1.3.cdf 112 -rw-r--r-- 1 root wheel 113848 Nov 19 07:51 log.1.4.cdf 2 -rw-r--r-- 1 root wheel 432 Nov 18 22:56 log.1.5.cdf 60 -rwxr-xr-x 1 root wheel 59776 Jun 11 2012 bandwidthd
-
It might turn out to be some file/dir protection issue? Or disk that has no space left? (and can add new filenames to an exist dir, but the files cannot get bigger than zero bytes) or something in bandwidthd ends up not being root and thus has not enough priv to write? or ?
Do you have anything interesting about your disk partitions?
As soon as I start bandwidthd, it produces 4 index html files wherever htdocs is (In /usr/local/bandwidthd (i386) or /var/bandwidth (nanobsd):-rw-r--r-- 1 root wheel 14735 Mar 7 12:46 index.html -rw-r--r-- 1 root wheel 13966 Mar 7 12:42 index2.html -rw-r--r-- 1 root wheel 653 Mar 7 11:52 index3.html -rw-r--r-- 1 root wheel 653 Mar 7 11:52 index4.html
I only get the png files after data has been collected for the first period (2.5 minutes) (or if recover_cdf is on, then the first action of bandwidthd is to regenerate all the index*.html and *.png).
Thinking… -
My hdd is only at 11% capacity and there are no special permissions or anything like that required.
All the HTML files (index and subnet) are being generated and the usage is definitely being recorded as the table at the top is displaying the usage per IP correctly.
It only appears to be the graphs that have broken.
Are there an specific libraries required to generate the .png graph files? -
Having the same Problem here with 1.3 everything was fine 1.4 just not working. try´d reinstall. manually delete…
MFG Alex -
Hi Phil
I have noticed that on my system I have two png libraries installed.
png-1.4.8 Library for manipulating PNG images png-1.5.14 Library for manipulating PNG images
The 1.4.8 library is installed by Bandwidthd and the 1.5.14 is installed by Lightsquid. Could it be possible that the method required by bandwidthd to produce the .png files is no longer available in the 1.5 version and is therefore failing to produce the files?
Would you be able to test this by installing Lightsquid on the test instance you setup? And if this is the case how would one go about resolving the issue?
Sorry for all the questions.
Thanks
-
I also have lightsquid installed, but i already testet it with all packages uninstalled except for bandwithd.
This was making no difference on my system. -
I reproduced the symptoms. Installed 2.0.2 on a VM, then bandwidthd, enable bandwidthd defaults plus output_cdf and recover_cdf, reboot, all is good.
Installed squid, enable transparent proxy and all other defaults, reboot, all is good.
Installed lightsquid, did not much with it (pressed the "Refresh Now" button), rebooted.
The packages all start fine. bandwidthd produces numeric output, but does not make the graphs.
Here is a dir listing:[2.0.2-RELEASE][admin@front.localdomain]/usr/local/bandwidthd(14): ls -l total 82 -rwxr-xr-x 1 root wheel 59776 Jun 11 2012 bandwidthd drwxr-xr-x 2 root wheel 512 Mar 8 15:27 etc drwxr-xr-x 2 root wheel 512 Mar 8 16:05 htdocs -rw-r--r-- 1 root wheel 493 Mar 8 17:40 log.1.0.cdf -rw-r--r-- 1 root wheel 264 Mar 8 17:37 log.1.1.cdf -rw-r--r-- 1 root wheel 1671 Mar 8 17:37 log.1.2.cdf -rw-r--r-- 1 root wheel 254 Mar 8 17:22 log.1.3.cdf -rw-r--r-- 1 root wheel 2566 Mar 8 17:22 log.1.4.cdf -rw-r--r-- 1 root wheel 248 Mar 8 16:57 log.1.5.cdf -rw-r--r-- 1 root wheel 2654 Mar 8 17:28 log.2.0.cdf [2.0.2-RELEASE][admin@front.localdomain]/usr/local/bandwidthd(15): ls -l htdocs/ total 30 -rw-r--r-- 1 root wheel 0 Mar 8 17:40 10.99.1.42-1-R.png -rw-r--r-- 1 root wheel 0 Mar 8 17:40 10.99.1.42-1-S.png -rw-r--r-- 1 root wheel 0 Mar 8 17:33 10.99.1.42-2-R.png -rw-r--r-- 1 root wheel 0 Mar 8 17:33 10.99.1.42-2-S.png -rw-r--r-- 1 root wheel 2222 Mar 8 17:40 Subnet-1-10.99.1.0.html -rw-r--r-- 1 root wheel 2224 Mar 8 17:33 Subnet-2-10.99.1.0.html -rw-r--r-- 1 root wheel 0 Mar 8 17:40 Total-1-R.png -rw-r--r-- 1 root wheel 0 Mar 8 17:40 Total-1-S.png -rw-r--r-- 1 root wheel 0 Mar 8 17:33 Total-2-R.png -rw-r--r-- 1 root wheel 0 Mar 8 17:33 Total-2-S.png -rw-r--r-- 1 root wheel 2914 Mar 8 17:40 index.html -rw-r--r-- 1 root wheel 2916 Mar 8 17:33 index2.html -rw-r--r-- 1 root wheel 653 Mar 8 17:33 index3.html -rw-r--r-- 1 root wheel 653 Mar 8 17:33 index4.html -rw-r--r-- 1 root wheel 1649 Jun 11 2012 legend.gif -rw-r--r-- 1 root wheel 7573 Jun 11 2012 logo.gif
The "png" files are all zero bytes!
And the bandwidthd report looks like the screenshot below.
While lightsquid was installing, I noticed that it had different versions of the same tbz files that bandwidthd had. e.g.
bandwidthd - lightsquid
png-1.4.8.tbz - png-1.5.14.tbzI am guessing that one of these messes up the report generation?
And I guess uninstalling lightsquid does not remove whatever has gone wrong. The lightsquid versions were all later, so maybe they get left in place and then even a reinstall of bandwidthd does not overwrite them with the old version??? I am guessing here!I only changed the bandwidthd package xml and inc file. I didn't actually build any new binaries for this. I incremented the version number from 2.0.1.3 to 2.0.1.4 to flag a change to the package xml/inc and because actually the tbz file was already bandwidthd-2.0.1_4.tbz 2011-Oct-15 00:56:07 - so that tbz has been around for a long while also.
It's well past bedtime in my time zone! I will sleep on it - any further ideas are most welcome.
-
The following differences in dependent packages seemed to be breaking the graphing:
gd-2.0.35_7,1 A graphics library for fast creation of images gd-2.0.35_8,1 A graphics library for fast creation of images png-1.4.8 Library for manipulating PNG images png-1.5.14 Library for manipulating PNG images
The earlier versions get installed with bandwidthd, then LightSquid installs the later ones - presumably overwriting various so files.
I found ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8-stable/All/bandwidthd-2.0.1_5.tbz
That has later dependencies.
I did the following to use 2.0.1_5 bandwidthd:
a) Remove LightSquid package, using the GUI
b) Stop bandwidthd from Services->Status
c) At a command prompt:pkg_delete bandwidthd-2.0.1_4 pkg_delete gd-2.0.35_7,1 pkg_delete png-1.4.8 pkg_add -r ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8-stable/All/bandwidthd-2.0.1_5.tbz
The addition emits a bunch of warnings about various other dependency issues:
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8-stable/All/bandwidthd-2.0.1_5.tbz... Done. Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8-stable/All/png-1.5.12.tbz... Done. Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-8-stable/All/gd-2.0.35_8,1.tbz... Done. pkg_add: warning: package 'gd-2.0.35_8,1' requires 'jpeg-8_3', but 'jpeg-8_4' is installed pkg_add: warning: package 'gd-2.0.35_8,1' requires 'pkgconf-0.8.8', but 'pkgconf-0.8.9' is installed pkg_add: warning: package 'gd-2.0.35_8,1' requires 'freetype2-2.4.9_1', but 'freetype2-2.4.11' is installed pkg_add: warning: package 'gd-2.0.35_8,1' requires 'freetype2-2.4.9_1', but 'freetype2-2.4.7' is installed pkg_add: warning: package 'bandwidthd-2.0.1_5' requires 'jpeg-8_3', but 'jpeg-8_4' is installed pkg_add: warning: package 'bandwidthd-2.0.1_5' requires 'pkgconf-0.8.8', but 'pkgconf-0.8.9' is installed pkg_add: warning: package 'bandwidthd-2.0.1_5' requires 'freetype2-2.4.9_1', but 'freetype2-2.4.11' is installed pkg_add: warning: package 'bandwidthd-2.0.1_5' requires 'freetype2-2.4.9_1', but 'freetype2-2.4.7' is installed pkg_add: warning: package 'bandwidthd-2.0.1_5' requires 'libiconv-1.14', but 'libiconv-1.13.1_1' is installed
d) Install LightSquid from the pfSense GUI
2.0.1_5 pbi is used by 2.1-BETA1 - maybe the tbz will work with 2.0.2.
Can others try this?
If successful then we can update the pfSense package version to 2.0.1_5 -
Hi Phil
Thank you for your solution. I have updated my Bandwidthd package following your instructions and it is now working and displaying graphs correctly again.
:DI did encounter an issue with lightsquid once I had re-installed it. I experienced a "Server 500 - Internal Error", this is easily fixed however using my previous resolution for the issue:
http://forum.pfsense.org/index.php/topic,57505.0.htmlI have also check the RRD graphs and ntop Graphs and they all display correctly.
I would say you may go ahead and update the bandwidthd package to version 2.0.1_5 for pfSense 2.0.2
Thank you for your time.
-
I have submitted a pull request to make the GitHub changes: https://github.com/pfsense/pfsense-packages/pull/405
Someone with privs also needs time to put the 2.0.1_5 file into http://files.pfsense.org/packages/8/All/ - things look rather busy right now, as they have changed the GitHub repo addresses, so it might take some time to be looked at. -
Jimp has put bandwidthd-2.0.1_5.tbz into http://files.pfsense.org/packages/8/All/ and committed the version change - thanks.
I installed 2.0.2 to a fresh VM, added bandwidthd 2.0.1_5 from the packages menu. It works. Installed Squid and then Lightsquid. Bandwidthd still works. Rebooted - bandwidthd comes up and generates and updates its report and graphs. Seems fixed.
Can others on pfSense 2.0.n try this latest bandwidthd 2.0.1_5 and confirm it is OK. -
Hi Phil
Firstly, thank you to both you and Jimp for resolving this issue.
Just a quick note on the upgrading to the new bandwidthd 2.0.1_5 from previous versions.
I would advise that a full uninstall and re-install is done when upgrading. When running just the upgrade I experienced issues with the logo no longer being present.
Also it is not necessary to uninstall and re-install any other packages as previously stated when we were testing. (eg. Lightsquid)Thanks again for the resolution gents. Much appreciated.
-
Uninstalled 1.4, installed 1.5, pressed Reload on Graph Site and voila there they are.
Works fine.
THX Alex