RRD Graphs for Queues?

jwrascoe · Dec 29, 2009, 1:37 PM

Hello,

I have a question about the RRD Graphs….

System, Traffic & Packets all work fine...
Quality seems to always be blank
Queues and Queue Drops displays a graphic that says "There has been an error creating the graphs. Please check your systemlogs for further details"

I used the traffic shaper to create the VOIP rules (I can send the XML if necessary) and the queues show fine under Status->Queues but just not in RRD.

I have just a simple setup WAN(pppoe)/LAN to run things at my house.

I did notice 1 funny thing when I used the traffic shaper...
The traffic_shaper_wizzard.xml would not let me go past "Enter number of WAN connections:" I would put in 1 and it just kept issuing an error

So I used traffic_shaper_wizard_multi_all.xml and just say that I had 1 interface for each... then said I would be using VOIP/Vonage.. and then just pressed next thru all the other choices.

Everything works fine.. I do have a few questions about the shaper itself.. but will ask them on a different thread.

The system log errors are below....

Any help would be greatly appreciated.

Regards,

James Rascoe

Dec 29 07:54:08 php: /status_rrd_graph_img.php: failed to create graph from /var/db/rrd/lan-queues.rrd, removing database

Dec 29 07:54:08 php: /status_rrd_graph_img.php: Creating rrd update script

Dec 29 07:54:09 php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: Could not parse color in ''/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/wan-queues.rrd-1y.png --start -31622400 -e -43200 --vertical-label "bits/sec" --color SHADEA#eeeeee --color SHADEB#eeeeee --title "hostname - WAN :: Queues - 1 year - 12 hours average" --height 200 --width 620 -x "MONTH:1:MONTH:3:MONTH:1:0:%b" DEF:"wan=/var/db/rrd/wan-queues.rrd:wan:AVERAGE" CDEF:"wan-bytes_out=wan,0,1140224,LIMIT,UN,0,wan,IF" CDEF:"wan-bits_out=wan-bytes_out,8," AREA:"wan-bits_out#:wan" DEF:"qInternet=/var/db/rrd/wan-queues.rrd:qInternet:AVERAGE" CDEF:"qInternet-bytes_out=qInternet,0,1140224,LIMIT,UN,0,qInternet,IF" CDEF:"qInternet-bits_out=qInternet-bytes_out,8," AREA:"qInternet-bits_out#7B0000:qInternet" DEF:"qACK=/var/db/rrd/wan-queues.rrd:qACK:AVERAGE" CDEF:"qACK-bytes_out=qACK,0,1140224,LIMIT,UN,0,qACK,IF" CDEF:"qACK-bits_out=qACK-bytes_out,8,*" AREA:"qACK-bits_out#990000:qACK" DEF:"qDefault

jlepthien · Dec 29, 2009, 1:53 PM

Same problem as I had with the RRD graphs?

http://forum.pfsense.org/index.php/topic,21615.0.html

jwrascoe · Dec 29, 2009, 3:14 PM

jlepthlen,

Yes.. I saw that post and hoped it would fix…

First I removed the traffic & packets .rrd files as outlined in the post... that did not work..

So I then removed all .rrd files rm /var/db/rrd/*.rrd ... and still the same problem...

The actual log message says that it "could not parse color" .. I am not sure how to resolve this.. do you have any insight in this department?

--JR

php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: Could not parse color in ''/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/wan-queues.rrd-4y.png --start -126489600 -e -86400 --vertical-label "bits/sec" --color SHADEA#eeeeee --color SHADEB#eeeeee --title "hostname - WAN :: Queues - 4 years - 1 day average" --height 200 --width 620 -x "MONTH:1:YEAR:1:MONTH:3:0:%b" DEF:"wan=/var/db/rrd/wan-queues.rrd:wan:AVERAGE" CDEF:"wan-bytes_out=wan,0,114176,LIMIT,UN,0,wan,IF" CDEF:"wan-bits_out=wan-bytes_out,8," AREA:"wan-bits_out#:wan" DEF:"qInternet=/var/db/rrd/wan-queues.rrd:qInternet:AVERAGE" CDEF:"qInternet-bytes_out=qInternet,0,114176,LIMIT,UN,0,qInternet,IF" CDEF:"qInternet-bits_out=qInternet-bytes_out,8," AREA:"qInternet-bits_out#7B0000:qInternet" DEF:"qACK=/var/db/rrd/wan-queues.rrd:qACK:AVERAGE" CDEF:"qACK-bytes_out=qACK,0,114176,LIMIT,UN,0,qACK,IF" CDEF:"qACK-bits_out=qACK-bytes_out,8,*" AREA:"qACK-bits_out#990000:qACK" DEF:"qDefault=/var

jlepthien · Dec 29, 2009, 3:20 PM

Did you really mount your cf card read-write? Or are you talking about a full install?

jwrascoe · Dec 29, 2009, 3:28 PM

Ah.. maybe thats the problem… I am new to pfSense (a ZeroShell convert) and the nanobsd version on CF in my appliance.

I got to say this product really handles VOIP correctly in low bandwidth situations... ;)

Anyhow... how do you mount the CF card as read-write?

I just normally just copy over the image on to the CF, plug it into my appliance and let it boot up.

Would you be able to give me instructions on how to do this?

Thanks for your help.

Jim

jlepthien · Dec 29, 2009, 3:37 PM

Sure…

Just ssh to your box and then choose 8 so you get a command prompt. Then do a mount -o rw -u / then rm the files and after that do a mount -o ro -u /
Next go to the GUI and visit the RRD page, choose Settings and then click save. Now the RRD files are beeing generated....

jwrascoe · Dec 29, 2009, 3:46 PM

Humm.. it must be something else…

I followed you instructions with the mount and no luck..

On another note.. by default my CF must be mounted by default as R/W as I was always able to delete the .rrd files without having to do a mount -o rw -u /

any other ideas?

jlepthien · Dec 29, 2009, 4:11 PM

Sure? The Default is ro. Just type in mount and you will see how it is mounted.

jwrascoe · Dec 29, 2009, 4:25 PM

ok.. below is a trace of all my steps…

still no queue graphs..just the error in the log... how strange...

It is Beta code.. so its really no big deal.. just trying to figure out if anyone else is having the same problem.

Is there anything else I can try to assist if this is a bug?

--JR

I rebooted the system from the SSH .. option #5 when it came back up I ran the following commands.

[2.0-BETA1] [admin@pfSense.local]/root(1): mount
/dev/ufs/pfsense0 on / (ufs, local, read-only)
devfs on /dev (devfs, local)
/dev/md0 on /var/tmp (ufs, local)
/dev/md1 on /var (ufs, local)
/dev/ufs/cf on /cf (ufs, local, read-only)
devfs on /var/dhcpd/dev (devfs, local)
[2.0-BETA1] [admin@pfSense.local]/root(2): mount -o rw -u /
[2.0-BETA1] [admin@pfSense.local]/root(3): mount
/dev/ufs/pfsense0 on / (ufs, local)
devfs on /dev (devfs, local)
/dev/md0 on /var/tmp (ufs, local)
/dev/md1 on /var (ufs, local)
/dev/ufs/cf on /cf (ufs, local, read-only)
devfs on /var/dhcpd/dev (devfs, local)
[2.0-BETA1] [admin@pfSense.local]/root(4): rm /var/db/rrd/*.rrd
[2.0-BETA1] [admin@pfSense.local]/root(5): mount -o ro -u /
[2.0-BETA1] [admin@pfSense.local]/root(6): mount
/dev/ufs/pfsense0 on / (ufs, local, read-only)
devfs on /dev (devfs, local)
/dev/md0 on /var/tmp (ufs, local)
/dev/md1 on /var (ufs, local)
/dev/ufs/cf on /cf (ufs, local, read-only)
devfs on /var/dhcpd/dev (devfs, local)
[2.0-BETA1] [admin@pfSense.local]/root(7):
[2.0-BETA1] [admin@pfSense.local]/root(7):

jwrascoe · Dec 29, 2009, 4:29 PM

just for the record.. this is what I am running..

pfSense-2.0-BETA1-4g-20091228-0825-nanobsd.img.gz

2.0-BETA1
built on Sat Dec 26 03:11:15 EST 2009
FreeBSD pfSense.local 8.0-RELEASE-p1 FreeBSD 8.0-RELEASE-p1 #0: Sat Dec 26 03:07:26 EST 2009 sullrich@FreeBSD_8.0_pfSense_2.0-snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense_wrap.8.i386 i386

jlepthien · Dec 29, 2009, 4:42 PM

After you rm the files please type an ls -l /var/db/rrd/ and look if the files are really deleted. And also do not forget to click save on the rrd settings tab in the GUI.

jwrascoe · Dec 29, 2009, 4:51 PM

yes.. it actually deletes the files…

Here is the trace below.. and of course the system, traffic & packets graphs all reset so I know the files have been deleted and recreated... its just the queue graphs that never seem to want to generate...

Here is the trace that shows my steps...

Anything else I can do to assist in the debug process?

Jim

[2.0-BETA1]
[admin@pfSense.local]/root(7): ls /var/db/rrd/
ipsec-packets.rrd lan-queuedrops.rrd system-memory.rrd updaterrd.sh wan-queues.rrd
ipsec-traffic.rrd lan-queues.rrd system-processor.rrd wan-packets.rrd wan-traffic.rrd
lan-packets.rrd lan-traffic.rrd system-states.rrd wan-queuedrops.rrd

[2.0-BETA1]
[admin@pfSense.local]/root(8): rm /var/db/rrd/*.rrd

[2.0-BETA1]
[admin@pfSense.local]/root(9): ls /var/db/rrd/
updaterrd.sh

I went to the web interface and pressed save on Status -> RRD Graphs -> Settings

[2.0-BETA1]
[admin@pfSense.local]/root(10): ls /var/db/rrd/
ipsec-packets.rrd lan-queuedrops.rrd system-memory.rrd updaterrd.sh wan-queues.rrd
ipsec-traffic.rrd lan-queues.rrd system-processor.rrd wan-packets.rrd wan-traffic.rrd
lan-packets.rrd lan-traffic.rrd system-states.rrd wan-queuedrops.rrd

stephenyeong · Dec 31, 2009, 5:00 AM

@jwrascoe:

jlepthlen,

Yes.. I saw that post and hoped it would fix…

First I removed the traffic & packets .rrd files as outlined in the post... that did not work..

So I then removed all .rrd files rm /var/db/rrd/*.rrd ... and still the same problem...

The actual log message says that it "could not parse color" .. I am not sure how to resolve this.. do you have any insight in this department?

--JR

php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: Could not parse color in ''/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/wan-queues.rrd-4y.png --start -126489600 -e -86400 --vertical-label "bits/sec" --color SHADEA#eeeeee --color SHADEB#eeeeee --title "hostname - WAN :: Queues - 4 years - 1 day average" --height 200 --width 620 -x "MONTH:1:YEAR:1:MONTH:3:0:%b" DEF:"wan=/var/db/rrd/wan-queues.rrd:wan:AVERAGE" CDEF:"wan-bytes_out=wan,0,114176,LIMIT,UN,0,wan,IF" CDEF:"wan-bits_out=wan-bytes_out,8," AREA:"wan-bits_out#:wan" DEF:"qInternet=/var/db/rrd/wan-queues.rrd:qInternet:AVERAGE" CDEF:"qInternet-bytes_out=qInternet,0,114176,LIMIT,UN,0,qInternet,IF" CDEF:"qInternet-bits_out=qInternet-bytes_out,8," AREA:"qInternet-bits_out#7B0000:qInternet" DEF:"qACK=/var/db/rrd/wan-queues.rrd:qACK:AVERAGE" CDEF:"qACK-bytes_out=qACK,0,114176,LIMIT,UN,0,qACK,IF" CDEF:"qACK-bits_out=qACK-bytes_out,8,*" AREA:"qACK-bits_out#990000:qACK" DEF:"qDefault=/var

it seems related to

AREA:"wan-bits_out#:

missing color value.
I found many lines of this error in my system.log
just copy one line and replace the color with a valid color code, a valid png produced.

stephenyeong · Dec 31, 2009, 7:16 AM

Solution found
file /usr/local/www/status_rrd_graph_img.php
at about line 677
missing initialize of $t = 0

jwrascoe · Dec 31, 2009, 3:26 PM

Humm….

Mine seems to have it in it already.. but still does not work....

Here is the code section that you are talking about in that file...

could it be something else?

Let me know..

Jim

elseif((strstr($curdatabase, "-queues.rrd")) && (file_exists("$rrddbpath$curdatabase"))) {
/* define graphcmd for queue stats /
$graphcmd = "$rrdtool graph $rrdtmppath$curdatabase-$interval.png ";
$graphcmd .= "--start -$seconds -e -$average ";
$graphcmd .= "--vertical-label "bits/sec" ";
$graphcmd .= "--color SHADEA#eeeeee --color SHADEB#eeeeee ";
$graphcmd .= "--title "hostname - {$prettydb} - {$hperiod} - {$havg} average" ";
$graphcmd .= "--height 200 --width 620 -x "$scale" ";
if ($altq) {
$a_queues =& $altq->get_queue_list();
} else {
$a_queues = array();
$i = 0;
$t = 0;
}
foreach ($a_queues as $name => $q) {
$color = "$colorqueuesup[$t]";
if($t > 0) { $stack = ":STACK"; }
$graphcmd .= "DEF:"$name=$rrddbpath$curdatabase:$name:AVERAGE" ";
$graphcmd .= "CDEF:"$name-bytes_out=$name,0,$speedlimit,LIMIT,UN,0,$name,IF" ";
$graphcmd .= "CDEF:"$name-bits_out=$name-bytes_out,8," ";
$graphcmd .= "$AREA:"$name-bits_out#${color}:$name" ";
$t++;
if($t > 7) { $t = 0; }
}
$graphcmd .= "COMMENT:"\n" ";
$graphcmd .= "COMMENT:"\t\t\t\t\t\t\t\t\t\t\t\t\tdate +\"%b %d %H\:%M\:%S %Y\"" ";
}

jwrascoe · Dec 31, 2009, 5:35 PM

After I actually read the code it was obvious where it was supposed to be… the code is below

All works fine now...

Thanks again for your help..

Happy New Year!

Jim

if ($altq) {
$a_queues =& $altq->get_queue_list();
$t = 0; // This is the line that needs to be added!
} else {
$a_queues = array();
$i = 0;
$t = 0;
}

stephenyeong · Dec 31, 2009, 5:37 PM

Queue-Drop got same problem ( same solution )

cmb · Dec 31, 2009, 8:24 PM

@jwrascoe:

if ($altq) {
$a_queues =& $altq->get_queue_list();
$t = 0; // This is the line that needs to be added!

I committed this fix, thanks

stephenyeong · Jan 1, 2010, 3:18 AM

Also line 705, for the queuedrop, same case, $t need initialize to 0