RRD graphs error on 2.0-BETA4 (i386) Aug 27 23:40:39 (RESOLVED)



  • There seems to be an error in the scripts involved in the current build.  It seems that when the graphs are viewed just after midnight (it happens all the time), the scripts attempt to generate the RRD graphs with a Start Time that begins later than the End time?

    Edit:  
    Just to update, this is a full harddisk install on a Western Digital Raptor 74GB.
    No packages are installed.
    Oddly, the problem goes away when I access the WebGUI via Internet Explorer 8 (from my CyberCafe via WAN).
    The problem persists with Firefox 3.6 though (internally from LAN or remotely from my Laptop).

    Example of the error I've gotten from the system logs.

    php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: start (1283126400) should be less than end (1283103301)/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/lan-queues.rrd-12hour.png –start 1283126400 --end 1283103301 --vertical-label "bits/sec" --color SHADEA#eeeeee --color SHADEB#eeeeee --title "hostname - LAN :: Queues - 6 hours - 1 minute average" --height 200 --width 620 DEF:"lan=/var/db/rrd/lan-queues.rrd:lan:AVERAGE" CDEF:"lan-bytes_out=lan,0,209715200,LIMIT,UN,0,lan,IF" CDEF:"lan-bits_out=lan-bytes_out,8," AREA:"lan-bits_out#666666:lan" DEF:"qInternet=/var/db/rrd/lan-queues.rrd:qInternet:AVERAGE" CDEF:"qInternet-bytes_out=qInternet,0,209715200,LIMIT,UN,0,qInternet,IF" CDEF:"qInternet-bits_out=qInternet-bytes_out,8," AREA:"qInternet-bits_out#FF3300:qInternet" DEF:"qACK=/var/db/rrd/lan-queues.rrd:qACK:AVERAGE" CDEF:"qACK-bytes_out=qACK,0,209715200,LIMIT,UN,0,qACK,IF" CDEF:"qACK-bits_out=qACK-bytes_out,8,*" AREA:"qACK-bits_out#3366FF:

    Update:
    I've just suffered a major problem (filter.inc.php got wiped out by the Edit File function) and did a Full Install using:
    i386 build of 2.0 Beta4 built-on Fri Sep 17 14:32:23 EDT 2010.

    The hourly graph is now showing up correctly after midnight so the issue can be considered to be resolved.  Kudos to the pfsense team!



  • just switched to Beta4 on a clean CF-card with my alix box and can confirm this.

    2.0-BETA4  (i386)
    built on Thu Sep 2 02:39:10 EDT 2010

    regards,

    foo



  • The error still remains after upgrading to:  2.0-BETA4  (i386) built on Thu Sep 2 00:18:53 EDT 2010

    I've tried removing all the files in /var/db/rrd followed by disabling and re-enabling the RRD graphing but the problem still remains.



  • Oddly, I've actually tried accessing the RRD graphs (remotely from WAN) using Internet Explorer 8 and the problem went away.  Local/ Remote access using Firefox 3.6 still shows the problem.  It is most intriguing.  I'll install Chrome and Opera to check again.



  • i can confirm this too, last update was last sunday 05.



  • but neither with cleared cache, nor remotely over wan i couldn't display the rrdgraphs.. i don't think the browsers rendering engine is the fault..


  • Rebel Alliance Developer Netgate

    Graphs show up fine for me on a snap from yesterday:
    2.0-BETA4  (i386)
    built on Mon Sep 6 22:04:59 EDT 2010

    I don't get any errors in the system logs, and the graphs looks ok.



  • problem gone, without restarting or upgrading.. abra cadabra.. don't know what happened, but anyway thanks for taking a look everyone..



  • Having exact same problem with latest snap

    Sep 8 06:45:54
    –--------
    php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: start (1283950800) should be less than end (1283946354)/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/system-processor.rrd-12hour.png --start 1283950800 --end 1283946354 --vertical-label "utilization, number" --color SHADEA#eeeeee --color SHADEB#eeeeee --title "hostname - System :: Processor - 1 hour - 1 minute average" --height 200 --width 620 DEF:"user=/var/db/rrd/system-processor.rrd:user:AVERAGE" DEF:"nice=/var/db/rrd/system-processor.rrd:nice:AVERAGE" DEF:"system=/var/db/rrd/system-processor.rrd:system:AVERAGE" DEF:"interrupt=/var/db/rrd/system-processor.rrd:interrupt:AVERAGE" DEF:"processes=/var/db/rrd/system-processor.rrd:processes:AVERAGE" AREA:"user#990000:user" AREA:"nice#a83c3c:nice:STACK" AREA:"system#b36666:system:STACK" AREA:"interrupt#bd9090:interrupt:STACK" LINE2:"processes#cccccc:processes" COMMENT:"\n" COMMENT:" minimum average maximum current


    I'll check this afternoon to see if it fixes itself for me too :)



  • @jimp:

    Graphs show up fine for me on a snap from yesterday:
    2.0-BETA4  (i386)
    built on Mon Sep 6 22:04:59 EDT 2010

    I don't get any errors in the system logs, and the graphs looks ok.

    Ok..  I've narrowed down the problem.

    The first graph seems to be the problem.

    This only happens when the local time on the system is between 12 midnight and 12 noon.

    Viewing from LAN/ WAN with any browser yields the correct graphs only between 12 noon and 12 midnight.

    Seems to me that the date/ time calculation is messed up when the RRD graphs are shifted to daily 12 hour graphs and the graph starts on or after 12 midnight (where the days are split).
    The script somehow attempts to generate a graph where the end time is before the start time.


  • Rebel Alliance Developer Netgate

    That might explain why I don't see it when I'm testing in my VMs, most of them are left on UTC, so during my working hours it's always after noon and before midnight.



  • Sep 10 00:40:13 php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: start (1284120000) should be less than end (1284093613)/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/GW_WAN-quality.rrd-12hour.png \ –start 1284120000 --end 1284093613 \ --title "hostname - GW_WAN :: Quality - 7 hours - 1 minute average" \ --color SHADEA#eeeeee --color SHADEB#eeeeee \ --vertical-label "ms / %" \ --height 200 --width 620 \ --lower-limit 0 \ DEF:delayraw=/var/db/rrd/GW_WAN-quality.rrd:delay:AVERAGE \ DEF:loss=/var/db/rrd/GW_WAN-quality.rrd:loss:AVERAGE \ "CDEF:delay=delayraw,1000," \ "CDEF:roundavg=delay,PREV(delay),+,2,/" \ "CDEF:loss10=loss,-1," \ "CDEF:r0=delay,20,MIN" \ "CDEF:r1=delay,60,MIN" \ "CDEF:r2=delay,180,MIN" \ "CDEF:r3=delay,420,MIN" \ COMMENT:" Delay Packet loss\n" \ AREA:delay#990000:"> 420 ms" \ GPRINT:delay:MIN:" Min: %7.2lf ms" \ GPRINT:loss:MIN:" Min: %3.1lf %%\n" \ AREA:r3#a83c3c:"180-420
    Sep 10 00:40:00 php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: start (1284120000) should be less than end (1284093600)/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/GW_WAN-quality.rrd-12hour.png \ --start 1284120000 --end 1284093600 \ --title "hostname - GW_WAN :: Quality - 7 hours - 1 minute average" \ --color SHADEA#eeeeee --color SHADEB#eeeeee \ --vertical-label "ms / %" \ --height 200 --width 620 \ --lower-limit 0 \ DEF:delayraw=/var/db/rrd/GW_WAN-quality.rrd:delay:AVERAGE \ DEF:loss=/var/db/rrd/GW_WAN-quality.rrd:loss:AVERAGE \ "CDEF:delay=delayraw,1000," \ "CDEF:roundavg=delay,PREV(delay),+,2,/" \ "CDEF:loss10=loss,-1," \ "CDEF:r0=delay,20,MIN" \ "CDEF:r1=delay,60,MIN" \ "CDEF:r2=delay,180,MIN" \ "CDEF:r3=delay,420,MIN" \ COMMENT:" Delay Packet loss\n" \ AREA:delay#990000:"> 420 ms" \ GPRINT:delay:MIN:" Min: %7.2lf ms" \ GPRINT:loss:MIN:" Min: %3.1lf %%\n" \ AREA:r3#a83c3c:"180-420

    Yep..happens after midnight.  Its OK during the day.



  • 2.0-BETA4  (i386) built on Fri Sep 10 05:09:52 EDT 2010

    09-11-2010 00:52:40 User.Warning 192.168.1.1 Sep 11 00:52:46 php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: start (1284202800) should be less than end (1284177166)/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/wan-packets.rrd-12hour.png –start 1284202800 --end 1284177166 --vertical-label "packets/sec" --color SHADEA#eeeeee --color SHADEB#eeeeee --title "hostname - INTERNET :: Packets - 7 hours - 1 minute average" --height 200 --width 620 DEF:"wan-in_pps_pass=/var/db/rrd/wan-packets.rrd:inpass:AVERAGE" DEF:"wan-out_pps_pass=/var/db/rrd/wan-packets.rrd:outpass:AVERAGE" DEF:"wan-in_pps_block=/var/db/rrd/wan-packets.rrd:inblock:AVERAGE" DEF:"wan-out_pps_block=/var/db/rrd/wan-packets.rrd:outblock:AVERAGE" CDEF:"wan-in_pps=wan-in_pps_pass,wan-in_pps_block,+" CDEF:"wan-out_pps=wan-out_pps_pass,wan-out_pps_block,+" CDEF:"wan-out_pps_pass_neg=wan-out_pps_pass,-1," CDEF:"wan-out_pps_block_neg=wan-out_pps_block,-1," CDEF:"wan-pps_in_pass=wan-in_pps_pass,0,12500000,LIMIT,UN,0,wan-in_pps_pass,IF,60,*" C<000>



  • @jimp:

    That might explain why I don't see it when I'm testing in my VMs, most of them are left on UTC, so during my working hours it's always after noon and before midnight.

    Aye.  It is about the same for me.  I have one box in a production environment (my CyberCafe) and one box at home (testing done here before rolling out).  Unfortunately, I can only check in after midnight since I can only work after closing hours and generally get home after midnight as well.

    Tentatively, I've had both boxes rolled-backed to Aug 8 build.  This issue only seemed to arise after the 'fix' for the RRD graphs were commited on 27th Aug.



  • I'll look into it. This is something that never occured to me, I don't get to code a lot after 12 pm anymore which kind of explains the issue.



  • @databeestje:

    I'll look into it. This is something that never occured to me, I don't get to code a lot after 12 pm anymore which kind of explains the issue.

    Thanks.
    It isn't extremely important as far as bugs go (the RRD data isn't corrupted at all) but it will be nice to have it fixed.
    IMO, the RRD graphing bug fix commited on 27th Aug is probably more important for most users since it consistently solved the issues with the quality and queues graphs not showing up.  I can always live with viewing the graphs during the day when required.

    On a side note, does anyone know how the custom graphs timing is formatted?  eg.  1283103301  Which would be the year/ month/ week or day etc?


  • Rebel Alliance Developer Netgate

    It's a unix timestamp, though it would be nice to have that be a more "friendly" format…



  • Thanks!
    I'll just bookmark the Unix time stamp converter and use it then.



  • If someone has a good calendar or time picker tool that easily fills the field with the correct unix timestamp value I'm open for suggestions.


  • Rebel Alliance Developer Netgate

    This one looks neat…
    http://www.rainforestnet.com/datetimepicker.htm

    You can use PHP's time functions to convert any date/time string to (and from) a unix timestamp before using it, so it doesn't necessarily have to expose a unix timestamp to the user.



  • @databeestje:

    If someone has a good calendar or time picker tool that easily fills the field with the correct unix timestamp value I'm open for suggestions.

    I would firmly recommend jQuery and one of the many associated plugins if time picking is desired as well.
    I have worked a tremendous amount with jQuery and it never fails to impress me.

    Here is the "builtin" datepicker with an icon trigger: http://jqueryui.com/demos/datepicker/#icon-trigger
    It is extremely flexible as you can see from the "Examples" section on the right hand side of the page. You can have it restrict to certain rangers, give drop down selectors for the months/years, display multiple months at a time, and it will output to any string format you want or multiple to multiple target locations if you wanted. I would recommend however placing a human readable format, ideally the ISO/ANSI standard of YYYY-MM-DD HH:II:SS since you can wrap most anything with strtotime() in PHP to get the unix timestamp you need on the back end. That gives more freedom to the person inputing as well if they wish to do it manually instead of using the picker.

    Now in addition to the native picker, there are some other options to extend functionality. For example:
    http://www.filamentgroup.com/lab/date_range_picker_using_jquery_ui_16_and_jquery_ui_css_framework/
    That simplifies the selecting of a date range if you want to use only one input field. When triggered it offers one click access to "Today", "Last week", "Last month" etc… while also giving the three click option of a custom date range.

    Time pickers are a little more tricky because they can also be more trouble than they are worth in terms of how easy they are to use.
    These use Google Calendar style time pickers:

    These use sliders to select time:

    This one uses a more flow oriented point click method:

    I would strongly recommend looking this route as you will end up with a highly browser independent solution which is backed by a very reliable JS framework. Now jQuery is not without its flaws, but back when I was in the position I had to choose one to live with, after all the evaluations I did, I picked it over the other options and have been happy with my choice. I have no qualms with the fact that I am now married to it. And you needn't only take my word for it, it seems to be doing well for these guys too: http://docs.jquery.com/Sites_Using_jQuery (I realize it is a logical fallacy to think something is good based on how many or who is using it, but I did want to point out that there are some precedents out there).

    I hope this helps, and if you need help implementing one of these, get my attention somehow and I will lend a hand as time permits.

    Cheers,

    CHgsd


  • Rebel Alliance Developer Netgate

    We can't use jquery. Currently, prototype is used and unless someone wants to completely rewrite all of the JS to be jquery, it'll stay prototype :-)

    It's a discussion that's been raised before, and already decided upon.

    Anything used has to either be based on prototype, or neutral in terms of the framework.



  • Well it is what it is I suppose. With that in mind, Any+Time used to use Prototype before they evolved to jQuery. They still have their old Prototype version available here:

    It is probably your best bet because it has I believe all the features you need (down to time selection). Then if you, me, or someone else, were to ever make the migration from Prototype to jQuery like the aforementioned project, there would be a simpler upgrade path ;)

    Other Prototype options:

    I hope this helps.

    Cheers,

    CHgsd



  • This is on the latest update (from the autoupdater page)

    
    Dec 31 01:50:34 	kernel: FreeBSD 8.1-RELEASE-p2 #1: Thu Dec 30 20:39:36 EST 2010
    Dec 31 01:50:34 	kernel: sullrich@FreeBSD_8.0_pfSense_2.0-snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense_SMP.8 i386
    
    

    I have this a bit up in the log as well:

    
    Dec 31 01:50:48 	php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/lan-queues.rrd -t :lan:qInternet:qACK:qDefault:qP2P:qGames:qOthersHigh:qOthersLow N:U:U:U:U:U:U:U:U' returned exit code '1', the output was 'ERROR: unknown DS name '''
    Dec 31 01:50:48 	php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/lan-queuedrops.rrd -t :lan:qInternet:qACK:qDefault:qP2P:qGames:qOthersHigh:qOthersLow N:U:U:U:U:U:U:U:U' returned exit code '1', the output was 'ERROR: unknown DS name '''
    Dec 31 01:50:48 	php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/wan-queues.rrd -t :wan:qACK:qDefault:qP2P:qGames:qOthersHigh:qOthersLow N:U:U:U:U:U:U:U' returned exit code '1', the output was 'ERROR: unknown DS name '''
    Dec 31 01:50:48 	php: : The command '/usr/bin/nice -n20 /usr/local/bin/rrdtool update /var/db/rrd/wan-queuedrops.rrd -t :wan:qACK:qDefault:qP2P:qGames:qOthersHigh:qOthersLow N:U:U:U:U:U:U:U' returned exit code '1', the output was 'ERROR: unknown DS name '''
    
    

    Then a bit further down, I see:

    
    Dec 31 02:01:29 	php: /status_rrd_graph_img.php: Failed to create graph with error code 1, the error is: ERROR: start (1293800400) should be less than end (1293778889)/usr/bin/nice -n20 /usr/local/bin/rrdtool graph /tmp/system-processor.rrd-8hour.png --start 1293800400 --end 1293778889 --vertical-label "utilization, number" --color SHADEA#eeeeee --color SHADEB#eeeeee --title "`hostname` - System :: Processor - 6 hours - 1 minute average" --height 200 --width 620 DEF:"user=/var/db/rrd/system-processor.rrd:user:AVERAGE" DEF:"nice=/var/db/rrd/system-processor.rrd:nice:AVERAGE" DEF:"system=/var/db/rrd/system-processor.rrd:system:AVERAGE" DEF:"interrupt=/var/db/rrd/system-processor.rrd:interrupt:AVERAGE" DEF:"processes=/var/db/rrd/system-processor.rrd:processes:AVERAGE" AREA:"user#990000:user" AREA:"nice#a83c3c:nice:STACK" AREA:"system#b36666:system:STACK" AREA:"interrupt#bd9090:interrupt:STACK" LINE2:"processes#cccccc:processes" COMMENT:"\n" COMMENT:" minimum average maximum current
    
    

Locked