Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    RRD graphs throughput but not totals

    Scheduled Pinned Locked Moved General pfSense Questions
    37 Posts 6 Posters 6.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C Offline
      churchi
      last edited by

      Cheers thanks Charliem.

      Here is my output:
      [2.2.1-RELEASE][root@server]/var/db/rrd: file ./wan-traffic.rrd
      ./wan-traffic.rrd: RRDTool DB version 0003 64bit aligned little-endian 64bit long (alpha/amd64/ia64)
      [2.2.1-RELEASE][root@server]/var/db/rrd:

      So it looks good to me. Looks like i have 64bit counters in place already. hmm.

      So with this ntopng package, if i install this will it update the default RRD on the system?

      Maybe i need to ask to have the RRD version upgraded to a later release in which some issues may be fixed.

      1 Reply Last reply Reply Quote 0
      • C Offline
        churchi
        last edited by

        Hi charliem,

        i have installed ntopng and that has not made a difference to the graphs. I tested a 1GB download and still the graph increased but the total down the bottom is still broken.

        I guess i will need to log a bug on the tracker.

        1 Reply Last reply Reply Quote 0
        • C Offline
          charliem
          last edited by

          @churchi:

          So with this ntopng package, if i install this will it update the default RRD on the system?

          No, it is installed in parallel.  Each package includes its own set of binaries and libraries, so versions can differ without causing problems with the base install.

          The newer version is /usr/pbi/ntopng-amd64/local/bin/rrdtool.  If you are comfortable coding and experimenting, you could edit a single line in /usr/local/www/status_rrd_graph_img.php, about line 144, to point to the newer rrdtool.  But be ready for things to break!  Try it in a test VM first.

          Edit: Don't do that, it breaks the font rendering.  The newer rrdtool is not a drop-in replacement!

          1 Reply Last reply Reply Quote 0
          • C Offline
            churchi
            last edited by

            ok cool thanks for the clarification and the explanation as to why the other package didnt update the base system.

            So i guess i should just log a bug now and see if we can get the pfSense team to see what the issue is and maybe update the RRD version in the base image?

            Do you think this is the best next step?

            Do you think the pfsense maintainers/admins will be likely to read this thread?

            1 Reply Last reply Reply Quote 0
            • C Offline
              charliem
              last edited by

              Do you have traffic shaper enabled?  The rrd graph generation code does some data validation before plotting, discarding data values that lie outside the range of 0 to $speedlimit.  If the traffic shaper is enabled, the value for $speedlimit is taken from bandwidth defined for the queue, otherwise it's set to 12,500,000

              If you increased your wan bandwidth significantly, you need to adjust your interface bandwidth definitions in the traffic shaper.

              To check what $speedlimit is currently set to, you can add this to /usr/local/www/status_rrd_graph_img.php around line 253:

              $speedlimit = ($upstream + $downstream);                                                                      <===== Existing line
              log_error("speedlimit $speedlimit, upstream $upstream, downstream $downstream");    <===== Add this line

              Then view an rrd graph and check the result with:

              [2.2.1-DEVELOPMENT][root@pfsense.localdomain]/usr/local/www: clog /var/log/system.log | grep speedlimit |tail
              Mar 21 09:28:15 pfsense php-fpm[97229]: /status_rrd_graph_img.php: sspeedlimit 248320, upstream 124160, downstream 124160

              Don't forget to go back and comment or remove the debugging line when you're done.

              1 Reply Last reply Reply Quote 0
              • C Offline
                churchi
                last edited by

                Sorry for the delay in getting back to you.

                I have done what you asked and here is the results.

                Mar 24 23:04:01 gw1 php-fpm[89472]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                Mar 24 23:04:01 gw1 php-fpm[89472]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                Mar 24 23:04:02 gw1 php-fpm[89472]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                Mar 24 23:04:02 gw1 php-fpm[89472]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                Mar 24 23:04:02 gw1 php-fpm[89472]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                Mar 24 23:04:02 gw1 php-fpm[823]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                Mar 24 23:04:02 gw1 php-fpm[823]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                Mar 24 23:04:02 gw1 php-fpm[89472]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                Mar 24 23:04:02 gw1 php-fpm[89472]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                Mar 24 23:04:02 gw1 php-fpm[89472]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400

                1 Reply Last reply Reply Quote 0
                • C Offline
                  churchi
                  last edited by

                  Ok so i did another test. When my Limiter changed at 11:30pm to 950Kbps up from 450Kbps on the earlier test, this is the results.

                  Mar 24 23:57:38 gw1 php-fpm[88321]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                  Mar 24 23:57:38 gw1 php-fpm[88321]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                  Mar 24 23:57:38 gw1 php-fpm[88321]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                  Mar 24 23:57:39 gw1 php-fpm[88321]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                  Mar 24 23:57:39 gw1 php-fpm[88321]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                  Mar 24 23:57:39 gw1 php-fpm[88321]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                  Mar 24 23:57:39 gw1 php-fpm[88321]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                  Mar 24 23:57:39 gw1 php-fpm[88321]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                  Mar 24 23:57:39 gw1 php-fpm[88321]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400
                  Mar 24 23:57:39 gw1 php-fpm[88321]: /status_rrd_graph_img.php: speedlimit 460800, upstream 230400, downstream 230400

                  No change.

                  So it looks like the speed limit is being set somewhere. Where do you think that limit will be set?

                  How can i go about resetting this to correct values so that the data is not purged/disguarded by the graphing?

                  The actual graphs are fine, just the numeral calculations down the bottom of the graph that are not adding up correctly.

                  Thanks.

                  1 Reply Last reply Reply Quote 0
                  • C Offline
                    charliem
                    last edited by

                    @churchi:

                    Ok so i did another test. When my Limiter changed at 11:30pm …  No change.

                    Not the limiter, but the traffic shaper I think.

                    So it looks like the speed limit is being set somewhere. Where do you think that limit will be set?

                    How can i go about resetting this to correct values so that the data is not purged/disguarded by the graphing?

                    $speedlimit is calculated in the very file you just modified.  If you suspect it could be causing an issue, just set it to 12500000 and try it, though the limits you posted should be OK.

                    The alternative is for you to find the rrdtool man pages, and try to analyze the RPN expressions in the "$graphcmd" stanzas in that file (/usr/local/www/status_rrd_graph_img.php).

                    With your graphs and the others posted here, it seems that the average is being calculated correctly.  That is: (printed average throughput) * (time span of graph) == (about what you think is the correct total traffic for the period).  Can you confirm that?

                    If you really need to generate a periodic throughput summary, I'd suggest you script up an alternative using the pfctl command I listed early in the thread, rather than going through the web interface.

                    1 Reply Last reply Reply Quote 0
                    • C Offline
                      churchi
                      last edited by

                      Thanks Charlie.

                      $speedlimit is calculated in the very file you just modified.  If you suspect it could be causing an issue, just set it to 12500000 and try it, though the limits you posted should be OK.
                      Where abouts in the .php file do i manually set the 12500000 to?

                      And where is it automatically getting the speedlimit 460800 from? I have not set this anywhere in my pfSense config at all? Seems like a random number for me. Then it is splitting it equally to upstream and down stream.

                      Thanks.

                      1 Reply Last reply Reply Quote 0
                      • C Offline
                        churchi
                        last edited by

                        Hi Charlie,

                        i have checked out my other graphs, like the LAN interface and other VLAN interfaces and i can see the same issue. There is a speed limit set somewhere in my config that has affected the RRD reporting on the graphs.

                        Here is another example.

                        ![pfsense WAN traffic pic -03.png](/public/imported_attachments/1/pfsense WAN traffic pic -03.png)
                        ![pfsense WAN traffic pic -03.png_thumb](/public/imported_attachments/1/pfsense WAN traffic pic -03.png_thumb)

                        1 Reply Last reply Reply Quote 0
                        • C Offline
                          charliem
                          last edited by

                          @churchi:

                          Where abouts in the .php file do i manually set the 12500000 to?

                          Honestly, if you cannot follow the code in that file then you shouldn't be poking around in it; sorry for the suggestions.

                          File a bug.

                          1 Reply Last reply Reply Quote 0
                          • I Offline
                            icemanncsu
                            last edited by

                            Can someone please open a bug on this… Us fiber users obviously exceed the normal limits on the WAN interface.

                            1 Reply Last reply Reply Quote 0
                            • H Offline
                              Harvy66
                              last edited by

                              So again, the "period" in-pass is obviously not correct. The claimed 224.78Mb/s over the 8 hour period is about 750GB, not 21.71GB as claimed.

                              Question for OP. Have you fried a fresh install? Obviously something is wrong, but you seem to be the only person complaining about this issue. Not to say others don't have it, just that you're the only one in here talking about it.

                              1 Reply Last reply Reply Quote 0
                              • I Offline
                                icemanncsu
                                last edited by

                                @Harvy66:

                                So again, the "period" in-pass is obviously not correct. The claimed 224.78Mb/s over the 8 hour period is about 750GB, not 21.71GB as claimed.

                                Question for OP. Have you fried a fresh install? Obviously something is wrong, but you seem to be the only person complaining about this issue. Not to say others don't have it, just that you're the only one in here talking about it.

                                I have installed pfSense several times, and this is the case with each install, this one being v2.2.2.

                                1 Reply Last reply Reply Quote 0
                                • T Offline
                                  tim.mcmanus
                                  last edited by

                                  @icemanncsu:

                                  Can someone please open a bug on this… Us fiber users obviously exceed the normal limits on the WAN interface.

                                  Is it only happening to the WAN graph, or all of them?  My LAN looks like your WAN connection, and I don't have any issues on any of my interfaces.  So I'm curious if it's just this one interface.

                                  1 Reply Last reply Reply Quote 0
                                  • I Offline
                                    icemanncsu
                                    last edited by

                                    Happens on both interfaces… See below.

                                    1 Reply Last reply Reply Quote 0
                                    • T Offline
                                      tim.mcmanus
                                      last edited by

                                      When you install, are you restoring your configuration, or are you configuring the settings from scratch?

                                      I would suggest a clean install and a manual reconfigure.  I'm guessing that there might be something wonky in your configurations that is causing this.  I cannot recreate the problem.

                                      1 Reply Last reply Reply Quote 0
                                      • I Offline
                                        icemanncsu
                                        last edited by

                                        Always clean installs, no restoration of previous configs.

                                        If its worth noting, I have a static IP on my WAN interface.

                                        1 Reply Last reply Reply Quote 0
                                        • T Offline
                                          tim.mcmanus
                                          last edited by

                                          I too have a static IP on one WAN interface and another DHCP on the second WAN interface.

                                          The only other thing I can think of is a bug in the interface driver supplied by BSD.

                                          1 Reply Last reply Reply Quote 0
                                          • C Offline
                                            charliem
                                            last edited by

                                            Do you have the traffic shaper enabled?

                                            Have you tried the '/sbin/pfctl -vvsI -i <your_interface>' commands suggested early in this thread?  (https://forum.pfsense.org/index.php?topic=89923.msg498803#msg498803)  Try it say in the morning, then 8 hours later and look for rollover.</your_interface>

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.