• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Grafana Dashboard using Telegraf with additional plugins

Scheduled Pinned Locked Moved pfSense Packages
173 Posts 28 Posters 71.8k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • E
    erbalo @bigjohns97
    last edited by Jan 27, 2021, 9:13 PM

    @bigjohns97 said in Grafana Dashboard using Telegraf with additional plugins:

    @erbalo said in Grafana Dashboard using Telegraf with additional plugins:

    @bigjohns97 said in Grafana Dashboard using Telegraf with additional plugins:

    @erbalo said in Grafana Dashboard using Telegraf with additional plugins:

    Should that be ok so?

    #!/bin/sh
    /usr/local/sbin/unbound-control -c /var/unbound/unbound.conf $* | grep -vE 'thread[0-9]+'
    unbound-control -c /var/unbound/unbound.conf stats_noreset | grep total.num
    

    When I tried running that command that was originally in there (your top line) it didn't work.

    Just it should be:

    #!/bin/sh
    unbound-control -c /var/unbound/unbound.conf stats_noreset | grep total.num
    

    ?

    That is what I am running, correct, and it didn't affect any other metrics.

    I don't receive any data to grafana, what can be wrong?

    B 1 Reply Last reply Jan 27, 2021, 9:20 PM Reply Quote 0
    • B
      bigjohns97 @erbalo
      last edited by bigjohns97 Jan 27, 2021, 9:31 PM Jan 27, 2021, 9:20 PM

      @erbalo said in Grafana Dashboard using Telegraf with additional plugins:

      @bigjohns97 said in Grafana Dashboard using Telegraf with additional plugins:

      @erbalo said in Grafana Dashboard using Telegraf with additional plugins:

      @bigjohns97 said in Grafana Dashboard using Telegraf with additional plugins:

      @erbalo said in Grafana Dashboard using Telegraf with additional plugins:

      Should that be ok so?

      #!/bin/sh
      /usr/local/sbin/unbound-control -c /var/unbound/unbound.conf $* | grep -vE 'thread[0-9]+'
      unbound-control -c /var/unbound/unbound.conf stats_noreset | grep total.num
      

      When I tried running that command that was originally in there (your top line) it didn't work.

      Just it should be:

      #!/bin/sh
      unbound-control -c /var/unbound/unbound.conf stats_noreset | grep total.num
      

      ?

      That is what I am running, correct, and it didn't affect any other metrics.

      I don't receive any data to grafana, what can be wrong?

      Make sure the data source and table is the same on your side as what I posted.

      @erbalo copy the JSON from above again I replaced some of my entries with variables so be more plug and play.

      1 Reply Last reply Reply Quote 0
      • V
        von Papst @bigjohns97
        last edited by Jan 28, 2021, 2:46 PM

        @bigjohns97 got it running. But still missing CPU, memory and system load data. What am I missing?

        B 1 Reply Last reply Jan 28, 2021, 2:53 PM Reply Quote 0
        • B
          bigjohns97 @von Papst
          last edited by Jan 28, 2021, 2:53 PM

          @von-papst This is just a single panel, add it to the dashboard being developed in this thread.

          J 1 Reply Last reply Feb 23, 2021, 2:18 AM Reply Quote 0
          • J
            jpcapone @bigjohns97
            last edited by Feb 23, 2021, 2:18 AM

            This post is deleted!
            1 Reply Last reply Reply Quote 0
            • J
              jpcapone
              last edited by Feb 23, 2021, 3:04 AM

              I think I am late to the party but I am trying to figure some things out. I am running pfsense in a vm on esxi 6.7. I was able to figure out enough to get most of the panels working but I think the scripts arent working. I am not super familiar with FreeBSD so I am finding it difficult to determine how to test run the scripts so that I can remediate. I am pretty sure the scripts arent running because all of the panels aren't populated with data and when I do a show measurements on the DB I get only the entries listed below. Any advice would be appreciated.
              cpu
              disk
              diskio
              mem
              net
              pf
              processes
              swap
              system

              B 1 Reply Last reply Feb 23, 2021, 1:47 PM Reply Quote 0
              • B
                bigjohns97 @jpcapone
                last edited by Feb 23, 2021, 1:47 PM

                @jpcapone This is the best way to troubleshoot the plugins

                Taken from https://github.com/VictorRobellini/pfSense-Dashboard

                36550fb1-a659-4d99-8e4b-7aa80294b608-image.png

                J 2 Replies Last reply Feb 23, 2021, 6:55 PM Reply Quote 1
                • J
                  jpcapone @bigjohns97
                  last edited by Feb 23, 2021, 6:55 PM

                  This post is deleted!
                  1 Reply Last reply Reply Quote 0
                  • J
                    jpcapone @bigjohns97
                    last edited by Feb 23, 2021, 7:05 PM

                    @bigjohns97
                    Thanks for that. I was able to figure out the issues with the plugins. Now I am just left with what I have pasted below. Can you please advise?

                    2021-02-23T19:01:58Z I! Loaded inputs: cpu disk diskio exec kernel logparser (2x) mem net pf processes swap system
                    2021-02-23T19:01:58Z I! Loaded aggregators:
                    2021-02-23T19:01:58Z I! Loaded processors:
                    2021-02-23T19:01:58Z I! Loaded outputs: influxdb
                    2021-02-23T19:01:58Z I! Tags enabled: host=xxxxpfSense.xxxxolutions.co
                    2021-02-23T19:01:58Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"xxxxpfSense.xxxxolutions.co", Flush Interval:10s
                    2021-02-23T19:01:58Z D! [agent] Initializing plugins
                    2021-02-23T19:01:58Z W! [inputs.logparser] The logparser plugin is deprecated; please use the 'tail' input with the 'grok' data_format
                    2021-02-23T19:01:58Z W! [inputs.logparser] The logparser plugin is deprecated; please use the 'tail' input with the 'grok' data_format
                    2021-02-23T19:01:58Z D! [agent] Connecting outputs
                    2021-02-23T19:01:58Z D! [agent] Attempting connection to [outputs.influxdb]
                    2021-02-23T19:01:58Z D! [agent] Successfully connected to outputs.influxdb
                    2021-02-23T19:01:58Z D! [agent] Starting service inputs
                    2021-02-23T19:01:58Z E! [inputs.logparser] Error in plugin: open /var/log/pfblockerng/dnsbl.log: no such file or directory
                    2021-02-23T19:01:58Z E! [inputs.logparser] Error in plugin: open /var/log/pfblockerng/ip_block.log: no such file or directory
                    2021-02-23T19:02:00Z E! [inputs.logparser] Error in plugin: open /var/log/pfblockerng/dnsbl.log: no such file or directory
                    2021-02-23T19:02:00Z E! [inputs.logparser] Error in plugin: open /var/log/pfblockerng/ip_block.log: no such file or directory

                    B 1 Reply Last reply Feb 23, 2021, 8:33 PM Reply Quote 0
                    • B
                      bigjohns97 @jpcapone
                      last edited by Feb 23, 2021, 8:33 PM

                      @jpcapone said in Grafana Dashboard using Telegraf with additional plugins:

                      @bigjohns97
                      Thanks for that. I was able to figure out the issues with the plugins. Now I am just left with what I have pasted below. Can you please advise?

                      2021-02-23T19:01:58Z I! Loaded inputs: cpu disk diskio exec kernel logparser (2x) mem net pf processes swap system
                      2021-02-23T19:01:58Z I! Loaded aggregators:
                      2021-02-23T19:01:58Z I! Loaded processors:
                      2021-02-23T19:01:58Z I! Loaded outputs: influxdb
                      2021-02-23T19:01:58Z I! Tags enabled: host=xxxxpfSense.xxxxolutions.co
                      2021-02-23T19:01:58Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"xxxxpfSense.xxxxolutions.co", Flush Interval:10s
                      2021-02-23T19:01:58Z D! [agent] Initializing plugins
                      2021-02-23T19:01:58Z W! [inputs.logparser] The logparser plugin is deprecated; please use the 'tail' input with the 'grok' data_format
                      2021-02-23T19:01:58Z W! [inputs.logparser] The logparser plugin is deprecated; please use the 'tail' input with the 'grok' data_format
                      2021-02-23T19:01:58Z D! [agent] Connecting outputs
                      2021-02-23T19:01:58Z D! [agent] Attempting connection to [outputs.influxdb]
                      2021-02-23T19:01:58Z D! [agent] Successfully connected to outputs.influxdb
                      2021-02-23T19:01:58Z D! [agent] Starting service inputs
                      2021-02-23T19:01:58Z E! [inputs.logparser] Error in plugin: open /var/log/pfblockerng/dnsbl.log: no such file or directory
                      2021-02-23T19:01:58Z E! [inputs.logparser] Error in plugin: open /var/log/pfblockerng/ip_block.log: no such file or directory
                      2021-02-23T19:02:00Z E! [inputs.logparser] Error in plugin: open /var/log/pfblockerng/dnsbl.log: no such file or directory
                      2021-02-23T19:02:00Z E! [inputs.logparser] Error in plugin: open /var/log/pfblockerng/ip_block.log: no such file or directory

                      Looks like you aren't using pfblockerng is that the case?

                      Are you now getting data on the influxdb side and in turn on your dashboard?

                      J 2 Replies Last reply Feb 24, 2021, 12:41 AM Reply Quote 0
                      • J
                        jpcapone @bigjohns97
                        last edited by Feb 24, 2021, 12:41 AM

                        @bigjohns97
                        yup, I am getting data but I am still not seeing the same measurements in my DB that you see in the in the troubleshooting section. Also, I had to turn on pfblockerng and now but I am still not getting any data from it in grafana. Any suggestions?
                        9c0b0377-11f1-4e51-8e28-9fdd20cac828-image.png

                        1 Reply Last reply Reply Quote 0
                        • J
                          jpcapone @bigjohns97
                          last edited by Feb 24, 2021, 1:46 AM

                          @bigjohns97 I think I got it. No data was being generated because I wasn't surfing after I set up pfblockerng. I am very new to this as you can tell. Thanks for your help!!!!

                          1 Reply Last reply Reply Quote 2
                          • V
                            VictorRobellini
                            last edited by VictorRobellini Mar 5, 2021, 1:29 AM Mar 4, 2021, 11:53 PM

                            I've made some updates to the dashboard. I would love feedback.

                            https://github.com/VictorRobellini/pfSense-Dashboard/commit/520eea4f49b5107cb79e887ec94951c015d52a6e

                            B 1 Reply Last reply Mar 6, 2021, 6:22 PM Reply Quote 2
                            • B
                              bigjohns97 @VictorRobellini
                              last edited by Mar 6, 2021, 6:22 PM

                              @victorrobellini said in Grafana Dashboard using Telegraf with additional plugins:

                              I've made some updates to the dashboard. I would love feedback.

                              https://github.com/VictorRobellini/pfSense-Dashboard/commit/520eea4f49b5107cb79e887ec94951c015d52a6e

                              You didn't like my unbound cache hit panel above?

                              V 3 Replies Last reply Mar 6, 2021, 6:59 PM Reply Quote 0
                              • V
                                VictorRobellini
                                last edited by Mar 6, 2021, 6:55 PM

                                More updates and bugfixes

                                telegraf config update required! Please read this in the Readme or things won't work

                                I updated the parser for the pfBlocker logs. It now uses the non-deprecated tails plugin and also fixes parsing errors that prevented data from being inserted into the influxdb.

                                1 Reply Last reply Reply Quote 0
                                • V
                                  VictorRobellini @bigjohns97
                                  last edited by Mar 6, 2021, 6:59 PM

                                  @bigjohns97 said in Grafana Dashboard using Telegraf with additional plugins:

                                  The extra panels are great, but it's not something I use and since I don't know how to properly partition and rollup influx data, I haven't implemented the unbound data. If influx had an automatic rollup like RRD, I would absolutely include it. I'm just being mindful of database growth.

                                  1 Reply Last reply Reply Quote 1
                                  • V
                                    VictorRobellini @bigjohns97
                                    last edited by Mar 6, 2021, 7:40 PM

                                    @bigjohns97

                                    I went back to poke around the unbound plugin, it seems to be working fine without the wrapper. I read through the plugin docs to find a use case for collecting the data and showing metrics, but there's just soo much data and I can't think of a use that would justify the additional collection and overhead. Here's your panel in template format and additional telegraf config:

                                    Telegraf config

                                    [[inputs.unbound]]
                                        server = "127.0.0.1:953"
                                        binary = "/usr/local/sbin/unbound-control"
                                        config_file = "/var/unbound/unbound.conf"
                                        timeout = "1s"
                                        thread_as_tag = true
                                    

                                    Grafana 7 graph

                                    {
                                      "aliasColors": {
                                        "Hits": "#629e51",
                                        "Misses": "#bf1b00"
                                      },
                                      "breakPoint": "50%",
                                      "cacheTimeout": null,
                                      "combine": {
                                        "label": "Others",
                                        "threshold": 0
                                      },
                                      "decimals": null,
                                      "fieldConfig": {
                                        "defaults": {
                                          "custom": {}
                                        },
                                        "overrides": []
                                      },
                                      "fontSize": "100%",
                                      "format": "short",
                                      "gridPos": {
                                        "h": 5,
                                        "w": 5,
                                        "x": 0,
                                        "y": 1
                                      },
                                      "hideTimeOverride": false,
                                      "id": 23763571993,
                                      "interval": null,
                                      "legend": {
                                        "header": "",
                                        "percentage": true,
                                        "percentageDecimals": 0,
                                        "show": true,
                                        "sortDesc": true,
                                        "values": false
                                      },
                                      "legendType": "On graph",
                                      "links": [],
                                      "maxDataPoints": 3,
                                      "nullPointMode": "connected",
                                      "pieType": "donut",
                                      "pluginVersion": "6.3.3",
                                      "strokeWidth": "2",
                                      "targets": [
                                        {
                                          "alias": "Hits",
                                          "groupBy": [],
                                          "measurement": "unbound",
                                          "orderByTime": "ASC",
                                          "policy": "default",
                                          "refId": "A",
                                          "resultFormat": "time_series",
                                          "select": [
                                            [
                                              {
                                                "params": [
                                                  "total_num_cachehits"
                                                ],
                                                "type": "field"
                                              }
                                            ]
                                          ],
                                          "tags": [
                                            {
                                              "key": "host",
                                              "operator": "=~",
                                              "value": "/^$Host$/"
                                            }
                                          ]
                                        },
                                        {
                                          "alias": "Misses",
                                          "groupBy": [],
                                          "measurement": "unbound",
                                          "orderByTime": "ASC",
                                          "policy": "default",
                                          "refId": "B",
                                          "resultFormat": "time_series",
                                          "select": [
                                            [
                                              {
                                                "params": [
                                                  "total_num_cachemiss"
                                                ],
                                                "type": "field"
                                              }
                                            ]
                                          ],
                                          "tags": [
                                            {
                                              "key": "host",
                                              "operator": "=~",
                                              "value": "/^$Host$/"
                                            }
                                          ]
                                        }
                                      ],
                                      "thresholds": [],
                                      "timeFrom": null,
                                      "timeShift": null,
                                      "title": "DNS Cache Hit/Miss Ratio",
                                      "type": "grafana-piechart-panel",
                                      "valueName": "current",
                                      "datasource": null
                                    }
                                    
                                    1 Reply Last reply Reply Quote 1
                                    • V
                                      VictorRobellini @bigjohns97
                                      last edited by VictorRobellini Mar 6, 2021, 8:22 PM Mar 6, 2021, 8:21 PM

                                      @bigjohns97

                                      Here's a plugin - telegraf_unbound_lite.sh - that pulls in just the metrics associated with your graph. It uses the same structure as the unbound plugin but the names use "." rather than "_", other than that, it's a drop-in replacement.

                                      Just add it to the telegraf config under the telegraf_temperature.sh entry.

                                      B 1 Reply Last reply Mar 6, 2021, 10:04 PM Reply Quote 1
                                      • B
                                        bigjohns97 @VictorRobellini
                                        last edited by Mar 6, 2021, 10:04 PM

                                        This post is deleted!
                                        1 Reply Last reply Reply Quote 0
                                        • D
                                          Doboy @VictorRobellini
                                          last edited by Doboy Apr 3, 2021, 1:50 PM Apr 3, 2021, 1:50 PM

                                          @victorrobellini Any idea why I still have this error when trying to execute the gateways script from the telegraf conf file?

                                          [inputs.exec] Error in plugin: exec: fork/exec /usr/local/bin/gateways.py: no such file or directory for command '/usr/local/bin/gateways.py':

                                          I have checked the permissions on the script file and it's set to 0555, no weird characters in the script and when I run the file from CLI I get proper behavior.

                                          [2.4.5-RELEASE][admin@******]/root: /usr/local/bin/python3.7 /usr/local/bin/gateways.py
                                          gateways,gateway_name=WAN_DHCP rtt=769.0,rttsd=562.0,loss=0i
                                          
                                          

                                          I'm on pfsense 2.4.5 and latest telegraf, rest of settings in telegraf working fine.

                                          D 1 Reply Last reply Apr 5, 2021, 10:55 AM Reply Quote 0
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                            This community forum collects and processes your personal information.
                                            consent.not_received