Grafana Dashboard using Telegraf with additional plugins

erbalo · Sep 9, 2020, 6:49 PM

@bigjohns97 Just successfully enabled the logs in telegraf and what i am now seeing is lot of entry's in the logs:

D! Grok no match found for: "May 25 17:34:24 ...................

It points to my WAN gateway and WAN interface.

What should be the next too look up?

erbalo · Sep 23, 2020, 2:42 PM

This post is deleted!

erbalo · Sep 23, 2020, 7:19 PM

How easy is change the show the speeds as MBPS instead of MB/s?

This is which i am seeing on your original Git page:

and this is what i want(downloaded from grafana.com from your dashboard):

VictorRobellini · Sep 24, 2020, 1:29 AM

@erbalo So you want the big graph to be bits per second and the smaller one to be bytes per second? If so, it should be really easy to do. Just look at the Queries and Visualizations when you edit the graph.

erbalo · Sep 25, 2020, 1:11 PM

@VictorRobellini said in Grafana Dashboard using Telegraf with additional plugins:

@erbalo So you want the big graph to be bits per second and the smaller one to be bytes per second? If so, it should be really easy to do. Just look at the Queries and Visualizations when you edit the graph.

Thanks , it solved now. I have added *8 after the bytes)

Another question:
When i look this graph, it is not really my montly WAN download and upload. It updates well and working but is not correct. What is going wrong here?

SELECT sum("bytes_recv") / 1000000 FROM "net" WHERE ("host" =~ /^MYROUTERNAME.MYROUTERDOMAIN$/ AND "interface" =~ /^ipx0$/) AND time >= 1598911200000ms GROUP BY time(30m) fill(previous)

cburbs · Sep 25, 2020, 6:38 AM

Did you remove the below from your Dashboard upload file? If so how do we turn it back on?

erbalo · Sep 25, 2020, 6:38 AM

@cburbs said in Grafana Dashboard using Telegraf with additional plugins:

Did you remove the below from your Dashboard upload file? If so how do we turn it back on?

load the old JSON dashbord as new and copy that panel and paste on your new Dashboard.

VictorRobellini · Sep 25, 2020, 1:08 PM

@cburbs It still appears on my dashboard. I'll do another sync this weekend.

VictorRobellini · Sep 25, 2020, 1:11 PM

@erbalo If the question is "Why doesn't the total data transferred for the last 30 days or month not match what my ISP is showing as my total transferred for the last 30 days or month", I'm not sure. How far off are the numbers? It could either be the query and calculation happening in Grafana or a question for the broader pfSense forums.

cburbs · Sep 25, 2020, 8:04 PM

@erbalo That worked if I copied the code from here - https://github.com/VictorRobellini/pfSense-Dashboard

I originally pulled it from here though - https://grafana.com/grafana/dashboards/12023

Does the one at Grafana not get updated?

VictorRobellini · Sep 26, 2020, 1:50 AM

@cburbs

It would be too much of a PITA to update both. Plus GitHub is a much better place to host the project since there are multiple readmes, additional files, bugs, and merge/pull options. Formatting things in the right way isn't possible on Grafana's site.

Which is why the very first line of my Grafana post reads:

For the latest version, visit: https://github.com/VictorRobellini/pfSense-Dashboard

cburbs · Sep 26, 2020, 3:58 AM

@VictorRobellini Totally understand and thanks for the heads up.

_rp · Oct 6, 2020, 4:32 PM

@bigjohns97 said in Grafana Dashboard using Telegraf with additional plugins:

Any idea why I would be getting the following when trying the telegraf_gateways.py option?

[inputs.exec] Error in plugin: exec: fork/exec /usr/local/bin/telegraf_gateways.py: no such file or directory for command '/usr/local/bin/telegraf_gateways.py'

I'm having the same issues, what did you do to fix this?

I am using the fastjack version from here https://gist.github.com/fastjack/a0feb792a9655da7aa3e2a7a1d9f812f

Crimson11 · Oct 27, 2020, 7:32 PM

Thank you very much for your work.

I had to install Grafana & Influxdb on Windows 2019. :|

Everything is working fine except Active users / Uptime / Interface Summary. It says no data.

I downloaded the files directly on Pfsense.

The data is being populated on Influxdb, though I could not make the telegraf.log to work.

Any idea on what am I missing?

Thank you.

VictorRobellini · Oct 27, 2020, 8:23 PM

If you are seeing the "Active users / Uptime / Interface Summary" in the influx DB but not on the graph, it's possible that there's an issue with the query and I don't think the telegraf.log is going to help. It could be a result of a bad dashboard import or a newer versions of Grafana.

You will want to specifically see what gets returned by the below queries:

select * from system limit 5
select * from net limit 5

Crimson11 · Oct 28, 2020, 9:58 AM

@VictorRobellini Thank you very much for the fast reply. I agree that it could be relative to a bad dashboard import.

For the uptime panel I was able to import it from another dashboard and compared to yours and it was the same but the imported one was working.

I was able to fix interface summary by importing directly from the Grafana website and copy/paste the panel.

Active users was not working either on the other dashboard I imported.

These gives data but I did not had time to format it.
select * from system limit 5
select * from net limit 5

Thank you again.

JohnKap · Nov 3, 2020, 7:45 AM

@romprod said in Grafana Dashboard using Telegraf with additional plugins:

@bigjohns97 said in Grafana Dashboard using Telegraf with additional plugins:

Any idea why I would be getting the following when trying the telegraf_gateways.py option?

[inputs.exec] Error in plugin: exec: fork/exec /usr/local/bin/telegraf_gateways.py: no such file or directory for command '/usr/local/bin/telegraf_gateways.py'

I'm having the same issues, what did you do to fix this?

I am using the fastjack version from here https://gist.github.com/fastjack/a0feb792a9655da7aa3e2a7a1d9f812f

after copying the file to this folder, did you chmod +x it?

pfsense99 · Nov 18, 2020, 12:34 AM

For the unbound - on telegrah - looks like does not work with the latest version of pfsense and telegrah

It used to work..

In telegraf config..

[[inputs.unbound]]
server = "127.0.0.1:953"
binary = "/usr/local/bin/telegraf_unbound.sh"

Command to check:

/usr/local/bin/telegraf -config=/usr/local/etc/telegraf.conf --test --input-filter unbound

Output:
2020-11-18T00:31:41Z I! Starting Telegraf 1.13.4
2020-11-18T00:31:45Z E! [inputs.unbound] Error in plugin: error gathering metrics: error running unbound-control: Command timed out. (/usr/local/bin/telegraf_unbound.sh [-s 127.0.0.1@953 stats_noreset])
2020-11-18T00:31:45Z E! [telegraf] Error running agent: One or more input plugins had an error

bigjohns97 · Dec 2, 2020, 2:11 PM

Recently tried out the new version of PfBlockerNG 3.0.0_1 and was looking to test the new python integration with unbound that produces the client ip in the logs when using null blocking. (requires pfsense 2.5.x and latest version of unbound)

Wanted to confirm that not only does it work but everything still shows up in this grafana dashboard.

iso667 · Dec 14, 2020, 4:37 PM

@erbalo I am facing this similar issue, when I see the telegraf log I have this message everywhere... Did you find any solution?

For example: