Grafana Dashboard using Telegraf with additional plugins
-
Thanks for this Dashboard. I am just starting to have a play around with it and now have most things working.
My setup is IGB0.10 is my WAN and have this set in the variable
I have IGB1 and my LAN, IGB2 as my 10GB LAN, IGB1.30 as my IP Phone network and IGB1.40 as my Guest network.The WAN shows but in and out look the same on the graph but show different values.
For the LAN it only shows IGB0 and OPVNS1. Not sure where all the other LANS are or why it does not display them.This is my first look at Telegraf and InfluxDB so am a complete noob at it.
-
@veldthui I'm not sure I understand what's going on. Can you post a screenshot?
-
Attached is an image of the lower part which should show the WAN and LANs. It shows the WAN but does not look right and does not show the correct LANs
-
@veldthui Can you share how you have the interfaces defined in the grafana variable config?
-
@VictorRobellini Okay I worked out that the WAN only had one of the values multiplied by 8 to change it to bytes. Changed the other and it looks okay now.
Below is the definition which is how it was downloaded. If I add a $ to igb1, igb2, and igb3 I can then see the igb1.30 and igb1.40 networks but not igb1 and igb0.10 should not be in the list as that is the WAN.
-
@veldthui said in Grafana Dashboard using Telegraf with additional plugins:
What is your Regex?
What adapter(s) is your WAN?
What adapters to you want displayed?Please take a look at the github page. There's plenty written up in the Readme
-
The regex is as above. My WAN is on the IGB0 adapter as VLAN 10 which shows as igb0.10.
I want to display this as the WAN which it is doing. For the LAN I want to display igb1, igb1.30, igb1.40, and igb2.
igb3 and igb4 are not in use at present but may be in future. I have read the readme file but it does not explain it very well. I have only very basic understanding of regex as well -
@veldthui said in Grafana Dashboard using Telegraf with additional plugins:
The content of the "Regex" field is the same as what I have posted to github and it does not match what you are saying.
Please review the content on Github, specifically the section titled, Configuration
If the instructions are unclear, please let me know what I can add to make them better. -
Okay I think I have finally worked it out. my regex now looks like /^(?!igb0)/ to filter out everything on igb0 and leave the rest showing. Appears to have worked.
Thanks for the help. -
any chance of an idiots guide (me = dummy) to installing this lot?
Tried from your Github page and first problem was the docker-compose example threw errors. So gave up. -
@oldfart The docker-compose is just a quick way to setup influx and grafana using docker and docker-compose. There are tutorials on the internet that show how to set this up, a quick google search will turn up plenty of resources, for example: https://www.homelabrat.com/influxdb-homelab-dashboard/
-
-
@VictorRobellini Basically this link covers all needed components setup ?
-
Hi Guys,
I have built pfsense with grafana etc. However I am trying to figure it out how to setup addons,
https://forum.netgate.com/topic/152132/grafana-dashboard-using-telegraf-with-additional-plugins
"Does /usr/local/bin/python3.7 exist on your pfSense system? If so, use this telegraf_gateways-3.7.py"
How could I upload this file to pfesense? as I do not see it in pfsense file hierarchy. Thanks for help
-
"Does /usr/local/bin/python3.7 exist on your pfSense system? If so, use this telegraf_gateways-3.7.py"
The addons can be downloaded from the repo - look under plugins. https://github.com/VictorRobellini/pfSense-Dashboard
How could I upload this file to pfesense? as I do not see it in pfsense file hierarchy. Thanks for help
use an FTP program - my preference is Filezilla.
Make sure you set the permissions as per instructions - https://github.com/VictorRobellini/pfSense-Dashboard#plugins
-
Looks like this is still working under the new P1 release - just a heads up! :)
-
Hi, my dears friends, i´m searching for a visualizer or dashboard
to view live firewall activity.Does this tool or utility exist?
thanks
Grafana its like a dashboard to see Active Users
Uptime
CPU Load total
Disk Utilization
Memory Utilization
CPU Utilization per core (Single Graph)
Ram Utilization time graph
Load Average
Load Average Graph
CPU and ACPI Temperature Sensors
pfBlocker IP Stats
pfBlocker DNS Stats
Gateway Response time - dpinger
List of interfaces with IP, MAC, Status and pfSesnse labels thanks to /u/trumee
WAN Statistics - Traffic & Throughput (Identified by dashboard variable)
LAN Statistics - Traffic & Throughput (Identified by dashboard variable)
Unbound stats - Plugin and config included and working but not implemented -
I would highly suggest reading the linked github page on the initial post and then reforming your question.
-
First of all really love your tutorial and work, i have installed all of the stuff, receiving data in influxDB and connected to Grafana and added dashboard.
But,
I am not receiving data from pfblocker, i see such as dnsbl log on influxdb but when i expand that o don't see anyting. I am using pfblocker-ng Devel
On the Grafana dashboard i see only data from Disk Utlizization. On influxDB i see well CPU/Memory etc..
I have uploaded the telegraf plugins using Wincp and gived 0555 permissions using Wincscp.
What is really my problem too look up?
-
@erbalo Please review the troubleshooting section that's posted on GitHub
-
Is this correct? I did gived the permission via WinSCP.
Also i don't get the Downloads/Uploads state.
-
@erbalo You need to enable logging using the instructions on the github link above and it should point you in the right direction.
-
@bigjohns97 Just successfully enabled the logs in telegraf and what i am now seeing is lot of entry's in the logs:
D! Grok no match found for: "May 25 17:34:24 ...................
It points to my WAN gateway and WAN interface.
What should be the next too look up?
-
This post is deleted! -
How easy is change the show the speeds as MBPS instead of MB/s?
This is which i am seeing on your original Git page:
and this is what i want(downloaded from grafana.com from your dashboard):
-
@erbalo So you want the big graph to be bits per second and the smaller one to be bytes per second? If so, it should be really easy to do. Just look at the Queries and Visualizations when you edit the graph.
-
@VictorRobellini said in Grafana Dashboard using Telegraf with additional plugins:
@erbalo So you want the big graph to be bits per second and the smaller one to be bytes per second? If so, it should be really easy to do. Just look at the Queries and Visualizations when you edit the graph.
Thanks , it solved now. I have added *8 after the bytes)
Another question:
When i look this graph, it is not really my montly WAN download and upload. It updates well and working but is not correct. What is going wrong here?SELECT sum("bytes_recv") / 1000000 FROM "net" WHERE ("host" =~ /^MYROUTERNAME.MYROUTERDOMAIN$/ AND "interface" =~ /^ipx0$/) AND time >= 1598911200000ms GROUP BY time(30m) fill(previous)
-
Did you remove the below from your Dashboard upload file? If so how do we turn it back on?
-
@cburbs said in Grafana Dashboard using Telegraf with additional plugins:
Did you remove the below from your Dashboard upload file? If so how do we turn it back on?
load the old JSON dashbord as new and copy that panel and paste on your new Dashboard.
-
@cburbs It still appears on my dashboard. I'll do another sync this weekend.
-
@erbalo If the question is "Why doesn't the total data transferred for the last 30 days or month not match what my ISP is showing as my total transferred for the last 30 days or month", I'm not sure. How far off are the numbers? It could either be the query and calculation happening in Grafana or a question for the broader pfSense forums.
-
@erbalo That worked if I copied the code from here - https://github.com/VictorRobellini/pfSense-Dashboard
I originally pulled it from here though - https://grafana.com/grafana/dashboards/12023
Does the one at Grafana not get updated?
-
It would be too much of a PITA to update both. Plus GitHub is a much better place to host the project since there are multiple readmes, additional files, bugs, and merge/pull options. Formatting things in the right way isn't possible on Grafana's site.
Which is why the very first line of my Grafana post reads:
For the latest version, visit: https://github.com/VictorRobellini/pfSense-Dashboard
-
@VictorRobellini Totally understand and thanks for the heads up.
-
@bigjohns97 said in Grafana Dashboard using Telegraf with additional plugins:
Any idea why I would be getting the following when trying the telegraf_gateways.py option?
[inputs.exec] Error in plugin: exec: fork/exec /usr/local/bin/telegraf_gateways.py: no such file or directory for command '/usr/local/bin/telegraf_gateways.py'
I'm having the same issues, what did you do to fix this?
I am using the fastjack version from here https://gist.github.com/fastjack/a0feb792a9655da7aa3e2a7a1d9f812f
-
Thank you very much for your work.
I had to install Grafana & Influxdb on Windows 2019. :|
Everything is working fine except Active users / Uptime / Interface Summary. It says no data.
I downloaded the files directly on Pfsense.
The data is being populated on Influxdb, though I could not make the telegraf.log to work.
Any idea on what am I missing?
Thank you.
-
If you are seeing the "Active users / Uptime / Interface Summary" in the influx DB but not on the graph, it's possible that there's an issue with the query and I don't think the telegraf.log is going to help. It could be a result of a bad dashboard import or a newer versions of Grafana.
You will want to specifically see what gets returned by the below queries:
select * from system limit 5 select * from net limit 5
-
@VictorRobellini Thank you very much for the fast reply. I agree that it could be relative to a bad dashboard import.
For the uptime panel I was able to import it from another dashboard and compared to yours and it was the same but the imported one was working.
I was able to fix interface summary by importing directly from the Grafana website and copy/paste the panel.
Active users was not working either on the other dashboard I imported.
These gives data but I did not had time to format it.
select * from system limit 5
select * from net limit 5Thank you again.
-
@romprod said in Grafana Dashboard using Telegraf with additional plugins:
@bigjohns97 said in Grafana Dashboard using Telegraf with additional plugins:
Any idea why I would be getting the following when trying the telegraf_gateways.py option?
[inputs.exec] Error in plugin: exec: fork/exec /usr/local/bin/telegraf_gateways.py: no such file or directory for command '/usr/local/bin/telegraf_gateways.py'
I'm having the same issues, what did you do to fix this?
I am using the fastjack version from here https://gist.github.com/fastjack/a0feb792a9655da7aa3e2a7a1d9f812f
after copying the file to this folder, did you chmod +x it?
-
For the unbound - on telegrah - looks like does not work with the latest version of pfsense and telegrah
It used to work..
In telegraf config..
[[inputs.unbound]]
server = "127.0.0.1:953"
binary = "/usr/local/bin/telegraf_unbound.sh"Command to check:
/usr/local/bin/telegraf -config=/usr/local/etc/telegraf.conf --test --input-filter unbound
Output:
2020-11-18T00:31:41Z I! Starting Telegraf 1.13.4
2020-11-18T00:31:45Z E! [inputs.unbound] Error in plugin: error gathering metrics: error running unbound-control: Command timed out. (/usr/local/bin/telegraf_unbound.sh [-s 127.0.0.1@953 stats_noreset])
2020-11-18T00:31:45Z E! [telegraf] Error running agent: One or more input plugins had an error