Telegraf configuration for influxdb 2

hispeed · Jun 1, 2024, 7:36 PM

Hi

I have installed Telegraf and want to send to influxdb v.2.x. Unfortunately this is not an easy task.

I vote for this:
https://redmine.pfsense.org/issues/12711?tab=history
So i hope this will be in in the next release. Maybe someone from Netgear looks at this, I have just bought an 8200 ;).

The main problem is that the configuration is always overwritten. It should be possible to click a checkbox don't overwrite the config. And then you have the possibility to do an own config. At the moment it's not usable this plugin in my opinion at least when you need to configure something.

hispeed · Jun 10, 2024, 7:45 PM

Hi @jimp
Maybe you can help and give redmine: 12711 a bump :).
It would be great to make there a bigger improvement which can be done for a good developer very fast and easy.

I'm sorry that I used the @ and disturbed you and it's probably not your construction site or business.

This can be closed.

hispeed · Jun 23, 2024, 3:39 PM

I got it now "working" but telegraf unfortunately is not stable and stops working randomly. Until you get there you probably have tried a lot like me.

I use this additional configuration. You can use that but there is a problem with: ^N (LF) Unix and the webinterface. If you open the configuration in "vi" you can see it.

[[inputs.exec]]
    commands = [
        "/usr/local/bin/telegraf_pfifgw.php",
        "sh /usr/local/bin/telegraf_temperature.sh"
    ]
    data_format = "influx"

[[inputs.tail]]
    files = ["/var/log/pfblockerng/dnsbl.log"]
    data_format = "grok"
    from_beginning = false
    name_suffix = "_dnsbl_log"
    grok_timezone = "Local"
    grok_patterns = ["^%{WORD:blocktype}-%{WORD:blocksubtype},%{SYSLOGTIMESTAMP:timestamp:ts-syslog},%{IPORHOST:domain},%{IPORHOST:src_ip:tag},%{GREEDYDATA:req_agent},%{WORD:blockmethod},%{WORD:blocklist:tag},%{IPORHOST:tld:tag},%{WORD:feed_name:tag},%{GREEDYDATA:duplicateeventstatus}"]

[[inputs.tail]]
    files = ["/var/log/pfblockerng/ip_block.log"]
    data_format = "grok"
    from_beginning = false
    name_suffix = "_ip_block_log"
    grok_timezone = "Local"
    grok_patterns = ["^%{SYSLOGTIMESTAMP:timestamp:ts-syslog},%{NUMBER:rulenum},%{DATA:interface},%{WORD:friendlyname},%{WORD:action},%{NUMBER:ip_version},%{NUMBER:protocolid},%{DATA:protocol:tag},%{IPORHOST:src_ip:tag},%{IPORHOST:dest_ip:tag},%{WORD:src_port:tag},%{NUMBER:dest_port:tag},%{WORD:direction},%{WORD:geoip_code:tag},%{DATA:ip_alias_name},%{DATA:ip_evaluated},%{DATA:feed_name:tag},%{HOSTNAME:resolvedhostname},%{GREEDYDATA:clienthostname},%{GREEDYDATA:ASN},%{GREEDYDATA:duplicateeventstatus}"]

[[outputs.influxdb_v2]]
   urls = ["http://IPADRESS:8086"]
   token = "YOURTOKEN"
   organization = "NAME_OF_ORGANIZATION"
   bucket = "NAME_OF_BUCKET"

Second problem if you only have a Influx DB V.2.0 you're temped to remove the login credentials for V.1.0 which is not a good idea because you recieve another error:

2024-06-16T11:59:39Z E! [agent] Failed to connect to [outputs.influxdb], retrying in 15s, error was "unsupported scheme [\"\"]: \"\""
2024-06-16T11:59:54Z E! [telegraf] Error running agent: connecting output outputs.influxdb: error connecting to output "outputs.influxdb": unsupported scheme [""]: ""

If everything is correct then you still have the problem with random stops of Telegraf. I use the stable branch and no develop.
I always recieve this in my debug log from Telegraf and then Telegraf needs a restart:

2024-06-23T14:46:44Z E! [outputs.influxdb] E! [outputs.influxdb] Failed to write metric (will be dropped: 401 Unauthorized):
2024-06-23T14:46:44Z D! [outputs.influxdb] Wrote batch of 156 metrics in 6.772392ms
2024-06-23T14:46:44Z D! [outputs.influxdb] Buffer fullness: 1 / 10000 metrics
2024-06-23T14:46:45Z D! [outputs.influxdb_v2] Wrote batch of 156 metrics in 291.641419ms
2024-06-23T14:46:45Z D! [outputs.influxdb_v2] Buffer fullness: 2 / 10000 metrics
2024-06-23T14:46:54Z E! [outputs.influxdb] E! [outputs.influxdb] Failed to write metric (will be dropped: 401 Unauthorized):
2024-06-23T14:46:54Z D! [outputs.influxdb] Wrote batch of 147 metrics in 4.850251ms
2024-06-23T14:46:54Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2024-06-23T14:46:54Z D! [outputs.influxdb_v2] Wrote batch of 147 metrics in 41.016964ms
2024-06-23T14:46:54Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2024-06-23T14:47:01Z D! [agent] Stopping service inputs
2024-06-23T14:47:01Z D! [inputs.tail] Recording offset 3289668 for "/var/log/pfblockerng/dnsbl.log"
2024-06-23T14:47:01Z D! [inputs.tail] Tail removed for "/var/log/pfblockerng/dnsbl.log"
2024-06-23T14:47:01Z D! [inputs.tail] Recording offset 189121 for "/var/log/pfblockerng/ip_block.log"
2024-06-23T14:47:01Z D! [inputs.tail] Tail removed for "/var/log/pfblockerng/ip_block.log"
2024-06-23T14:47:01Z D! [agent] Input channel closed
2024-06-23T14:47:01Z I! [agent] Hang on, flushing any cached metrics before shutdown
2024-06-23T14:47:01Z E! [outputs.influxdb] E! [outputs.influxdb] Failed to write metric (will be dropped: 401 Unauthorized):
2024-06-23T14:47:01Z D! [outputs.influxdb] Wrote batch of 117 metrics in 3.136646ms
2024-06-23T14:47:01Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2024-06-23T14:47:01Z D! [outputs.influxdb_v2] Wrote batch of 117 metrics in 22.786287ms
2024-06-23T14:47:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2024-06-23T14:47:01Z I! [agent] Stopping running outputs
2024-06-23T14:47:01Z D! [agent] Stopped Successfully

This error can happen after an hour or 30 Minutes or 3 hours. Maybe it has something to do with the files and PfblockerNG concerning read/write permissions.
I also always see that the cpu is rising when it stops.

Also I have this error in the log because I have Influx V.1.0 configured but i'm not using it.

E! [outputs.influxdb] E! [outputs.influxdb] Failed to write metric (will be dropped: 401 Unauthorized):