Telegraf stopped working after update to 2.7
-
Hi, until a couple of weeks ago i was happily using Telegraf on 2.6 to send data to an influxdb. On upgrading to 2.7 i get the following crash report
Crash report begins. Anonymous machine information:
amd64
14.0-CURRENT
FreeBSD 14.0-CURRENT #1 RELENG_2_7_0-n255866-686c8d3c1f0: Wed Jun 28 04:21:19 UTC 2023 root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_7_0-main/obj/amd64/LwYAddCr/var/jenkins/workspace/pfSense-CE-snapshots-2_7_0-main/sources/FreeBSD-src-RELCrash report details:
PHP Errors:
[27-Sep-2023 23:32:36 Europe/London] PHP Fatal error: Uncaught TypeError: implode(): Argument #1 ($pieces) must be of type array, string given in /usr/local/pkg/telegraf.inc:132
Stack trace:
#0 /usr/local/pkg/telegraf.inc(132): implode(',', NULL)
#1 /etc/inc/pkg-utils.inc(775) : eval()'d code(1): telegraf_resync_config()
#2 /etc/inc/pkg-utils.inc(775): eval()
#3 /etc/inc/pkg-utils.inc(899): eval_once(' telegra...')
#4 /etc/rc.packages(76): install_package_xml('Telegraf')
#5 {main}
thrown in /usr/local/pkg/telegraf.inc on line 132
[27-Sep-2023 23:32:45 Europe/London] PHP Fatal error: Uncaught TypeError: implode(): Argument #1 ($pieces) must be of type array, string given in /usr/local/pkg/telegraf.inc:132
Stack trace:
#0 /usr/local/pkg/telegraf.inc(132): implode(',', NULL)
#1 /etc/inc/pkg-utils.inc(709) : eval()'d code(1): telegraf_resync_config()
#2 /etc/inc/pkg-utils.inc(709): eval()
#3 /etc/rc.start_packages(66): sync_package('Telegraf')
#4 {main}
thrown in /usr/local/pkg/telegraf.inc on line 132No FreeBSD crash data found.
I saw some posts back in April on a similar topic but they didnt really go anywhere. i tried uninstalling and reinstalling the plugin but thats about the extent of my expertise here.
Anyone, any ideas? Cheers
-
From the PHP errors generated, it appears nobody has updated the Telgraf package for PHP 8.1 and higher. Or if they did, they missed some areas of the code that are still incompatible with the PHP 8.1 and higher language version used in pfSense 2.7 and up.
There is nothing you can do on your end unless you manually edit/rewrite the PHP source code files for the package. A true fix will take action on the part of the pfSense package creator/maintainer.
-
@bmeeks thanks! oh well, thats unfortunate - although does that mean i'm the only pfsense user using Telegraf on 2.7?
definitely no possibility of me fixing it - i will wait.
cheers!
-
@spaceboy I also have issues with the latest update of 2.7 well actually pfsense plus 23.05.01 which is 2.7 ce.
I had mine running fine and then I had to do a reinstall. Doing that broke everything. I know that I had pfsense running version after version going up.
-
@spaceboy please open a redmine with your php output..
-
@vasquezmi said in Telegraf stopped working after update to 2.7:
I had mine running fine and then I had to do a reinstall. Doing that broke everything. I know that I had pfsense running version after version going up.
pfSense 2.7.0 CE and pfSense Plus 23.01 made a big jump with the PHP language package from version 7.x to 8.1. The PHP upstream developer team made many alterations to the way PHP behaves. Chief among those were that it stopped automatically fixing things invisibly using its "best guest" of what the programmer may have intended. This involved automatic conversions between strings and numbers. It also started enforcing some formerly optional rules about how arrays were defined and accessed. What were once warnings that were suppressed unless optionally enabled became hard-stop errors with 8.x of PHP.
PHP is what the entire web GUI on pfSense is written in. So, that meant a ton of pfSense PHP code had to be changed. Little tidbits are still showing up in the pfSense base that need fixing. The third-party packages have issues as well. Some have been fixed, but certainly not all. It depends on whether the package has a currently active maintainer or not. If the maintainer is not highly involved and staying on top of the PHP changes, then his package will have problems on the newest pfSense versions.
-
@michmoor done! cheers
-
hello - sorry to say that this issue still persists.
i logged the issue with redmine https://redmine.pfsense.org/issues/14861 and as far as i can see i was told it was not reproducible. in the meantime i have updated to 2.7.2 but nothing has changed. Telegraf does not work and i still get the telegraf errors as previously reported.
in the meantime the following has also happened:
- the hard disk in my router died and when i restored a backup that included my Telegraf config, i could at least see the option to access it in the menu and the status of the service (stopped, would not start). i still got the PHP errors
- if i remove the Telegraf plugin the errors stop, but on reinstalling the plugin, i'm back to not being able to access the Telegraf config.
- i'm also getting other errors with the Telegraf plugin installed. randomly HaProxy and Snort services stop and HaProxy doesnt start automatically on boot, but does manually.
I'm honestly at the point of a clean start to see if i can get Telgraf working, because I'm already at the limit of my own knowledge. or does this seem recoverable? cheers
-
I want to report that I'm having the same issue with 23.09.1 on the 7100-1U. It's happening on multiple devices, and I think it's related to gateway alarms, but I can't say for sure.
The only fix for me at the time is service watchdog package.
-
Let me add some details ... I have two firewalls, both CE 2.7.2 but they have different built dates.
vault
zotac
On vault, I cannot install telegraf, it throws the php error, line 132 during pkg install.
On zotac, I CAN install telegraf, configure it to ping 8.8.8.8 and 1.1.1.1, it writes to the database and I can see the dashboard in grafana. But ... if I change telegraf settings and add at the custom box at the bottom
[[inputs.ping]]
urls = ["8.8.8.8","1.1.1.1"]
count = 5telegraf blows up with the php error 132
The different built dates are confusing, why are there two of them?
Both firewalls run the watchdog service to monitor unbound and restart it.
-
@spaceboy I experienced the same issue. I researched several articles to solve it and finally managed to fix it. I have summarized the solution on my blog. If you haven't solved it yet, I hope this helps!
-
@Vulcan_ thank you for the detailed analysis.
The question remains, why different built dates? 2.7.2 stable should have a frozen repo. All of us who d/l pfsense CE 2.7.2 should have the same built date.
Second, your analysis is very clear, how come the Developer on redmine claims he cannot reproduce the issue?
-
@pavlos said in Telegraf stopped working after update to 2.7:
The question remains, why different built dates? 2.7.2 stable should have a frozen repo. All of us who d/l pfsense CE 2.7.2 should have the same built date.
Very interesting. I stumbled upon this post looking for something else. Well, my build date is different from your two:
2.7.2-RELEASE (amd64)
built on Wed Dec 6 15:10:00 EST 2023
FreeBSD 14.0-CURRENTI wonder what's up with that.
