6100-upgrade to 26.03 and can no longer log in via GUI or SSH
-
If you're able to test again try to see what's creating them using something like:
ps -auxwwd | grep snowflakeIf it's happening almost continually you should be able to catch it.
-
Before I restored to 25.11.1 I tried ps -aux | grep snow but didn't catch anything. I didn't use the -i caps switch. my bad.
I'm in 25.11.1 and don't see any tmp directory creation. Only been up for 20 minutes. All same services running.
I'll upgrade back to 26.03 tomorrow.
-
Mmm, it looks likely to be this but AFAIK we don't ship it: https://docs.snowflake.com/en/developer-guide/golang/go-driver
Do you have any 3rd party packages installed?
Other suggestions from our devs here are to try using fstat or install lsof and see what's holding the files.
-
I don't have anything else installed. Everything is from the pfsense package manager.
-
Further suggestion is to run:
dtrace -n 'fbt::kern_mkdirat:entry /strstr(copyinstr(arg2), "gosnowflake") != 0/ { printf("%s (pid %d) created %s", execname, pid, copyinstr(arg2)); }'That should show whats creating then straight away if it's doing so that quickly.
-
@stephenw10 said in 6100-upgrade to 26.03 and can no longer log in via GUI or SSH:
dtrace -n 'fbt::kern_mkdirat:entry /strstr(copyinstr(arg2), "gosnowflake") != 0/ { printf("%s (pid %d) created %s", execname, pid, copyinstr(arg2)); }'
Awesome.
0 67432 kern_mkdirat:entry telegraf (pid 34233) created /tmp/gosnowflake-cgo3048104734
Seems telegraf is the issue. This only happens on 26.03.
I am disabling the service for now.
-
Thank you for your help.
-
Thank you
-
Ooo interesting. Do you have anything custom configured there that might have been causing it?
-
I was using telegraf to send discrete data to influxdb to be graphed into grafana. I also was using it as an alert if my wan failback to LTE activated.
-
Hmm, so nothing to the snowflake cloud specifically?
-
correct.
-
We were experiencing the same problem on a Netgate 4100 running
26.03.1-RELEASE (amd64).Fixed it by logging in via ssh, running the dtrace commands above to verify that it was indeed the telegraf service that was causing the issue, then using ps and kill to kill the service. Then manually running
rm -rf /tmp/gosnowflake-cgo9*while stepping through all the numbers to get small enough amount of directories each time. Solved the login problem, but not the underlying issue. -
How do you have telegraf cofigured?
-
For reference:
https://github.com/influxdata/telegraf/issues/19072
https://github.com/snowflakedb/gosnowflake/issues/1800 -
Incorrectly, it seems..
We have used Telegraf previously, but on two of our firewalls, this was not disabled when the server was shut off.
This means: Telegraf was installed and enabled, but there would be not be any reply from the server/IP address associated with "InfluxDB server".
On one of the firewalls, we upgraded to 26.03.1 - with ensuing problems as above (affecting the GUI, not SSH logins).
On the other firewall, version 26.03 was still installed, no problems with login there - even though there are many(*)
gosnowflakedirectories.(*) Count is not easy to get, since there are so many files. There were far above 500'000 files, though.
I'm going to skip my Telegraf config, since the linked issues show the root cause is upstream. There are extra items there.
-
It seems that disabling Telegraf and stopping the service was not enough to get the directory creation to stop - I had to also remove the Telegraf package.
-
Hmm, that's odd. I wouldn't expect anything to be running with it disabled. Perhaps something had failed to stop? Had it been rebooted?
-
Hi, sorry for the slow reply.
Yes, that is possible - I did not reboot inbetween. It was easier to get rid of Telegraf, as it was not in use anyway.
Privacy Policy · Cookie Policy