<![CDATA[6100-upgrade to 26.03 and can no longer log in via GUI or SSH]]>The router is still routing correctly.

The webgui login page loads correctly, once I enter my login credentials, the screen blinks and goes back to empty fields.

When I try to SSH, I get the following:

login as: admin
Keyboard-interactive authentication prompts from server:
| Password for admin@pfsense.devan.click:
End of keyboard-interactive prompts from server
Mon Apr 13 20:38:30 2026 (78571): Fatal Error Unable to create opcache lock file                                                                                   in /tmp: No space left on device (28)
*** *** *** *** *** ***

Password MUST be changed.
If the password has already been changed in the GUI, press ctrl-c to cancel.
*** *** *** *** *** ***
Mon Apr 13 20:38:30 2026 (79825): Fatal Error Unable to create opcache lock file                                                                                   in /tmp: No space left on device (28)
Mon Apr 13 20:38:30 2026 (80197): Fatal Error Unable to create opcache lock file                                                                                   in /tmp: No space left on device (28)

My other site has not upgraded and has no issues. Not sure if it's hardware failure or upgrade related.

Any tips appreciated. I haven't tried anything yet.

Thanks,

Devan

]]>https://forum.netgate.com/topic/200524/6100-upgrade-to-26.03-and-can-no-longer-log-in-via-gui-or-sshRSS for NodeSun, 16 Aug 2026 11:21:49 GMTTue, 14 Apr 2026 00:45:56 GMT60<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Sat, 15 Aug 2026 12:21:39 GMT]]>Hi, sorry for the slow reply.

Yes, that is possible - I did not reboot inbetween. It was easier to get rid of Telegraf, as it was not in use anyway.

]]>https://forum.netgate.com/post/1246782https://forum.netgate.com/post/1246782Sat, 15 Aug 2026 12:21:39 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 04 Aug 2026 20:51:56 GMT]]>Hmm, that's odd. I wouldn't expect anything to be running with it disabled. Perhaps something had failed to stop? Had it been rebooted?

]]>https://forum.netgate.com/post/1246088https://forum.netgate.com/post/1246088Tue, 04 Aug 2026 20:51:56 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 04 Aug 2026 19:57:56 GMT]]>It seems that disabling Telegraf and stopping the service was not enough to get the directory creation to stop - I had to also remove the Telegraf package.

]]>https://forum.netgate.com/post/1246086https://forum.netgate.com/post/1246086Tue, 04 Aug 2026 19:57:56 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 04 Aug 2026 06:16:42 GMT]]>Incorrectly, it seems..

We have used Telegraf previously, but on two of our firewalls, this was not disabled when the server was shut off.

This means: Telegraf was installed and enabled, but there would be not be any reply from the server/IP address associated with "InfluxDB server".

On one of the firewalls, we upgraded to 26.03.1 - with ensuing problems as above (affecting the GUI, not SSH logins).

On the other firewall, version 26.03 was still installed, no problems with login there - even though there are many(*) gosnowflake directories.

(*) Count is not easy to get, since there are so many files. There were far above 500'000 files, though.

I'm going to skip my Telegraf config, since the linked issues show the root cause is upstream. There are extra items there.

]]>https://forum.netgate.com/post/1246032https://forum.netgate.com/post/1246032Tue, 04 Aug 2026 06:16:42 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Mon, 03 Aug 2026 17:32:04 GMT]]>For reference:
https://github.com/influxdata/telegraf/issues/19072
https://github.com/snowflakedb/gosnowflake/issues/1800

]]>https://forum.netgate.com/post/1246013https://forum.netgate.com/post/1246013Mon, 03 Aug 2026 17:32:04 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Mon, 03 Aug 2026 11:00:42 GMT]]>How do you have telegraf cofigured?

]]>https://forum.netgate.com/post/1245986https://forum.netgate.com/post/1245986Mon, 03 Aug 2026 11:00:42 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Mon, 03 Aug 2026 10:26:47 GMT]]>We were experiencing the same problem on a Netgate 4100 running 26.03.1-RELEASE (amd64).

Fixed it by logging in via ssh, running the dtrace commands above to verify that it was indeed the telegraf service that was causing the issue, then using ps and kill to kill the service. Then manually running rm -rf /tmp/gosnowflake-cgo9* while stepping through all the numbers to get small enough amount of directories each time. Solved the login problem, but not the underlying issue.

]]>https://forum.netgate.com/post/1245983https://forum.netgate.com/post/1245983Mon, 03 Aug 2026 10:26:47 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Thu, 23 Apr 2026 12:35:07 GMT]]>@stephenw10

correct.

]]>https://forum.netgate.com/post/1241734https://forum.netgate.com/post/1241734Thu, 23 Apr 2026 12:35:07 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Thu, 23 Apr 2026 01:57:10 GMT]]>Hmm, so nothing to the snowflake cloud specifically?

]]>https://forum.netgate.com/post/1241722https://forum.netgate.com/post/1241722Thu, 23 Apr 2026 01:57:10 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Wed, 22 Apr 2026 14:14:08 GMT]]>@stephenw10

I was using telegraf to send discrete data to influxdb to be graphed into grafana. I also was using it as an alert if my wan failback to LTE activated.

]]>https://forum.netgate.com/post/1241690https://forum.netgate.com/post/1241690Wed, 22 Apr 2026 14:14:08 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Wed, 22 Apr 2026 07:44:46 GMT]]>Ooo interesting. Do you have anything custom configured there that might have been causing it?

]]>https://forum.netgate.com/post/1241676https://forum.netgate.com/post/1241676Wed, 22 Apr 2026 07:44:46 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 23:56:17 GMT]]>@SteveITS

Thank you

]]>https://forum.netgate.com/post/1241669https://forum.netgate.com/post/1241669Tue, 21 Apr 2026 23:56:17 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 23:55:36 GMT]]>@stephenw10

Thank you for your help.

]]>https://forum.netgate.com/post/1241668https://forum.netgate.com/post/1241668Tue, 21 Apr 2026 23:55:36 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 23:42:57 GMT]]>@stephenw10 said in 6100-upgrade to 26.03 and can no longer log in via GUI or SSH:

dtrace -n 'fbt::kern_mkdirat:entry /strstr(copyinstr(arg2), "gosnowflake") != 0/ { printf("%s (pid %d) created %s", execname, pid, copyinstr(arg2)); }'

Awesome.

0 67432 kern_mkdirat:entry telegraf (pid 34233) created /tmp/gosnowflake-cgo3048104734

Seems telegraf is the issue. This only happens on 26.03.

I am disabling the service for now.

]]>https://forum.netgate.com/post/1241667https://forum.netgate.com/post/1241667Tue, 21 Apr 2026 23:42:57 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 19:02:44 GMT]]>Further suggestion is to run:

dtrace -n 'fbt::kern_mkdirat:entry /strstr(copyinstr(arg2), "gosnowflake") != 0/ { printf("%s (pid %d) created %s", execname, pid, copyinstr(arg2)); }'

That should show whats creating then straight away if it's doing so that quickly.

]]>https://forum.netgate.com/post/1241650https://forum.netgate.com/post/1241650Tue, 21 Apr 2026 19:02:44 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 18:53:53 GMT]]>@stephenw10

I don't have anything else installed. Everything is from the pfsense package manager.

]]>https://forum.netgate.com/post/1241649https://forum.netgate.com/post/1241649Tue, 21 Apr 2026 18:53:53 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 18:49:02 GMT]]>Mmm, it looks likely to be this but AFAIK we don't ship it: https://docs.snowflake.com/en/developer-guide/golang/go-driver

Do you have any 3rd party packages installed?

Other suggestions from our devs here are to try using fstat or install lsof and see what's holding the files.

]]>https://forum.netgate.com/post/1241647https://forum.netgate.com/post/1241647Tue, 21 Apr 2026 18:49:02 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 18:45:11 GMT]]>@stephenw10

Before I restored to 25.11.1 I tried ps -aux | grep snow but didn't catch anything. I didn't use the -i caps switch. my bad.

I'm in 25.11.1 and don't see any tmp directory creation. Only been up for 20 minutes. All same services running.

I'll upgrade back to 26.03 tomorrow.

]]>https://forum.netgate.com/post/1241646https://forum.netgate.com/post/1241646Tue, 21 Apr 2026 18:45:11 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 18:39:05 GMT]]>If you're able to test again try to see what's creating them using something like: ps -auxwwd | grep snowflake

If it's happening almost continually you should be able to catch it.

]]>https://forum.netgate.com/post/1241645https://forum.netgate.com/post/1241645Tue, 21 Apr 2026 18:39:05 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 18:36:09 GMT]]>And definitely not happening in 25.11.1 with the same packages/services enabled?

]]>https://forum.netgate.com/post/1241644https://forum.netgate.com/post/1241644Tue, 21 Apr 2026 18:36:09 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 17:36:57 GMT]]>Troubleshooting steps:

killed watchdog
stopped OpenVPN
stopped frr mgmt.
stopped haproxy
stopped dpinger
stopped arpwatch
stopped radvd
stopped dhcpd
stopped ntpd
stopped syslogd
stopped udpbroadcastrelay
stopped unbound
stopped wireguard

unbound, dhcpd, radvd, dpinger all restarted by themselves.

Router is still creating gosnowflake directores 2 per second.

Every other service is off.

Rolling back to 25.11.1...

]]>https://forum.netgate.com/post/1241640https://forum.netgate.com/post/1241640Tue, 21 Apr 2026 17:36:57 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 17:05:14 GMT]]>@stephenw10

nope

]]>https://forum.netgate.com/post/1241638https://forum.netgate.com/post/1241638Tue, 21 Apr 2026 17:05:14 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 17:04:35 GMT]]>You have Nexus enabled?

]]>https://forum.netgate.com/post/1241637https://forum.netgate.com/post/1241637Tue, 21 Apr 2026 17:04:35 GMT<![CDATA[Reply to 6100-upgrade to 26.03 and can no longer log in via GUI or SSH on Tue, 21 Apr 2026 17:03:38 GMT]]>It's creating two directories every second.

What process spawns these directories?

]]>https://forum.netgate.com/post/1241636https://forum.netgate.com/post/1241636Tue, 21 Apr 2026 17:03:38 GMT