Tailscale service won't start, after upgrading to 23.01 on SG-3100
-
I upgraded my SG-3100 to 23.01 yesterday, and the Tailscale service is reported as stopped, and trying to manually start the service hasn't worked. I have rebooted, uninstalled, and then reinstalled the package but not change. Looking at the system logs it does appear a file is missing: "tun".
Anyone seeing a similar issue, or thoughts on how to resolve?
ThanksFeb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 logtail started
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 Program starting: v1.36.0, Go 1.19.2: []string{"/usr/local/bin/tailscaled", "-port", "41641", "-tun", "tailscale0"}
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 LogID: 39dd9283ba8d870687f87fe5368a283276614800f0c5a11fb7b0f2bddf759a4d
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 logpolicy: using system state directory "/var/db/tailscale"
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 wgengine.NewUserspaceEngine(tun "tailscale0") ...
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 no TUN failure diagnostics for OS "freebsd"
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 wgengine.NewUserspaceEngine(tun "tailscale0") error: tstun.New("tailscale0"): open /dev/tun: no such file or directory
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 flushing log.
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 logger closing down
Feb 16 08:33:04 tailscaled 47361 2023/02/16 08:33:04 getLocalBackend error: createEngine: tstun.New("tailscale0"): open /dev/tun: no such file or directory
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 logtail started
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 Program starting: v1.36.0, Go 1.19.2: []string{"/usr/local/bin/tailscaled", "-port", "41641", "-tun", "tailscale0"}
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 LogID: 39dd9283ba8d870687f87fe5368a283276614800f0c5a11fb7b0f2bddf759a4d
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 logpolicy: using system state directory "/var/db/tailscale"
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 wgengine.NewUserspaceEngine(tun "tailscale0") ...
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 no TUN failure diagnostics for OS "freebsd"
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 wgengine.NewUserspaceEngine(tun "tailscale0") error: tstun.New("tailscale0"): open /dev/tun: no such file or directory
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 flushing log.
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 logger closing down
Feb 16 08:33:10 tailscaled 78903 2023/02/16 08:33:10 getLocalBackend error: createEngine: tstun.New("tailscale0"): open /dev/tun: no such file or directory -
@mt835 Probably this bug/solution:
https://forum.netgate.com/topic/177887/openvpn-could-not-be-established-after-upgrade-to-23-01-on-sg-3100/16 -
Thanks
-
How did you generate that log file?
I'm having the same issue with TailScale too.
-
I just cut and pasted from the system logs, I just looked for the Tailscale process. I checked right after I reinstalled the package, though you can check "enable syslog output" in Tailscale.
I did try the shell command referenced in the OPNVPN post linked above by SteveITS, but it did not appear to work here. I still can't start the Tailscale service.
-
I got the same issue with my SG-3100. In the office we have a SG-4100 which is updated to 23.01 and Tailscale is running there without any issue. So I am looking forward for a solution.
-
@j_belmans said in Tailscale service won't start, after upgrading to 23.01 on SG-3100:
I got the same issue with my SG-3100. In the office we have a SG-4100 which is updated to 23.01 and Tailscale is running there without any issue. So I am looking forward for a solution.
In my case, 23.01 messed up TailScale's authorization and no matter what I did from the GUI I could not get the tunnel back up.
Try reauthorizing TailScale from the command line, the basic format is this:
sudo tailscale up --authkey tskey-abcdef1432341818TailScale may complain options are missing, so just copy and paste the "error" message which will have all the options necessary and rerun.
-
@coolspot I ran the command and received the following error message:
panic: runtime error: slice bounds out of range [:-1]
goroutine 3 [running]:
github.com/mitchellh/go-ps.copy_params(...)
github.com/mitchellh/go-ps@v1.0.0/process_freebsd.go:159
github.com/mitchellh/go-ps.processes()
github.com/mitchellh/go-ps@v1.0.0/process_freebsd.go:200 +0x3ac
github.com/mitchellh/go-ps.Processes(...)
github.com/mitchellh/go-ps@v1.0.0/process.go:31
tailscale.com/safesocket.init.0.func1()
tailscale.com/safesocket/safesocket_ps.go:17 +0x18
tailscale.com/safesocket.tailscaledStillStarting()
tailscale.com/safesocket/safesocket.go:57 +0xac
tailscale.com/safesocket.Connect(0x314c3c54)
tailscale.com/safesocket/safesocket.go:106 +0x84
tailscale.com/client/tailscale.(*LocalClient).defaultDialer(0x93a9e0, {0x650640, 0x314a4010}, {0x57b8b6, 0x3}, {0x3141e048, 0x18})
tailscale.com/client/tailscale/localclient.go:104 +0xc0
net/http.(*Transport).dial(0x3147c000, {0x650640, 0x314a4010}, {0x57b8b6, 0x3}, {0x3141e048, 0x18})
net/http/transport.go:1170 +0x158
net/http.(*Transport).dialConn(0x3147c000, {0x650640, 0x314a4010}, {{}, 0x0, {0x31462030, 0x4}, {0x3141e048, 0x18}, 0x0})
net/http/transport.go:1608 +0x69c
net/http.(*Transport).dialConnFor(0x3147c000, 0x31418120)
net/http/transport.go:1450 +0x64
created by net/http.(*Transport).queueForDial
net/http/transport.go:1419 +0x9cNot sure what the above all means.
I rebooted the SG-3100, but still no connection with tailscale.Maybe you know more about it?
-
I get the exact same error on my 3100.
-
I upgraded to 23.01 this morning and no matter what I try, tailscale won't start also on my 3100
-
Firstly, I'll admit that I am just a regular user, not a programer but I have a question. I clicked on the Tailscale package dependency link which takes me to Freshports.org, and I am looking at the Arm7 version. Is it normal that the FBSD 14 version (1.34.2_1) is older than the FBSD 13 version (1.36.0)?
-
@mt835 I saw this as well and I even tried to manually install that older package and that didn't help. I am not sure if that is the issue that freshports doesn't also have 1.36 as the version for ARMV7 which is what is needed for the 3100.
If it is not resolved soon, I may have to request a iso to backout to 22.05.. But ideally would like to avoid that.
-
I decided to reinstall 22.05, so everything back to normal. I'll try the upgrade again once I have a clear idea that this has been resolved. This experience makes miss the ability to have ZFS and boot environments on the 3100.
-
@mt835 Thanks for posting this thread. Exact same issue here on my 3100.
I will try exploring some of the options provided here tomorrow but will likely go back to 22.05.Yes I miss not having the ZFS roll back option as well.
Cheers -
@steveits said in Tailscale service won't start, after upgrading to 23.01 on SG-3100:
https://forum.netgate.com/topic/177887/openvpn-could-not-be-established-after-upgrade-to-23-01-on-sg-3100/16
Did anyone get this working with their 23.01 upgrade on the 3100?
-
@wavesound I wasn't able to get it to work.
I saw a forum post about OpenVPN issues where the command
kldxref /boot/kernelfixed a tailscale and OpenVPN issue. I tried it but it didn't work for me. Maybe someone else got it to work? -
In the meanwhile I opened a support ticket at Netgate. I encourage everybody here to do the same, the more support tickets on the same issue, the quicker we will have a solution.
What does not work so far:
-Reinstall of 23.01
-the kernel solution
-the patch JimP mentioned: https://redmine.pfsense.org/issues/13963#note-4you can open a support ticket: https://www.netgate.com/tac-support-request
-
@j_belmans Flooding TAC with tickets does less than you think it might.
Adding your experiences to the redmine will be seen by our engineers, like JimP, and that's where things happen.
-
@rcoleman-netgate said in Tailscale service won't start, after upgrading to 23.01 on SG-3100:
@j_belmans Flooding TAC with tickets does less than you think it might.
Adding your experiences to the redmine will be seen by our engineers, like JimP, and that's where things happen.
@rcoleman-netgate That is not the correct Redmine issue. https://redmine.pfsense.org/projects/pfsense/activity?from=2023-02-27
No ticket currently exists and one should be added to Redmine for the pfSense package here: https://redmine.pfsense.org/projects/pfsense-packages/
@j_belmans (or anyone else in this thread): Have you heard back on your ticket, are they going to create a bug in Redmine or should we do it separately. I can't provide diagnostics since I NEED Tailscale running on my SG-3100 appliance.
-
Meant to post this link instead above: https://redmine.pfsense.org/issues/13963#note-8
-
@rcoleman-netgate I created the following issue. If others could add their feedback and log output, it would be much appreciated! https://redmine.pfsense.org/issues/14064
-
@wavesound
Done -
I've narrowed down the root cause, working on a fix now.
-
Fixed and pushed to snapshots and 23.01. Look for Tailscale
0.1.2in the package manager soon -
@cmcdonald Many thanks!
-
@cmcdonald Thanks!
Great video's you've posted as well, Thanks for those too. -
We are working another issue that is presently blocking 23.01 package updates from being published publicly. I have attached the updated Tailscale package here for testing.: https://redmine.pfsense.org/attachments/4767
install with
pkg add -f pfSense-pkg-Tailscale-0.1.2.pkg -
I did install this package on 23.01 on my SG-3100, and it appears to have resolved the issue for me. Rebooted as well. Tailscale is running fine. Thanks
-
@mt835 Thanks for the feedback!
-
No problem. Thanks for your efforts. If something changes I’ll post it.
-
I forgot to uninstall the previous version of tailscale before installing this package.
The install seemed to go ok:
Installing pfSense-pkg-Tailscale-0.1.2...
package pfSense-pkg-Tailscale is already installed, forced install
Extracting pfSense-pkg-Tailscale-0.1.2: 100%
Saving updated package information...
overwrite!
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Executing custom_php_install_command()...
Creating Tailscale interface group... done.
done.
Executing custom_php_resync_config_command()...done.
Menu items... done.
Services... done.
Writing configuration... done.Unfortunately Tailscale still fails to start:
Mar 6 05:15:29 php-fpm 364 /pkg_edit.php: The command '/usr/sbin/sysrc -c -f /usr/local/etc/rc.conf.d/tailscaled tailscaled_enable="YES" tailscaled_exitnode_enable="NO" tailscaled_port="41641" tailscaled_syslog_output_enable="YES" tailscaled_syslog_output_facility="daemon" tailscaled_syslog_output_priority="warning"' returned exit code '1', the output was ''As well, I no longer see Tailscale in my package list.
I've tried manually uninstalling Tailscale using:
pkg delete pfSense-pkg-Tailscale-0.1.2.pkgWhich fails:
Updating database digests format: 100%
No packages matched for pattern 'pfSense-pkg-Tailscale-0.1.2.pkg'Checking integrity... done (0 conflicting)
1 packages requested for removal: 0 locked, 1 missingThat said using pkg info I can see pfSense-pkg-Tailscale-0.1.2.pkg and package tailscale-1.36.0
Any recommendations before I make things worse.
:) -
Same thing happened to me. It came up after a reboot.
-
Thanks.
I'll try another reboot then ...
Fingers crossed.Unfortunately no luck with the reboot
-
I was able to install the https://redmine.pfsense.org/attachments/4767 with
pkg add -f /tmp/pfSense-pkg-Tailscale-0.1.2.pkgthat you posted. Got it working after a quick reboot. Thanks you for releasing that early for us, much appreciated. -
@sabennett I had to refresh the tailscale status tab in pfsense in order for it to click.
-
Same issue - using Tailscale 0.1.2 and dependency 1.36.0 with Netgate 3100 on v23.01 - tried to install/re-install w/ reboot but cannot seem to get the service started.
-
Did you reboot after installing the new package? I had to reboot.
Did you also install the separate patch mentioned here? https://redmine.pfsense.org/issues/13963#note-5
-
@wavesound said in Tailscale service won't start, after upgrading to 23.01 on SG-3100:
https://redmine.pfsense.org/issues/13963#note-5
Thanks @wavesound ! I didn't apply the patch - working now with patch applied and rebooted.
-
This post is deleted! -
You're correct,
My mistake, I guess I hadn't applied the patch mentioned here --> https://redmine.pfsense.org/issues/13963#note-5
Once I did this, reboot, added Tailscale from the package manager and it worked.
