Tailscale service won't start, after upgrading to 23.01 on SG-3100

mt835

I upgraded my SG-3100 to 23.01 yesterday, and the Tailscale service is reported as stopped, and trying to manually start the service hasn't worked. I have rebooted, uninstalled, and then reinstalled the package but not change. Looking at the system logs it does appear a file is missing: "tun".

Anyone seeing a similar issue, or thoughts on how to resolve?
Thanks

Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 logtail started
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 Program starting: v1.36.0, Go 1.19.2: []string{"/usr/local/bin/tailscaled", "-port", "41641", "-tun", "tailscale0"}
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 LogID: 39dd9283ba8d870687f87fe5368a283276614800f0c5a11fb7b0f2bddf759a4d
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 logpolicy: using system state directory "/var/db/tailscale"
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 wgengine.NewUserspaceEngine(tun "tailscale0") ...
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 no TUN failure diagnostics for OS "freebsd"
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 wgengine.NewUserspaceEngine(tun "tailscale0") error: tstun.New("tailscale0"): open /dev/tun: no such file or directory
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 flushing log.
Feb 16 08:33:03 tailscaled 47361 2023/02/16 08:33:03 logger closing down
Feb 16 08:33:04 tailscaled 47361 2023/02/16 08:33:04 getLocalBackend error: createEngine: tstun.New("tailscale0"): open /dev/tun: no such file or directory
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 logtail started
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 Program starting: v1.36.0, Go 1.19.2: []string{"/usr/local/bin/tailscaled", "-port", "41641", "-tun", "tailscale0"}
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 LogID: 39dd9283ba8d870687f87fe5368a283276614800f0c5a11fb7b0f2bddf759a4d
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 logpolicy: using system state directory "/var/db/tailscale"
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 wgengine.NewUserspaceEngine(tun "tailscale0") ...
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 no TUN failure diagnostics for OS "freebsd"
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 wgengine.NewUserspaceEngine(tun "tailscale0") error: tstun.New("tailscale0"): open /dev/tun: no such file or directory
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 flushing log.
Feb 16 08:33:09 tailscaled 78903 2023/02/16 08:33:09 logger closing down
Feb 16 08:33:10 tailscaled 78903 2023/02/16 08:33:10 getLocalBackend error: createEngine: tstun.New("tailscale0"): open /dev/tun: no such file or directory

SteveITS

@mt835 Probably this bug/solution:
https://forum.netgate.com/topic/177887/openvpn-could-not-be-established-after-upgrade-to-23-01-on-sg-3100/16

mt835

Thanks

coolspot

How did you generate that log file?

I'm having the same issue with TailScale too.

mt835

I just cut and pasted from the system logs, I just looked for the Tailscale process. I checked right after I reinstalled the package, though you can check "enable syslog output" in Tailscale.

I did try the shell command referenced in the OPNVPN post linked above by SteveITS, but it did not appear to work here. I still can't start the Tailscale service.

j_belmans

I got the same issue with my SG-3100. In the office we have a SG-4100 which is updated to 23.01 and Tailscale is running there without any issue. So I am looking forward for a solution.

coolspot

@j_belmans said in Tailscale service won't start, after upgrading to 23.01 on SG-3100:

I got the same issue with my SG-3100. In the office we have a SG-4100 which is updated to 23.01 and Tailscale is running there without any issue. So I am looking forward for a solution.

In my case, 23.01 messed up TailScale's authorization and no matter what I did from the GUI I could not get the tunnel back up.

Try reauthorizing TailScale from the command line, the basic format is this:

sudo tailscale up --authkey tskey-abcdef1432341818

TailScale may complain options are missing, so just copy and paste the "error" message which will have all the options necessary and rerun.

j_belmans

@coolspot I ran the command and received the following error message:

panic: runtime error: slice bounds out of range [:-1]

goroutine 3 [running]:
github.com/mitchellh/go-ps.copy_params(...)
github.com/mitchellh/go-ps@v1.0.0/process_freebsd.go:159
github.com/mitchellh/go-ps.processes()
github.com/mitchellh/go-ps@v1.0.0/process_freebsd.go:200 +0x3ac
github.com/mitchellh/go-ps.Processes(...)
github.com/mitchellh/go-ps@v1.0.0/process.go:31
tailscale.com/safesocket.init.0.func1()
tailscale.com/safesocket/safesocket_ps.go:17 +0x18
tailscale.com/safesocket.tailscaledStillStarting()
tailscale.com/safesocket/safesocket.go:57 +0xac
tailscale.com/safesocket.Connect(0x314c3c54)
tailscale.com/safesocket/safesocket.go:106 +0x84
tailscale.com/client/tailscale.(*LocalClient).defaultDialer(0x93a9e0, {0x650640, 0x314a4010}, {0x57b8b6, 0x3}, {0x3141e048, 0x18})
tailscale.com/client/tailscale/localclient.go:104 +0xc0
net/http.(*Transport).dial(0x3147c000, {0x650640, 0x314a4010}, {0x57b8b6, 0x3}, {0x3141e048, 0x18})
net/http/transport.go:1170 +0x158
net/http.(*Transport).dialConn(0x3147c000, {0x650640, 0x314a4010}, {{}, 0x0, {0x31462030, 0x4}, {0x3141e048, 0x18}, 0x0})
net/http/transport.go:1608 +0x69c
net/http.(*Transport).dialConnFor(0x3147c000, 0x31418120)
net/http/transport.go:1450 +0x64
created by net/http.(*Transport).queueForDial
net/http/transport.go:1419 +0x9c

Not sure what the above all means.
I rebooted the SG-3100, but still no connection with tailscale.

Maybe you know more about it?

smyers

@j_belmans

I get the exact same error on my 3100.

davidnolf

I upgraded to 23.01 this morning and no matter what I try, tailscale won't start also on my 3100

mt835

Firstly, I'll admit that I am just a regular user, not a programer but I have a question. I clicked on the Tailscale package dependency link which takes me to Freshports.org, and I am looking at the Arm7 version. Is it normal that the FBSD 14 version (1.34.2_1) is older than the FBSD 13 version (1.36.0)?

davidnolf

@mt835 I saw this as well and I even tried to manually install that older package and that didn't help. I am not sure if that is the issue that freshports doesn't also have 1.36 as the version for ARMV7 which is what is needed for the 3100.

If it is not resolved soon, I may have to request a iso to backout to 22.05.. But ideally would like to avoid that.

mt835

I decided to reinstall 22.05, so everything back to normal. I'll try the upgrade again once I have a clear idea that this has been resolved. This experience makes miss the ability to have ZFS and boot environments on the 3100.

sabennett

@mt835 Thanks for posting this thread. Exact same issue here on my 3100.
I will try exploring some of the options provided here tomorrow but will likely go back to 22.05.

Yes I miss not having the ZFS roll back option as well.
Cheers

wavesound

@steveits said in Tailscale service won't start, after upgrading to 23.01 on SG-3100:

https://forum.netgate.com/topic/177887/openvpn-could-not-be-established-after-upgrade-to-23-01-on-sg-3100/16

Did anyone get this working with their 23.01 upgrade on the 3100?

mmrk

@wavesound I wasn't able to get it to work.

I saw a forum post about OpenVPN issues where the command kldxref /boot/kernel fixed a tailscale and OpenVPN issue. I tried it but it didn't work for me. Maybe someone else got it to work?

j_belmans

In the meanwhile I opened a support ticket at Netgate. I encourage everybody here to do the same, the more support tickets on the same issue, the quicker we will have a solution.

What does not work so far:
-Reinstall of 23.01
-the kernel solution
-the patch JimP mentioned: https://redmine.pfsense.org/issues/13963#note-4

you can open a support ticket: https://www.netgate.com/tac-support-request

rcoleman-netgate

@j_belmans Flooding TAC with tickets does less than you think it might.

Adding your experiences to the redmine will be seen by our engineers, like JimP, and that's where things happen.

https://redmine.pfsense.org/issues/13963

wavesound

@rcoleman-netgate said in Tailscale service won't start, after upgrading to 23.01 on SG-3100:

@j_belmans Flooding TAC with tickets does less than you think it might.

Adding your experiences to the redmine will be seen by our engineers, like JimP, and that's where things happen.

https://redmine.pfsense.org/issues/13963

@rcoleman-netgate That is not the correct Redmine issue. https://redmine.pfsense.org/projects/pfsense/activity?from=2023-02-27

No ticket currently exists and one should be added to Redmine for the pfSense package here: https://redmine.pfsense.org/projects/pfsense-packages/

@j_belmans (or anyone else in this thread): Have you heard back on your ticket, are they going to create a bug in Redmine or should we do it separately. I can't provide diagnostics since I NEED Tailscale running on my SG-3100 appliance.

wavesound

Meant to post this link instead above: https://redmine.pfsense.org/issues/13963#note-8