Tailscale not online

mcury

@IanMcLeish said in Tailscale not online:

I think I will give up.

Weird, those exactly steps worked for me yesterday..

IanMcLeish

@mcury
I got it!

I had to go to the command prompt and run the command tailscale up which then gave me a link to the tailscale web admin page to authenticate. I kinda thought that the whole point of the key was to avoid that, but it is back up and running anyway.

Thanks for your help - was nice at least to know i wasn't doing anything stupid, but maybe missing that last part was me doing something stupid!?

mcury

@IanMcLeish said in Tailscale not online:

I got it!

great

totalimpact

This still seems to be an issue, and makes the Tailscale client unreliable. I have 4 nodes down now with expiry disabled, after some unknown time, and then a router reboot they can no longer authenticate.

Error executing command (/usr/local/bin/tailscale status)
# Health check:
#     - not logged in, last login error=invalid key: API key does not exist

unexpected state: NoState

From the CLI I can run tailscale login, and it re-authenticates the same node, I can tailscale down + up and it connects fine, status on the webpage looks good, but if I reboot or restart the Tailscale service in the webpage it can no longer connect again with the same error needing to login again. The only way to make it work reliably is to clear the config, delete the node and reconnect as a new node.

Pfsense 2.7.2, Tailscale package 0.1.4

elvisimprsntr

@totalimpact

Tailscale 1.54.0 is 2+ years out of date. Tailscale has made quite a number of changes since Tailscale 1.54.0, likely rendering it incompatible with their servers.

I would consider manually updating the Tailscale FreeBSD package.

FreshPorts does not maintain an archive of all the releases, only the latest compiled by the volunteer maintainers.

The key to manually upgrading is knowing which FreeBSD version your pfSense release is running, i.e. 14 or 15.

You can following along here.

ryan.goodfellow

Upgraded 25.07 and Tailscale is broken in the way users here describe. I can manually log in using sudo /usr/local/bin/tailscale login, but the tailscale service in pfSense does not pick this up and restarting the service clobbers the login state. Given 16004 was logged 7 months ago with zero activity, this is an indication that Netgate devices no longer support Tailscale.

satrajitdas

Upgraded to 25.07 and facing the same issue. Tried the "tailscale up" command as suggested above but restarting the tailscale service kills the login again.

Wolf666

@satrajitdas I had same problem, just go to tailscale control panel and generate a new key, then paste on pfsense gui, you should be ok.

totalimpact

@Wolf666 Just reboot your router - bet the key wont auth again.

sandie

Hi guys,

Just $0.01, because I faced same issue on pfSense+ 24.07.1 upgrade, but I think root cause may NOT be pfSense+ per se.

To my understanding (which may be wrong):

Tailscale uses 2 keys:

node auth key - by default expires after 180 days. We do not see it, but you can disable its expiration and we normally do that, right? So node registration should not expire after 180 days and re-authentication should not be necessary.
preauth key - it is valid for maximum 90 days and you input it to VPN / Tailscale / Authentication as Pre-authentication Key (eg. tskey-auth-123456789011CNTRL-2pz1kCcaSjJucckK7U5Xbz123456a890). Remember that this key is valid for no longer than 90 days. So usually when you upgrade pfSense+ (=> Tailscale upgrade) this key is well expired. Device can not (re)authenticate using it - it is expired:

- not logged in, last login error=invalid key: API key does not exist

(because key that pfSense+ tried to use is expired!)

Tailscale says that device may sometimes require re-authentication. Here is info from Tailscale KB:

Auth keys

Auth keys are available for all plans.

Pre-authentication keys (called auth keys) let you register new nodes without needing to sign in using a web browser. This is most useful when spinning up containers, IoT devices, or using infrastructure-as-code systems like Terraform.

An auth key authenticates a device as the user who generated the key. That is, if Alice generates an auth key, and uses it to add a server to her tailnet, then that device is authenticated with Alice's identity. Think of it as logging into a device.

However, if you use tags with an auth key, after a device logs in as the user who generated the auth key, the device assumes the identity of the auth key's tags.

As an alternative to directly creating auth keys, consider using an OAuth client. You can use an OAuth client and the Tailscale API to programmatically create auth keys.

Types of auth keys

Auth keys can either be:

One-off, for one-time use. They can only be used to connect a device or server one time.
This is meant for situations where you can't authenticate on the device yourself, so using a key is more practical.
For example, a cloud server might use a one-off key to connect.
Reusable, for multiple uses. They can be used to connect multiple devices. For example, multiple instances of an on-premises database might use a reusable key to connect.

Be very careful with reusable keys! These can be very dangerous if stolen. They're best kept in a key vault product specially designed for the purpose.

Key expiry

An auth key automatically expires after the number of days you specified when you generated the key. You can choose the number of days, between 1 and 90 inclusive, for the key expiry. If you don't specify an expiry time, the auth key will expire after the maximum of 90 days. If you want to continue using an auth key after it expires, you need to generate a new key.

You can enable or disable key expiry on a device by using the Machines page of the admin console and by using the Update device key method in the Tailscale API.

If an auth key expires, any device authorized by it remains authorized until its node key expires. Each device generates a node key when you log in to Tailscale and uses it to identify itself to the tailnet. By default, node keys automatically expire every 180 days. You can change the default node key expiry from the Key Expiry section of the Device management page of the admin console.

Learn more about key management.

You can use tags as "service accounts" and have some devices NOT bound to any specific user (removal of user removes devices he own). You can define 1 or multiple tag owners (users managing tag).

Tag vs. user authentication

Tags are parallel to user authentication. They serve the same role as a user account, except they're intended for service-based devices, such as a web server or an app connector. As a result, it's impossible for a user account identity and a tag identity to exist on the same device. Applying a tag to a device previously authenticated with a user account removes the user account. Similarly, authenticating a device with a user account removes all tags from the device.

Because tags are intended for non-user devices, they have qualities and limitations that make them unsuitable for authenticating end-user devices, such as a MacBook or a mobile device. For example, devices with a tag-based identity cannot use SSH to connect to a device with a user-based identity.

Key expiry

When you apply a tag to a device for the first time and authenticate it, the tagged device's key expiry is disabled by default.

If you re-authenticate a device tagged before March 10, 2022, its expiry will be disabled by default.

If you change the tags on the device from the admin console, Tailscale CLI, or the Tailscale API, the device's key expiry will not change unless you re-authenticate. After you re-authenticate, the device's key expiry will be disabled.

You can enable or disable key expiry on a device from the Machines page of the admin console or by using the Tailscale API.

Key expiry for tagged devices

Key expiry for tagged devices is disabled by default. If you change the tags on the device through the admin console, Tailscale CLI, or Tailscale API, the device's key expiry will not change unless you re-authenticate. That is, if it is enabled, it stays enabled; and if it is disabled, it stays disabled. After you re-authenticate, the device's key expiry will be disabled.
You can find recently revoked or expired keys on the Keys page of the admin console.

Best practices

Depending on what devices you're authenticating, consider using an auth key that is:

Ephemeral, for authenticating ephemeral nodes as part of short-lived workloads. Because node keys do not persist when a workload restarts, they reconnect as a different node. Tailscale automatically removes inactive nodes. For example, containers or Lambda functions should use an ephemeral key to connect.

Pre-approved, for servers. If your tailnet has device approval enabled, this lets you add a device to your tailnet without further authorization. For example, shared devices, such as servers, should use a pre-approved auth key to connect in a network with device approval.

Pre-signed, for nodes whose auth keys are signed locally on a signing node, which applies to tailnets with Tailnet Lock enabled. You can make an auth key (created by any means) pre-signed only by using the tailscale lock sign CLI command.

Tagged, for servers. You can automatically apply a tag to a device by including the tag in the auth key. Access control policies restricting the device's permissions based on the tag apply after provisioning the device. For example, shared devices, such as servers, should use a tagged auth key to connect.

I am personally going to try Tagged (preauth) key and all my pfSense+ exit nodes (3) are already tagged as "router". So I will remove nodes from tailnet and re-add with Tagged key providing "router" tag.
I read that Tailscale on software upgrade MAY (rarely) REQUIRE device reauthentication. Having in mind that normal preauth keys expire after 90 days you should provide valid (non-expired) preauth key before pfSense+ upgrade? Eventually maybe if routers are tagged and preauth key is tagged then there will be no problem?
Currently pfSense+ does not ask us about expiration date of Preauth Key, so it can not remind us that key is expired and it may lead to problems. When valid key is needed (reauthentication) then device will fail with the message we see.

I would not blame pfSense+ yet, because I think Tailscale may require device reauthorisation sometimes and message you got tells you are trying to use expired key for authentication thus process is failing. I agree reauthentications should be rare or non-needed, but we may not know everything here.