Gatillos

I set up a local kubernetes cluster using microk8s just as a development/home cluster and as part of it I ended up connecting to the kubernetes dashboard application using a service account token, as recommended

$ microk8s kubectl create token default
(...token output...)

This process lead me to a few questions:

• What is that token?
• Where does it come from?
• Where is it stored?
• Does it expire?

Essentially, "wait how does the whole token authentication thing work?"

Token types

So far we have dealt with two token types:

1. We have static tokens, like the one we added in the kube config file. This token is equivalent to a user:password pair. If exposed it could be used forever to act as that user.
   • These are either stored as secrets or via some other mechanism
   • In microk8s there are some credentials created during installation, including the admin user we have been using. This can be found in /var/snap/microk8s/current under credentials/known_tokens.csv. Changing the token can be done by editing the file.
2. Then we have temporary tokens, like the one we used for the dashboard access

Now, let's think about those... what is a temporary token? where are they stored? how do they expire? Those are the kind of questions I like to dig in to learn about the internals, and I end up finding a lot of this is implemented in a perfectly reasonable and standards-based way.

Inspecting a temporary token

As usual:

• Kubernetes authentication docs to the rescue

There we see:

• The created token is a signed JSON Web Token (JWT)
• The signed JWT can be used as a bearer token to authenticate as thegiven service account

Now, let's take our token and inspect it - pasting it into jwt.io to see its content (this is safe as long as you don't expose your cluster anywhere):

{
  "alg": "RS256",
  "kid": "_h3i-pvtWSsSXXVyXxxxxxxxxxxxxxxx"
}

{
  "aud": [
    "https://kubernetes.default.svc"
  ],
  "exp": 1664705605,
  "iat": 1664702005,
  "iss": "https://kubernetes.default.svc",
  "kubernetes.io": {
    "namespace": "default",
    "serviceaccount": {
      "name": "default",
      "uid": "755e1766-e817-xxxx-xxxx-xxxxxxxxxxxx"
    }
  },
  "nbf": 1664702005,
  "sub": "system:serviceaccount:default:default"
}

• The aud parameter indicates the audience, in this case kubernetes
• The sub indicates the 'subject' (user, service account, etc). When we created the token we didn't specify a service account, which means it used the 'default' service account (not great practice but hey we are starting)
  • This means the authentication request is coming from system:serviceaccount:default:default
  • You can take a look at this account with kubectl get sa default -o yaml
• The date parameters define the validity period: "nbf": 1664702005 (not before), "iat": 1664702005 (issued at) and "exp": 1664705605 (expiry) which you can quickly convert to a date with:

$ date -d@"1664702005"
Sun  2 Oct 10:13:25 BST 2022
$ date -d@"1664705605"
Sun  2 Oct 11:13:25 BST 2022

How can I use this token?

Any client application we give the token to will be sending an HTTPS request to the API server with a header set to Authentication: Bearer <token>. That's all is needed. Any requests will then be approved/rejected based on the permissions of that account.

If you want to test a particular call, you can use the token with for example:

kubectl --token="xxxxxxxxxxxxxx" get pods --all-namespaces

And finally, where is it stored?

• The token is not stored as a secret and it's not directly linked from the service account object
• In fact, the token is not stored anywhere in kubernetes, it's validated from the signature
• Secrets don't expire but these tokens do (an hour as seen above in the JWT token itself)

As usual, you will be able to dig into the Dashboard project itself, for instance here you have alternative access methods

A final note on temporary tokens: they do provide advantages in limiting the scope of attack as they will expire but you will still need to associate them to limited users or service accounts with limited restrictions, something I haven't gone into yet and we have been using the default service account. For example, if the temporary account is captured it can be used to create permanent accounts/secrets.