Token Refresher needs a ServiceAccount to list namespaces

[kingdonb]

The registry-token-refresher deployment needs a service account to refresh tokens for ECR (and other externally provided registry services, I presume).

The SA should get bound (ClusterRoleBinding) to a role that has permission to list namespaces, like this:

rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["list"]

and the ServiceAccount needs to be linked to the pod in the registry-token-refresher deployment's pod template.spec, like:

spec:
  template:
    spec:
      serviceAccount: deis-registry-token-refresher

That ServiceAccount needs to be created as well, since registry-token-refresher evidently didn't need any SA until RBAC in k8s 1.9

@bit-herder found this

[edisonwang]

Hi,
I just run into this as well.... is there any instruction so that I can grant permission safely to such service account?

Current error log:

2018/09/04 11:32:00 Error getting kubernetes namespaces namespaces is forbidden: User "system:serviceaccount:deis:default" cannot list namespaces at the cluster scope

Thanks.

[kingdonb]

If you're familiar with creating service accounts and binding roles, I think there's enough information in the issue to resolve it...

We should really put this fix into another new release. This could be v2.19.5. I didn't realize it was still open, I thought we'd already resolved all of the ServiceAccount issues, but evidently nobody is using external registry at Team Hephy so this one got skipped over.

At a glance, I think you could kubectl -n deis apply -f the_file.yaml with the file below and delete your deis-registry-token-refresher pod that is malfunctioning, hopefully this should solve it:

the_file.yaml

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: deis:deis-registry-token-refresher
  labels:
    app: deis-registry-token-refresher
    heritage: deis
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["list"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: deis:deis-registry-token-refresher
  labels:
    app: deis-registry-token-refresher
    heritage: deis
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: deis:deis-registry-token-refresher
subjects:
- kind: ServiceAccount
  name: deis-registry-token-refresher
  namespace: deis
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: deis-registry-token-refresher
  labels:
    heritage: deis

Let us know please if this helps! (Or if you get any more errors... I have a sneaking suspicion that registry-token-refresher is actually going to need more permissions than this.)

[kingdonb]

Actually I think you may also need to add the service account to the deployment spec before deleting the pod, like this:

(kubectl -n deis edit deploy deis-registry-token-refresher)

... skip to the deployment spec: template: section and add the serviceAccount like this, beneath metadata:

...
  template:
    metadata:
      labels:
        app: deis-registry-token-refresher
    spec:
      serviceAccount: deis-registry-token-refresher
      containers:
...

The new pod will pick up this new spec after you delete the running pod (or honestly I think editing the deployment should trigger that to happen automatically... as long as you do this step last)

[Cryptophobia]

@kingdonb , do we need a fix for this one?