Add support for service account token in ece-credential-provider

[0x221A]

What would you like to be added:

Kubernetes’ alpha feature KubeletServiceAccountTokenForCredentialProviders allows the kubelet to inject a projected ServiceAccount token into the credential-provider plugin’s request. By leveraging this token, on-premise or self-hosted clusters can avoid baking static cloud credentials into each node’s file system and instead perform OIDC-based authentication at runtime.

Why is this needed:

Using projected ServiceAccount tokens lets on-prem and air-gapped clusters avoid baking long-lived cloud secrets into every node. Instead, nodes request short-lived, scoped credentials at pull time via OIDC, improving security and automating rotation without impacting existing workflows.

/kind feature

[cartermckinnon]

Sounds cool! I'm not very familiar with this feature, are you aware of a reference impl I can take a look at?

I'm not sure what we'd do with the token and annotations in the request: https://github.com/kubernetes/kubernetes/blob/fcf22b3ebe00bdf5fb96691d9e05a0c01a661f83/staging/src/k8s.io/kubelet/pkg/apis/credentialprovider/v1/types.go#L36-L46

[cartermckinnon]

/triage accepted

[0x221A]

It's a lot like how eks-pod-identity-webhook works. The idea is to use the service account token passed in the CredentialProviderRequest to call AssumeRoleWithWebIdentity and get temporary AWS credentials — so no need to bake long-lived secrets into the node.

For the annotations, we could use them to optionally override values like the role ARN on a per-service-account basis (e.g., pulling from AWS_ROLE_ARN in the annotation instead of using a static value in the plugin config). That makes it more flexible for multi-tenant or varied workload setups.

To make it work, we’d just need to make sure serviceAccountTokenAudience is set to sts.amazonaws.com in the kubelet credential provider config, since that’s what STS expects.

[0x221A]

@cartermckinnon I put together a quick sample that shows the idea in code:

output, err := ecr.NewFromConfig(cfg).GetAuthorizationToken(ctx, &ecr.GetAuthorizationTokenInput{}, func(o *ecr.Options) {
	o.Credentials = aws.CredentialsProviderFunc(func(ctx context.Context) (aws.Credentials, error) {
		output2, err := sts.NewFromConfig(cfg).AssumeRoleWithWebIdentity(ctx, &sts.AssumeRoleWithWebIdentityInput{
			RoleArn:          aws.String(assumeRoleArn), // // This defaults to an environment variable, but can be overridden per service account via annotation.
			WebIdentityToken: aws.String(serviceAccountToken),
		})
		if err != nil {
			return aws.Credentials{}, fmt.Errorf("failed to assume role: %w", err)
		}
		return aws.Credentials{
			AccessKeyID:     *output2.Credentials.AccessKeyId,
			SecretAccessKey: *output2.Credentials.SecretAccessKey,
			SessionToken:    *output2.Credentials.SessionToken,
		}, nil
	})
})

We’ll need to pick an annotation prefix — eks.amazonaws.com/role-arn (used by eks-pod-identity-webhook) is familiar, but it might conflict with service accounts that already use it for other roles. Using something like credentialprovider.aws/role-arn keeps things separate and avoids any overlap. Either way works, just good to agree early.

[theobarberbany]

@cartermckinnon The relevant enhancement: kubernetes/enhancements#4412 and KEP: https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/4412-projected-service-account-tokens-for-kubelet-image-credential-providers/README.md should have additional context that may help! :)

Happy to help with anything that needs doing, I'm going to try and do the same across gcp and azure as well!

[cartermckinnon]

Thanks folks! I'll chat with the team and get a PR going.

[cartermckinnon]

@theobarberbany @0x221A if y'all can take a look at #1155 that'd be awesome!