Webhook Receiver for AWS ECR

[pdeva]

I was surpised that Receiver doesnt have built in support ECR (even though it seemingly supports every other docker registry).

What then is the recommended way to trigger webhooks to Flux on image push to ECR?

[kingdonb]

I suspect it must be possible to use the ECR EventBridge with the Generic or Generic HMAC receiver.

These receivers do very limited validation (HMAC checks a signature) or no validation at all on the request payload. They can be triggered by any HTTP POST so long as it is well-formed and targets a valid receiver URL path.

I am not sure how to configure outbound webhooks from the ECR, whether it uses EventBridge or CloudWatch Events or something else, but the webhook configuration would need to be provided from AWS's side. I went looking for docs and none of AWS's docs in this area were especially clear to me. I found these two:

• https://docs.aws.amazon.com/codepipeline/latest/userguide/create-cwe-ecr-source-console.html
• https://docs.aws.amazon.com/AmazonECR/latest/userguide/ecr-eventbridge.html

Both seemed relevant but neither was really clear on how to trigger a webhook in the way that Flux expects.

The actions that can be automatically triggered include the following ...

• (proceeds to list off actions that all take place entirely inside of the AWS control plane, none of which are a simple HTTP POST)

You can definitely trigger a Lambda here that will trigger the Flux webhook. I know there are some users who have provided examples of code that can sign an HMAC request according to the secret, but I have never seen one in the form of a lambda. This would be a nice addition. But for Flux, that's pretty complicated and far distant from Flux's mission, and a rigorous example may be a poor use of time anyway as the webhook may be a low value target, it doesn't accept any instructions by design, only triggers a sync on event.

I'd like to see that added, but I think it belongs in AWS's docs. Maybe we can add a reference from Flux's docs when they add the example.

There have been in-depth discussions about the security of the webhook here on this board before, but tl;dr: it might be sufficient to keep the URL secret in order to prevent a DoS. The best is HMAC, because requests being signed in this way means they should not be susceptible to inspection/capture and replay, or simple interception because of information leaked in HTTP ingress logs, etc.

If you can get something on ECR's side/the AWS control plane to talk to the Generic HMAC receiver, it would be a good addition to the docs since undoubtedly many users are on AWS (it follows that someone out here must have already dealt with this problem before.)

[stefanprodan]

You can use the generic receiver for ECR, to protect the endpoint from DDOS make sure to expose notification-controller using an internal ELB/NLB.

[fabidick22]

In case anyone is interested, I'm working on a solution with terraform, I'm testing it in a staging environment and so far everything works fine.
https://github.com/fabidick22/flux2-ecr-webhook

[kingdonb]

I've referred back to your docs today, and I may try setting this up for myself soon. I almost wonder if this isn't something that should go in the fluxcd-community org! I see you've done several releases and are maintaining this, thanks for publishing your work!

[stefanprodan]

This deserves a blog post on fluxcd.io. Great job @fabidick22 🥇

[fabidick22]

Thanks guys, I'm working on releasing a version that allows configuring webhooks in a more flexible way. So far, it's great to see how Flux reconciles a new version of a service in a matter of seconds 😍