❗NOTICE (aws-lambda): Lambda cdk managed log group duplicates

[mjgoble]

Please add your +1 👍 to let us know you have encountered this

Status: IN-PROGRESS

Overview:

The addition of cdk managed log group to lambda function breaks any lambda function being deployed(new or existing) as the default value of the feature flag : USE_CDK_MANAGED_LAMBDA_LOGGROUP (@aws-cdk/aws-lambda:useCdkManagedLogGroup) is set to true

Regression Issue

• Select this option if this issue appears to be a regression.

Complete Error Message:

CDK deployment fails with errors similar to

10:27:37 AM | CREATE_FAILED        | AWS::Logs::LogGroup        | OnFailureExtractorLambdaLogGroup...
Resource handler returned message: "Resource of type 'AWS::Logs::LogGroup' with identifier '{"/properties/LogGroupName":"/aws/lambda/OnFailureExt
ractorLambda-OnFailureExtractorLambda6-..."}' already exists." (RequestToken: ... , HandlerErrorCode: Al
readyExists)

❌  OnFailureExtractorLambda failed: _ToolkitError: The stack named OnFailureExtractorLambda failed to deploy: UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "Resource of type 'AWS::Logs::LogGroup' with identifier '{"/properties/LogGroupName":"/aws/lambda/OnFailureExtractorLambda-OnFailureExtractorLambda6-..."}' already exists." (RequestToken: ..., HandlerErrorCode: AlreadyExists)

Workaround:

As a workaround, one can pass the feature flag @aws-cdk/aws-lambda:useCdkManagedLogGroup value as false

new App({
  context: {
    '@aws-cdk/aws-lambda:useCdkManagedLogGroup': false,
  },
});

Further details can be found in below links :

• https://github.com/QuantumNeuralCoder/aws-cdk/blob/main/packages/aws-cdk-lib/README.md#app-context
• https://docs.aws.amazon.com/cdk/v2/guide/context.html#context-construct

Solution:

Upgrading to patch version v2.200.1 should resolve this issue for existing lambdas

Describe the bug

When upgrading to aws-cdk-lib 2.200.0 cdk is trying to create log groups for the custom resource provider framework lambdas, however the already exist and is causing the stack deployment to fail

Last Known Working CDK Library Version

2.199.0

Expected Behavior

CDK deployment should not change anything

Current Behavior

Additional LogGroups are created, and when deploying, fails due to

Resource of type 'AWS::Logs::LogGroup' with identifier '{"/properties/LogGroupName":"/
aws/lambda/my-cr-provider-function"}' already exists."

Reproduction Steps

CDK deploy a stack with a custom resource using < 2.200.0

Try again with v 2.200.0

Possible Solution

No response

Additional Information/Context

No response

AWS CDK Library version (aws-cdk-lib)

2.200.0

AWS CDK CLI version

2.1017.1

Node.js Version

n/a

OS

Linux

Language

Python

Language Version

3.12

Other information

No response

[QuantumNeuralCoder]

Can you please confirm the value of feature flag
USE_CDK_MANAGED_LAMBDA_LOGGROUP
for your CDK app

[mjgoble]

Can you please confirm the value of feature flag USE_CDK_MANAGED_LAMBDA_LOGGROUP for your CDK app

Not set

[mjgoble]

@QuantumNeuralCoder - this might not just be affecting custom resource provider framework lambdas, seems to be any lambda function created in the stack is now getting a new LogGroup in the cdk.out

[otbe]

Seeing the same problem also for regular lambdas in all of our stacks. USE_CDK_MANAGED_LAMBDA_LOGGROUP not st.

[khellan]

This bit us today. Setting the flag in cdk.json worked:
"@aws-cdk/aws-lambda:useCdkManagedLogGroup": false,

This is not ideal of course since we will have to do this in all our stacks and ideally, we would want the default behaviour for new lambdas.

[mjgoble]

This bit us today. Setting the flag in cdk.json worked: "@aws-cdk/aws-lambda:useCdkManagedLogGroup": false,

This is not ideal of course since we will have to do this in all our stacks and ideally, we would want the default behaviour for new lambdas.

Yeah not ideal, cdk v2 should not include breaking changes and I would class this as one. Setting the default behaviour appropriately or reverting the change that caused this should be done rather than us having to set flags in our cdk.json

[tikki100]

Seeing this as well when upgrading to 2.200.0:

10:27:37 AM | CREATE_FAILED        | AWS::Logs::LogGroup        | OnFailureExtractorLambdaLogGroup...
Resource handler returned message: "Resource of type 'AWS::Logs::LogGroup' with identifier '{"/properties/LogGroupName":"/aws/lambda/OnFailureExt
ractorLambda-OnFailureExtractorLambda6-..."}' already exists." (RequestToken: ... , HandlerErrorCode: Al
readyExists)

❌  OnFailureExtractorLambda failed: _ToolkitError: The stack named OnFailureExtractorLambda failed to deploy: UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "Resource of type 'AWS::Logs::LogGroup' with identifier '{"/properties/LogGroupName":"/aws/lambda/OnFailureExtractorLambda-OnFailureExtractorLambda6-..."}' already exists." (RequestToken: ..., HandlerErrorCode: AlreadyExists)

We're only deploying a Lambd and setting the iam Policystatement on it:

  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);
    const lambda = this.createLambdaFunction("OnFailureExtractorLambda");
    this.setLambdaPermissions(lambda);
  }

  private createLambdaFunction(name: string) {
    const lambda = new NodejsFunction(this, name, {
      architecture: cdk.aws_lambda.Architecture.ARM_64,
      entry: path.join(__dirname, "../../src/index.ts"),
      environment: {},
      handler: "handler",
      runtime: Runtime.NODEJS_20_X,
      timeout: cdk.Duration.seconds(8),
    });

    return lambda;
  }

[vdantuluri-takeda]

Custom CDK Deployment is also failing Resource handler returned message: "Resource of type 'AWS::Logs::LogGroup' with identifier '{"/properties/LogGroupName":"/aws/lambda/xxx--CustomCDKBucketDeploymen-xxx"}' already exists." (RequestToken: xxxx, HandlerErrorCode: AlreadyExists)

[leandrodamascena]

I'm having the same problem with regular Lambda. It was working yesterday before updating to CDK 2.200.0