Create ESM Lambda Layer

[Lms24]

(extracted from #12409)

Since using the Sentry AWS Serverless SDK is far from easy in ESM-based Lambda functions, we should look into creating a layer for ESM or making our current layer compatible with both, CJS and ESM.

I outlined some of the prior investigation to this in #12322 and #12409.

One major blocker for a code-less Sentry SDK usage like in CJS is that our tryPatchHandler function we use to auto-wrap CJS lambda functions can't be applied to ESM Lambda functions (it uses require 😢). However, we can create a layer and tell people to manually init the SDK in code. Having the layer is still a huge improvement because users don't have to deploy their node_modules like when installing the SDK via NPM. h/t @andreiborza for pointing this out today!

ESM Layer v0 (no auto-wrapping)

• Create ESM layer that provides the SDK dependencies
• Document how to use layer with code changes
• Add flag in event payload indicating if event was sent from lambda layer or from NPM-installed pacakge

ESM layer v1 (auto-wrapping)

• Investigate why AWS Lambda Otel instrumentation doesn't work in ESM (+ possibly fix it)
• Find way to auto-wrap handler to permit code-less SDK setup
• Document code-less SDK setup via environment variables

Stretch Goals

• Adapt Sentry integration to use ESM version of layer
• Document how to use Sentry integration for automatic adding of Sentry to (multiple) ESM lambda functions
• Look into Supporting SENTRY_DEBUG
• Look into the AWS Lambda Extensions API (@sentry/serverless adds ~300ms to function duration #3051 (comment))

[Lms24]

Extracted this issue so we can link to it from docs that mention the ESM limitations around the layer

[paolostyle]

Is there any progress on this? Also I'd suggest implementing the integration using AWS Lambda Extensions, the issues mentioned in #3051 are very much present currently, Sentry adds a lot of overhead and it sounds like the Lambda Extensions is the solution to that.

[Lms24]

Hi, no progress on this at the moment, as we're busy with a lot of other issues.

I'm gonna tag @andreiborza if you get some time or need a break from Solid(Start). I gotta admit I had no idea that #3051 was receiving replies (this issue predates almost everyone working on the team by now). The unfortunate thing is that this is a hard problem to solve.

IIRC there are some limitations to the Lambda Extension API why we can't use it but I'll ask around a bit.

[microadam]

@Lms24 would love to hear any issues you are aware of with the Lamba Extensions API, as having implemented this: #3051 (comment)

We are able to use sentry without the noticeable delay on a fairly high traffic system. Its probably far from perfect (better batching could be introduced for example and currently the data is sent on the NEXT lambda invocation). Any details would be great :) (or just a link to where any discussion on this might be taking place now / in the future so I could follow along!)