Description
Description of the feature request
It would be great if the logger could collect some unwritten logs (maybe a configurable amount that defaults to 10 or something) and if there is an error
logged it would also write any logs that were not already written.
Problem statement
I've always hated that the debug logs are never available when I need them. They are typically turned off in a production system, but when there is an error I'd like to have information from those logs. I don't want them most of the time, so really only when an error occurs. Sampling isn't a good option because the error may be intermittent and the logs will not be present for the relevant time.
Summary of the feature
Basically, at the point when the decision is made whether to log it would either log (current functionality) or store the log. If there was an error log it would log the error as well as everything stored (and flush the store).
Code examples
Benefits for you and the wider AWS community
Faster time to resolution on errors
Describe alternatives you've considered
I've looked at doing something like this myself but haven't tried it yet. As I mentioned, sampling doesn't accomplish what I'm looking for.
Additional context
Related issues, RFCs
Sub-issues
Sub-issues
- Manage this item control shift u
- Manage this item control shift u
- Manage this item control shift u
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Activity
flochaz commentedon Jan 28, 2022
Thx for your suggestion ! Interesting idea that can definitely help reducing logging cost while having a verbose output on error. we will look into it .
ijemmy commentedon Feb 14, 2022
I like this idea very much. It addresses my main use case for sampling rate. But it's more cost-efficient and we don't miss anything.
I suppose usage will be like this?
In this case, the client needs to make sure that they catch any error and print out the log to get it flushed out.
It can be used with Middy's error-logger, to ensure that uncaught error are always handled.
jasonwadsworth commentedon Feb 23, 2022
Yes, it is definitely dependent on the code catching and logging an error.
Came across this repo, which is doing what I'm talking about here. Doesn't seem too bad in this example, but I haven't looked at the logging code in this repo to see how complex it might be here.
ijemmy commentedon Mar 15, 2022
There is a blog about this. Useful for implementation reference: https://dev.to/aws-builders/saving-on-aws-lambda-amazon-cloudwatch-logs-costs-51od
Notice the
clear()
function or the logs can leak to the next Lambda call.[-](logger): Include Unwritten Log Entries On Error[/-][+]Feature (logger): Include Unwritten Log Entries On Error[/+]buggy commentedon Aug 17, 2022
I really like this idea.
Could I suggest using
levelOnError
instead oflogAllLevelsOnError
? This would provide additional flexibility by allowing the developer to set the log level on error instead of assuming they want everything.Use case:
level
toWARN
andlevelOnError
toINFO
.WARN
for warnings that should always be loggedINFO
for additional information (like the event) that will allow debugging when an error occursDEBUG
for low level debugging (i.e. logging each iteration of a loop)Why?
Frequently I only need a little extra information, like the event, to be logged on error. By allowing the developer to set the log level on error I can use
INFO
for the stuff I want to see when something goes wrong but still haveDEBUG
littered through my code for when I need to do development / debugging. This is particularly helpful if you only keep the last N items to be written on error because I want some things I log (INFO
) but not others (DEBUG
).22 remaining items
dreamorosi commentedon Feb 10, 2025
We have just concluded a RFC for a version of this feature that we believe will unlock this use case, you can find more details here: #3410 (start from this comment).
We'll work on this feature in the next 2-3 weeks.
dreamorosi commentedon Mar 7, 2025
This was released in v2.16.0
github-actions commentedon Mar 7, 2025
This issue is now closed. Please be mindful that future comments are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.