Queriers: Ignore context canceled#2440
Merged
joe-elliott merged 3 commits intografana:mainfrom May 8, 2023
Merged
Conversation
Signed-off-by: Joe Elliott <number101010@gmail.com>
joe-elliott
requested review from
annanay25,
electron0zero,
ie-pham,
mapno,
mdisibio,
stoewer,
yvrhdn and
zalegrala
as code owners
ie-pham
approved these changes
May 8, 2023
ie-pham
reviewed
May 8, 2023
Signed-off-by: Joe Elliott <number101010@gmail.com>
zalegrala
reviewed
May 8, 2023
| * [ENHANCEMENT] Add span filtering to spanmetrics processor [#2274](https://github.com/grafana/tempo/pull/2274) (@zalegrala) | ||
| * [BUGFIX] tempodb integer divide by zero error [#2167](https://github.com/grafana/tempo/issues/2167) (@kroksys) | ||
| * [CHANGE] **Breaking Change** Rename s3.insecure_skip_verify [#???](https://github.com/grafana/tempo/pull/???) (@zalegrala) | ||
| * [CHANGE] **Breaking Change** Rename s3.insecure_skip_verify [#2407](https://github.com/grafana/tempo/pull/2407) (@zalegrala) |
zalegrala
approved these changes
May 8, 2023
Contributor
zalegrala
left a comment
zalegrala
left a comment
There was a problem hiding this comment.
This will clean up the errors a bit and seems reasonable to me.
pull Bot
pushed a commit
to sonvt1710/tempo
that referenced
this pull request
May 8, 2023
* ignore context canceled Signed-off-by: Joe Elliott <number101010@gmail.com> * changelog Signed-off-by: Joe Elliott <number101010@gmail.com> * changelog fix Signed-off-by: Joe Elliott <number101010@gmail.com> --------- Signed-off-by: Joe Elliott <number101010@gmail.com>
pull Bot
pushed a commit
to sonvt1710/tempo
that referenced
this pull request
May 8, 2023
* ignore context canceled Signed-off-by: Joe Elliott <number101010@gmail.com> * changelog Signed-off-by: Joe Elliott <number101010@gmail.com> * changelog fix Signed-off-by: Joe Elliott <number101010@gmail.com> --------- Signed-off-by: Joe Elliott <number101010@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does:
Changes the querier to ignore context canceled errors and return 200s. Tempo cancels context as part of normal operation. On the trace by id path, if hedging is turned on, then context will be canceled for other the slower requests. On the search path we commonly kick off thousands of jobs most of which are canceled when the search limit is hit.
Before this change the queriers would spam the logs with "context canceled" errors and metric tons of failed requests.
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]