Fix compactor missing compaction_objects_combined_total metric and not honoring max_bytes_per_trace #1420
Merged
mdisibio merged 2 commits intografana:mainfrom May 4, 2022
Merged
Conversation
…ax_bytes_per_trace again
mdisibio
requested review from
annanay25,
dgzlopes,
joe-elliott,
mapno,
yvrhdn and
zalegrala
as code owners
joe-elliott
approved these changes
May 3, 2022
Collaborator
joe-elliott
left a comment
joe-elliott
left a comment
There was a problem hiding this comment.
This compaction combine/logic has gotten quite difficult to a follow. This was an easy mistake to make.
This feels like a consequence of Tempo being between two different design patterns? The old "data agnostic" tempodb and the new trace specific tempodb.
The good news is that all of this slowly unwinds:
- Introduce Parquet blocks
- Drop support for writing v2 blocks and only read them
- Replace v2 format in wal
- Replace v2 format in distributor <-> ingester communication
- Delete v2
- 🌮
As we work through the above we can slowly push this mess into smaller and smaller corners and replace everything with the new block interfaces and formats. I think /pkg/model will always live as a way to translate the backend formats to OTLP, but eventually we will be able to gut it as well.
The future is bright!
mapno
pushed a commit
that referenced
this pull request
May 5, 2022
…t honoring max_bytes_per_trace (#1420) * Fix compactor to record compaction_objects_combined_total and honor max_bytes_per_trace again * changelog
joe-elliott
added a commit
that referenced
this pull request
Jun 13, 2022
…ers based on the trace id hashes instead of all healthy ingesters (#1484) * metrics-generator: don't inject X-Scope-OrgID header for single-tenant setups (#1417) * metrics-generator: don't inject X-Scope-OrgID header for single-tenant setups * Update CHANGELOG.md * Fix compactor missing compaction_objects_combined_total metric and not honoring max_bytes_per_trace (#1420) * Fix compactor to record compaction_objects_combined_total and honor max_bytes_per_trace again * changelog * distributor: prevent panics when concurrently calling to forwarder's… (#1422) * distributor: prevent panics when concurrently calling to forwarder's queueManager * Update PR number * Pin k6 version to v0.37.0 * update querier to get ingester instance based on trace id * rebase with main * added change log for the config * update config name as per PR comments * Doc tweak Signed-off-by: Joe Elliott <number101010@gmail.com> Co-authored-by: Koenraad Verheyden <koenraad.verheyden@grafana.com> Co-authored-by: Martin Disibio <martin.disibio@grafana.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Joe Elliott <number101010@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does:
This PR fixes some things that were broken by refactor #1351. (1) the
compaction_objects_combined_totalmetric is now populated again. (2) the per-tenant overridemax_bytes_per_traceis honored again, and the corresponding metrictempo_discarded_spans_total{reason="trace_too_large_to_compact"}populated.Which issue(s) this PR fixes:
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]