Merged
Conversation
joe-elliott
reviewed
Mar 3, 2023
Collaborator
There was a problem hiding this comment.
Some other things we can try:
- We pass
common.DefaultSearchOptions()to all searches done on disk. this sets up reads of 1MB which is tuned for object storage. Parquet defaults to 4KB per read which is better tuned for disk access. Might be worth seeing what happens if leave the 4KB default. If you do this remember the potential issue with ReadRange() opening a file everytime! - wal is read synchronously and backend blocks are read async. if one is outperforming the other we could use that as a hint about what works best for performance on local disk ingester search.
I like the cleanup in this PR. Once you drop draft will give it a proper review.
mapno
requested review from
annanay25,
mdisibio,
yvrhdn and
zalegrala
as code owners
joe-elliott
approved these changes
Mar 8, 2023
Collaborator
joe-elliott
left a comment
joe-elliott
left a comment
There was a problem hiding this comment.
lgtm. some good cleanup here.
does locking only once in SearchTagValuesV2 improve performance? or just simplifies code?
Contributor
Author
Can't say for sure. Most of the improvements came after we stopped caching files. I think it's a nice improvement anyway. |
mdisibio
pushed a commit
to mdisibio/tempo
that referenced
this pull request
Apr 18, 2023
* Tag search improvements * Not reuse parquet files for search * Changelog
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does:
This PR contains a few fixes to tag search in vParquet.
searchTagValues()SearchTagValuesV2blocksMtxlock only once inSearchTagValuesV2Which issue(s) this PR fixes:
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]