enhancement: enable async parquet read mode for WAL completion path#6967
Merged
zhxiaogg merged 7 commits intografana:mainfrom Apr 16, 2026
Merged
Conversation
Add TEMPO_WAL_ASYNC_IO env var to gate enabling parquet ReadModeAsync when opening WAL files during block completion. This allows parquet-go to issue concurrent ReadAt calls, improving I/O throughput. OS page cache handles buffering for local disk so no userspace BufferedReaderAt is needed.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces an opt-in async parquet read mode for WAL block completion by gating parquet.ReadModeAsync behind a new TEMPO_WAL_ASYNC_IO environment variable in the vParquet4/vParquet5 WAL open path.
Changes:
- Add
TEMPO_WAL_ASYNC_IOenv var constant to vParquet4/vParquet5 packages. - When the env var is set, open WAL parquet files with
parquet.FileReadMode(parquet.ReadModeAsync)during block completion. - Add iterator tests that exercise the WAL iterator with async IO enabled (vParquet4/vParquet5).
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tempodb/encoding/vparquet5/wal_block.go | Gate parquet async read mode for WAL parquet file opens via env var. |
| tempodb/encoding/vparquet4/wal_block.go | Same async read mode gating for vParquet4 WAL file opens. |
| tempodb/encoding/vparquet5/block_findtracebyid.go | Define EnvVarWALAsyncIO constant (used by WAL code). |
| tempodb/encoding/vparquet4/block_findtracebyid.go | Define EnvVarWALAsyncIO constant (used by WAL code). |
| tempodb/encoding/vparquet5/wal_block_test.go | Add test coverage for WAL iterator with async IO enabled. |
| tempodb/encoding/vparquet4/wal_block_test.go | Add test coverage for WAL iterator with async IO enabled. |
zhxiaogg
requested review from
carles-grafana,
electron0zero,
ie-pham,
javiermolinar,
mapno,
mattdurham,
oleg-kozlyuk-grafana,
ruslan-mikhailov,
stoewer,
yvrhdn and
zalegrala
as code owners
mattdurham
reviewed
Apr 14, 2026
mapno
approved these changes
Apr 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does:
Based on the test result, we saw slightly improvements on the WAL completion duration. Didn't find major issues with cpu/meme/query etc.
Which issue(s) this PR fixes:
Fixes N/A
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]