Skip to content

Implementing Parquet Queryable with fallback #6743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
May 23, 2025

Conversation

alanprot
Copy link
Member

@alanprot alanprot commented May 14, 2025

What this PR does:

Introduces a new Queryable implementation to support querying blocks stored in Parquet format. When enabled, this queryable will attempt to read from Parquet files first and fall back to the existing BlocksStoreQueryable (i.e., store-gateway) if the Parquet files are not available.

This feature is gated behind the -querier.query-parquet-files flag (default: false). When the flag is disabled, this change is a complete NOOP and preserves existing behavior.

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • [Not Yet] Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@alanprot alanprot force-pushed the parquet-queryable branch 5 times, most recently from 24b597f to 5b6c823 Compare May 21, 2025 17:18
@alanprot alanprot marked this pull request as ready for review May 21, 2025 17:18
@dosubot dosubot bot added component/querier go Pull requests that update Go code storage/blocks Blocks storage engine sync-vendor labels May 21, 2025
@alanprot alanprot requested a review from yeya24 May 21, 2025 18:07
@alanprot alanprot force-pushed the parquet-queryable branch 4 times, most recently from 766009a to cf2fe6f Compare May 22, 2025 00:23
@alanprot alanprot force-pushed the parquet-queryable branch from cf2fe6f to 2d24435 Compare May 22, 2025 00:40
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add an integration test?

Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM

@alanprot alanprot force-pushed the parquet-queryable branch 4 times, most recently from b676e11 to 7df4ae5 Compare May 23, 2025 15:11
@alanprot alanprot force-pushed the parquet-queryable branch 3 times, most recently from 6fafb03 to f68780b Compare May 23, 2025 15:18
@alanprot alanprot force-pushed the parquet-queryable branch from f68780b to cef5e5c Compare May 23, 2025 15:22
@alanprot alanprot merged commit d103475 into cortexproject:master May 23, 2025
17 checks passed
@alanprot alanprot deleted the parquet-queryable branch May 23, 2025 15:51
@xiu xiu mentioned this pull request May 31, 2025
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/querier go Pull requests that update Go code size/XL storage/blocks Blocks storage engine sync-vendor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants