Skip to content

vParquet5: remove well-known attribute columns#5696

Merged
stoewer merged 25 commits intografana:mainfrom
stoewer:vp5-remove-well-known
Oct 17, 2025
Merged

vParquet5: remove well-known attribute columns#5696
stoewer merged 25 commits intografana:mainfrom
stoewer:vp5-remove-well-known

Conversation

@stoewer
Copy link
Copy Markdown
Contributor

@stoewer stoewer commented Sep 30, 2025

What this PR does:

This PR consists of the following changes:

  • Remove the existing columns for well-known attribute columns
  • Apply a default dedicated attribute column config when no dedicated columns are present
  • Encodings vp4 and vp3 ignore dedicated column configs that match existing well-known columns. This helps to avoid conflicts with the new default dedicated columns in older encodings
  • Use LRU cache for unmarshaled dedicated columns and avoid allocations for cache lookup: This is a small optimization to compensate for the larger and more frequently used dedicated column configs. I plan to implement further optimizations in a separate PR, if this becomes a problem.

Which issue(s) this PR fixes:
Contributes to #5665

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@stoewer stoewer force-pushed the vp5-remove-well-known branch from 89d5b67 to b65229b Compare October 1, 2025 04:51
@stoewer stoewer force-pushed the vp5-remove-well-known branch from b65229b to 4481531 Compare October 1, 2025 22:47
@stoewer stoewer force-pushed the vp5-remove-well-known branch from 7b9c44d to d44c059 Compare October 3, 2025 00:23
Copy link
Copy Markdown
Contributor

@mdisibio mdisibio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking really good, I think this migration path for vp4 to vp5 without performance penalty or interruption is great. Left a few qs and comments.

Comment thread tempodb/encoding/vparquet3/dedicated_columns.go Outdated
Comment thread tempodb/encoding/vparquet3/dedicated_columns.go Outdated
Comment thread tempodb/backend/dedicated_columns_cache.go Outdated
Comment thread tempodb/backend/dedicated_columns_cache.go
Comment thread tempodb/encoding/vparquet3/dedicated_columns.go
@stoewer stoewer merged commit c5b4b55 into grafana:main Oct 17, 2025
38 of 39 checks passed
@stoewer stoewer deleted the vp5-remove-well-known branch February 12, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants