Skip to content

vParquet5: virtual span row numbers#5943

Merged
stoewer merged 18 commits intografana:mainfrom
stoewer:vp5-virtual-span-row-numbers
Nov 27, 2025
Merged

vParquet5: virtual span row numbers#5943
stoewer merged 18 commits intografana:mainfrom
stoewer:vp5-virtual-span-row-numbers

Conversation

@stoewer
Copy link
Copy Markdown
Contributor

@stoewer stoewer commented Nov 14, 2025 •
edited
Loading

What this PR does:
Calculate virtual span row numbers for queries containing only resource attributes

Which issue(s) this PR fixes:
Fixes #5811

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@stoewer
Copy link
Copy Markdown
Contributor Author

stoewer commented Nov 14, 2025

Benchmark results:

goos: linux
goarch: amd64
pkg: github.com/grafana/tempo/tempodb/encoding/vparquet5
cpu: AMD Ryzen AI 9 HX PRO 370 w/ Radeon 890M       
                                                   │ bench-virtrows-01-vp5.txt │   bench-virtrows-01-vp5-virt.txt   │
                                                   │          sec/op           │   sec/op     vs base               │
BackendBlockTraceQL/spanAttValMatch-12                             255.9m ± 4%   256.6m ± 2%        ~ (p=0.574 n=8)
BackendBlockTraceQL/spanAttValMatchFew-12                          118.7m ± 2%   118.2m ± 1%        ~ (p=0.798 n=8)
BackendBlockTraceQL/unscopedAttValMatch-12                         954.2m ± 2%   870.3m ± 2%   -8.79% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttValMatch-12                         543.9m ± 1%   503.3m ± 2%   -7.47% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttValNoMatch-12                       10.89m ± 3%   11.19m ± 2%   +2.75% (p=0.003 n=8)
BackendBlockTraceQL/resourceAttIntrinsicMatch-12                   93.43m ± 2%   89.46m ± 2%   -4.25% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttIntrinsicNoMatch-12                 6.206m ± 6%   6.094m ± 5%        ~ (p=0.505 n=8)
BackendBlockTraceQL/mixedValNoMatch-12                             545.1m ± 2%   462.0m ± 3%  -15.24% (p=0.000 n=8)
BackendBlockTraceQL/mixedValMixedMatchAnd-12                       6.730m ± 4%   6.633m ± 4%        ~ (p=0.798 n=8)
BackendBlockTraceQL/mixedValMixedMatchOr-12                        527.5m ± 2%   432.0m ± 2%  -18.10% (p=0.000 n=8)
BackendBlockTraceQL/count-12                                        1.737 ± 1%    1.666 ± 2%   -4.08% (p=0.000 n=8)
BackendBlockTraceQL/struct-12                                       1.728 ± 2%    1.640 ± 2%   -5.12% (p=0.000 n=8)
BackendBlockTraceQL/||-12                                          848.5m ± 2%   769.7m ± 1%   -9.28% (p=0.000 n=8)
BackendBlockTraceQL/mixed-12                                       217.0m ± 2%   216.0m ± 2%        ~ (p=0.442 n=8)
BackendBlockTraceQL/complex-12                                     430.0m ± 1%   430.2m ± 2%        ~ (p=0.878 n=8)
BackendBlockTraceQL/select-12                                       1.162 ± 2%    1.117 ± 3%   -3.89% (p=0.000 n=8)
geomean                                                            228.0m        216.6m        -5.00%

                                                   │ bench-virtrows-01-vp5.txt │   bench-virtrows-01-vp5-virt.txt    │
                                                   │            B/s            │     B/s       vs base               │
BackendBlockTraceQL/spanAttValMatch-12                            390.1Mi ± 4%   389.2Mi ± 4%        ~ (p=0.574 n=8)
BackendBlockTraceQL/spanAttValMatchFew-12                         841.0Mi ± 2%   844.3Mi ± 2%        ~ (p=0.798 n=8)
BackendBlockTraceQL/unscopedAttValMatch-12                        112.5Mi ± 2%   122.7Mi ± 2%   +9.04% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttValMatch-12                        184.5Mi ± 2%   198.3Mi ± 2%   +7.44% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttValNoMatch-12                      794.1Mi ± 2%   772.9Mi ± 2%   -2.67% (p=0.003 n=8)
BackendBlockTraceQL/resourceAttIntrinsicMatch-12                  563.4Mi ± 3%   585.3Mi ± 2%   +3.89% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttIntrinsicNoMatch-12                282.1Mi ± 3%   287.3Mi ± 4%        ~ (p=0.505 n=8)
BackendBlockTraceQL/mixedValNoMatch-12                            26.04Mi ± 2%   29.44Mi ± 4%  +13.04% (p=0.000 n=8)
BackendBlockTraceQL/mixedValMixedMatchAnd-12                      1.026Gi ± 5%   1.041Gi ± 5%        ~ (p=0.798 n=8)
BackendBlockTraceQL/mixedValMixedMatchOr-12                       21.72Mi ± 3%   25.14Mi ± 2%  +15.77% (p=0.000 n=8)
BackendBlockTraceQL/count-12                                      57.72Mi ± 1%   59.82Mi ± 2%   +3.63% (p=0.000 n=8)
BackendBlockTraceQL/struct-12                                     66.75Mi ± 2%   69.98Mi ± 2%   +4.84% (p=0.000 n=8)
BackendBlockTraceQL/||-12                                         119.0Mi ± 1%   130.4Mi ± 1%   +9.59% (p=0.000 n=8)
BackendBlockTraceQL/mixed-12                                      535.9Mi ± 2%   538.4Mi ± 3%        ~ (p=0.442 n=8)
BackendBlockTraceQL/complex-12                                    468.5Mi ± 2%   468.4Mi ± 2%        ~ (p=0.903 n=8)
BackendBlockTraceQL/select-12                                     72.53Mi ± 3%   74.94Mi ± 2%   +3.32% (p=0.000 n=8)
geomean                                                           194.7Mi        203.2Mi        +4.37%

                                                   │ bench-virtrows-01-vp5.txt │   bench-virtrows-01-vp5-virt.txt   │
                                                   │         MB_io/op          │  MB_io/op   vs base                │
BackendBlockTraceQL/spanAttValMatch-12                              104.7 ± 0%   104.7 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/spanAttValMatchFew-12                           104.7 ± 0%   104.7 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/unscopedAttValMatch-12                          112.6 ± 0%   112.0 ± 0%  -0.53% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttValMatch-12                          105.2 ± 0%   104.6 ± 0%  -0.57% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttValNoMatch-12                        9.067 ± 0%   9.067 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/resourceAttIntrinsicMatch-12                    55.18 ± 0%   54.90 ± 0%  -0.51% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttIntrinsicNoMatch-12                  1.835 ± 0%   1.835 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/mixedValNoMatch-12                              14.89 ± 0%   14.26 ± 0%  -4.23% (p=0.000 n=8)
BackendBlockTraceQL/mixedValMixedMatchAnd-12                        7.414 ± 0%   7.414 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/mixedValMixedMatchOr-12                         12.01 ± 0%   11.39 ± 0%  -5.16% (p=0.000 n=8)
BackendBlockTraceQL/count-12                                        105.1 ± 0%   104.5 ± 0%  -0.57% (p=0.000 n=8)
BackendBlockTraceQL/struct-12                                       120.9 ± 0%   120.3 ± 0%  -0.50% (p=0.000 n=8)
BackendBlockTraceQL/||-12                                           105.9 ± 0%   105.3 ± 0%  -0.57% (p=0.000 n=8)
BackendBlockTraceQL/mixed-12                                        122.0 ± 0%   122.0 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/complex-12                                      211.3 ± 0%   211.3 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/select-12                                       88.36 ± 0%   87.73 ± 0%  -0.71% (p=0.000 n=8)
geomean                                                             46.54        46.15       -0.85%
¹ all samples are equal

                                                   │ bench-virtrows-01-vp5.txt │   bench-virtrows-01-vp5-virt.txt    │
                                                   │         spans/op          │  spans/op    vs base                │
BackendBlockTraceQL/spanAttValMatch-12                           18.28k ± 0%     18.28k ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/spanAttValMatchFew-12                        2.364k ± 0%     2.364k ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/unscopedAttValMatch-12                       49.93k ± 0%     49.93k ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/resourceAttValMatch-12                       49.93k ± 0%     53.77k ± 0%  +7.69% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttValNoMatch-12                      0.000 ± 0%      0.000 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/resourceAttIntrinsicMatch-12                 4.586k ± 0%     4.587k ± 0%  +0.02% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttIntrinsicNoMatch-12                0.000 ± 0%      0.000 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/mixedValNoMatch-12                            0.000 ± 0%      0.000 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/mixedValMixedMatchAnd-12                      0.000 ± 0%      0.000 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/mixedValMixedMatchOr-12                       0.000 ± 0%      0.000 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/count-12                                     231.9k ± 0%     231.9k ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/struct-12                                     84.00 ± 0%      84.00 ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/||-12                                        11.64k ± 0%     11.64k ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/mixed-12                                     1.910k ± 0%     1.910k ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/complex-12                                   3.571k ± 0%     3.571k ± 0%       ~ (p=1.000 n=8) ¹
BackendBlockTraceQL/select-12                                    1.184k ± 0%     1.189k ± 0%  +0.42% (p=0.000 n=8)
geomean                                                                      ²                +0.49%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                                   │ bench-virtrows-01-vp5.txt │    bench-virtrows-01-vp5-virt.txt    │
                                                   │           B/op            │     B/op       vs base               │
BackendBlockTraceQL/spanAttValMatch-12                           103.6Mi ±  2%   103.4Mi ±  1%        ~ (p=0.721 n=8)
BackendBlockTraceQL/spanAttValMatchFew-12                        17.47Mi ±  2%   18.44Mi ±  6%   +5.55% (p=0.002 n=8)
BackendBlockTraceQL/unscopedAttValMatch-12                       328.9Mi ±  1%   327.3Mi ±  2%        ~ (p=0.195 n=8)
BackendBlockTraceQL/resourceAttValMatch-12                       322.3Mi ±  1%   325.1Mi ±  0%   +0.86% (p=0.005 n=8)
BackendBlockTraceQL/resourceAttValNoMatch-12                     12.72Mi ±  1%   12.73Mi ±  1%        ~ (p=0.505 n=8)
BackendBlockTraceQL/resourceAttIntrinsicMatch-12                 22.00Mi ±  1%   22.31Mi ±  2%        ~ (p=0.083 n=8)
BackendBlockTraceQL/resourceAttIntrinsicNoMatch-12               7.044Mi ±  0%   7.002Mi ±  1%   -0.60% (p=0.050 n=8)
BackendBlockTraceQL/mixedValNoMatch-12                           19.81Mi ± 27%   13.52Mi ± 40%  -31.77% (p=0.001 n=8)
BackendBlockTraceQL/mixedValMixedMatchAnd-12                     7.523Mi ±  1%   7.508Mi ±  1%        ~ (p=0.442 n=8)
BackendBlockTraceQL/mixedValMixedMatchOr-12                      14.48Mi ± 11%   12.93Mi ± 48%        ~ (p=0.442 n=8)
BackendBlockTraceQL/count-12                                     2.069Gi ±  1%   2.057Gi ±  1%   -0.56% (p=0.003 n=8)
BackendBlockTraceQL/struct-12                                    191.1Mi ±  8%   193.0Mi ±  2%        ~ (p=0.442 n=8)
BackendBlockTraceQL/||-12                                        499.9Mi ±  1%   496.5Mi ±  2%        ~ (p=0.234 n=8)
BackendBlockTraceQL/mixed-12                                     24.62Mi ±  6%   24.47Mi ±  6%        ~ (p=0.645 n=8)
BackendBlockTraceQL/complex-12                                   58.96Mi ±  7%   58.98Mi ±  2%        ~ (p=0.878 n=8)
BackendBlockTraceQL/select-12                                    1.464Gi ±  0%   1.454Gi ±  1%   -0.70% (p=0.000 n=8)
geomean                                                          70.34Mi         68.40Mi         -2.76%

                                                   │ bench-virtrows-01-vp5.txt │  bench-virtrows-01-vp5-virt.txt   │
                                                   │         allocs/op         │  allocs/op   vs base              │
BackendBlockTraceQL/spanAttValMatch-12                             898.9k ± 0%   898.9k ± 0%       ~ (p=0.425 n=8)
BackendBlockTraceQL/spanAttValMatchFew-12                          159.4k ± 0%   159.4k ± 0%  +0.01% (p=0.001 n=8)
BackendBlockTraceQL/unscopedAttValMatch-12                         2.625M ± 0%   2.624M ± 0%  -0.06% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttValMatch-12                         2.628M ± 0%   2.624M ± 0%  -0.14% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttValNoMatch-12                       70.61k ± 0%   70.61k ± 0%  +0.00% (p=0.008 n=8)
BackendBlockTraceQL/resourceAttIntrinsicMatch-12                   256.8k ± 0%   244.4k ± 0%  -4.83% (p=0.000 n=8)
BackendBlockTraceQL/resourceAttIntrinsicNoMatch-12                 70.54k ± 0%   70.54k ± 0%       ~ (p=1.000 n=8)
BackendBlockTraceQL/mixedValNoMatch-12                             73.11k ± 0%   71.36k ± 0%  -2.39% (p=0.009 n=8)
BackendBlockTraceQL/mixedValMixedMatchAnd-12                       70.78k ± 0%   70.78k ± 0%       ~ (p=0.327 n=8)
BackendBlockTraceQL/mixedValMixedMatchOr-12                        73.06k ± 0%   71.36k ± 0%  -2.32% (p=0.010 n=8)
BackendBlockTraceQL/count-12                                       13.14M ± 0%   13.14M ± 0%  -0.02% (p=0.000 n=8)
BackendBlockTraceQL/struct-12                                      1.120M ± 0%   1.118M ± 0%  -0.15% (p=0.050 n=8)
BackendBlockTraceQL/||-12                                          2.760M ± 0%   2.758M ± 0%  -0.06% (p=0.010 n=8)
BackendBlockTraceQL/mixed-12                                       201.8k ± 0%   201.8k ± 0%       ~ (p=0.442 n=8)
BackendBlockTraceQL/complex-12                                     398.0k ± 0%   398.0k ± 0%       ~ (p=0.169 n=8)
BackendBlockTraceQL/select-12                                      6.570M ± 0%   6.570M ± 0%  -0.01% (p=0.000 n=8)
geomean

@stoewer stoewer marked this pull request as ready for review November 14, 2025 02:58
joe-elliott
joe-elliott reviewed Nov 14, 2025
View reviewed changes
Comment thread tempodb/encoding/vparquet5/block_traceql.go
Comment thread tempodb/encoding/vparquet5/row_number_iterator.go
Comment thread tempodb/encoding/vparquet5/row_number_iterator.go Outdated
joe-elliott
joe-elliott reviewed Nov 14, 2025
View reviewed changes
Comment thread tempodb/encoding/vparquet5/row_number_iterator.go
joe-elliott
joe-elliott approved these changes Nov 26, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@joe-elliott joe-elliott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spoke offline and understand the choices made in the virtual row number iterator. Seems like a few bugs were fixed and we're ready to merge!

@stoewer stoewer merged commit 7b00d6c into grafana:main Nov 27, 2025
21 checks passed
@knylander-grafana knylander-grafana mentioned this pull request Jan 9, 2026
Merged
3 tasks
@stoewer stoewer mentioned this pull request Feb 10, 2026
Merged
3 tasks
@stoewer stoewer deleted the vp5-virtual-span-row-numbers branch February 12, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@joe-elliott joe-elliott joe-elliott approved these changes

@mdisibio mdisibio Awaiting requested review from mdisibio mdisibio is a code owner

@mapno mapno Awaiting requested review from mapno mapno is a code owner

@yvrhdn yvrhdn Awaiting requested review from yvrhdn yvrhdn is a code owner

@zalegrala zalegrala Awaiting requested review from zalegrala zalegrala is a code owner

@electron0zero electron0zero Awaiting requested review from electron0zero electron0zero is a code owner

@ie-pham ie-pham Awaiting requested review from ie-pham ie-pham is a code owner

@javiermolinar javiermolinar Awaiting requested review from javiermolinar javiermolinar is a code owner

@carles-grafana carles-grafana Awaiting requested review from carles-grafana carles-grafana is a code owner

@ruslan-mikhailov ruslan-mikhailov Awaiting requested review from ruslan-mikhailov ruslan-mikhailov is a code owner

@mattdurham mattdurham Awaiting requested review from mattdurham mattdurham is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

vParquet5: virtual span row numbers

3 participants

@stoewer @joe-elliott @mdisibio