Skip to content

test: Add end to end tests for quantile digest functions#27582

Open
allenshen13 wants to merge 1 commit intomasterfrom
add-qdigest-e2e-tests
Open

test: Add end to end tests for quantile digest functions#27582
allenshen13 wants to merge 1 commit intomasterfrom
add-qdigest-e2e-tests

Conversation

@allenshen13
Copy link
Copy Markdown
Member

@allenshen13 allenshen13 commented Apr 14, 2026

Description

Add end-to-end tests for the quantile digest (qdigest) functions on the native worker: qdigest_agg (all
three variants), merge, value_at_quantile, values_at_quantiles, and scale_qdigest.

Motivation and Context

Closes #26939. The qdigest functions are registered in Velox
but had no e2e coverage comparing Java vs native worker results.

Impact

Tests only. No user-facing or API changes.

Test Plan

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.
  • If adding new dependencies, verified they have an OpenSSF Scorecard score of 5.0 or higher (or obtained explicit TSC approval for lower scores).

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== NO RELEASE NOTE ==

Summary by Sourcery

Add end-to-end native worker coverage for qdigest aggregation and related functions to ensure consistency with Java execution.

Tests:

  • Add native worker tests covering qdigest_agg for bigint, double, and real inputs, including weighted and accuracy-parameter variants.
  • Add tests for qdigest merge, value_at_quantile, values_at_quantiles, and scale_qdigest behavior on the nation dataset, including null-handling cases.

@prestodb-ci prestodb-ci added the from:IBM PR from IBM label Apr 14, 2026
@allenshen13 allenshen13 requested review from a team as code owners April 14, 2026 05:40
@prestodb-ci prestodb-ci requested review from a team, imsayari404 and xin-zhang2 and removed request for a team April 14, 2026 05:40
@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented Apr 14, 2026

Reviewer's Guide

Adds an end-to-end native worker aggregation test covering all qdigest functions (aggregation variants, merge, value_at_quantile, values_at_quantiles, and scale_qdigest) to ensure parity with Java execution.

File-Level Changes

Change Details Files
Add a consolidated end-to-end test for qdigest aggregation and related functions in the native worker aggregation test suite.
  • Introduce testQDigestFunctions() to AbstractTestNativeAggregations to validate qdigest_agg over bigint, double, and real inputs
  • Add assertions for weighted and accuracy-parameterized qdigest_agg variants using value_at_quantile
  • Test merge of per-group qdigests and validate results via value_at_quantile and values_at_quantiles
  • Verify scale_qdigest behavior by scaling a qdigest and checking the quantile result
  • Cover null-handling behavior for qdigest_agg, including all-null input
presto-native-execution/src/test/java/com/facebook/presto/nativeworker/AbstractTestNativeAggregations.java

Assessment against linked issues

Issue Objective Addressed Explanation
#26939 Add end-to-end tests for the scale_qdigest quantile digest function.
#26939 Add end-to-end tests for other quantile digest functions that were missing coverage (e.g., qdigest_agg variants, merge, value_at_quantile, values_at_quantiles).

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="presto-native-execution/src/test/java/com/facebook/presto/nativeworker/AbstractTestNativeAggregations.java" line_range="361" />
<code_context>
+    public void testQDigestFunctions()
+    {
+        // bigint, double, real
+        assertQuery("SELECT value_at_quantile(qdigest_agg(nationkey), 0e0) FROM nation", "SELECT BIGINT '0'");
+        assertQuery("SELECT value_at_quantile(qdigest_agg(cast(nationkey as DOUBLE)), 0e0) FROM nation", "SELECT DOUBLE '0'");
+        assertQuery("SELECT value_at_quantile(qdigest_agg(cast(nationkey as REAL)), 0e0) FROM nation", "SELECT REAL '0'");
</code_context>
<issue_to_address>
**suggestion (testing):** Add coverage for non-endpoint quantiles (e.g. 0.5) to better validate qdigest accuracy semantics.

Current assertions only cover endpoint quantiles (0 or 1), where behavior is less interesting for an approximate structure like qdigest. Please add at least one assertion for an interior quantile (e.g., 0.25, 0.5, 0.9) on a non-trivial distribution to better exercise the core quantile calculation and native vs Java consistency.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

public void testQDigestFunctions()
{
// bigint, double, real
assertQuery("SELECT value_at_quantile(qdigest_agg(nationkey), 0e0) FROM nation", "SELECT BIGINT '0'");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Add coverage for non-endpoint quantiles (e.g. 0.5) to better validate qdigest accuracy semantics.

Current assertions only cover endpoint quantiles (0 or 1), where behavior is less interesting for an approximate structure like qdigest. Please add at least one assertion for an interior quantile (e.g., 0.25, 0.5, 0.9) on a non-trivial distribution to better exercise the core quantile calculation and native vs Java consistency.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 14, 2026

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@allenshen13 allenshen13 force-pushed the add-qdigest-e2e-tests branch from 9e303aa to a79bd21 Compare April 14, 2026 05:43
Copy link
Copy Markdown
Contributor

@pramodsatya pramodsatya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding these tests @allenshen13. Could you please move these to AbstractTestAggregationsNative in presto-native-tests so it has expanded test coverage? Some of these function tests are already covered there as well.

@allenshen13 allenshen13 force-pushed the add-qdigest-e2e-tests branch from a79bd21 to 2bf4773 Compare April 14, 2026 19:51
@allenshen13 allenshen13 force-pushed the add-qdigest-e2e-tests branch from 2bf4773 to cf77ba3 Compare April 15, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:IBM PR from IBM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add end to end tests for quantile digest functions

3 participants