Skip to content

fix(api): reaggregate overview summaries after muting findings#10827

Merged
AdriiiPRodri merged 5 commits into
masterfrom
fix/api-overview-reaggregate-on-mute
Apr 22, 2026
Merged

fix(api): reaggregate overview summaries after muting findings#10827
AdriiiPRodri merged 5 commits into
masterfrom
fix/api-overview-reaggregate-on-mute

Conversation

@AdriiiPRodri
Copy link
Copy Markdown
Contributor

Context

After creating a mute rule, /finding-groups already reflected the newly-muted findings, but the Overview endpoints (/overviews/findings, /overviews/findings-severity, /overviews/services) kept showing the pre-mute totals. Those endpoints read from the pre-aggregated ScanSummary / DailySeveritySummary tables, which were not being recomputed after mute_historical_findings_task updated Finding.muted. reaggregate_all_finding_group_summaries_task only re-ran the finding-group aggregator, leaving the rest of the summaries stale until the next scan completed.

Description

Extend reaggregate_all_finding_group_summaries_task (already chained after mute_historical_findings_task in MuteRuleViewSet.create) to re-run the same per-scan pipeline that the scan-completion flow executes, on the latest completed scan per (provider, day) pair:

chain(
    perform_scan_summary_task.si(...),        # rebuilds ScanSummary
    group(
        aggregate_daily_severity_task.si(...),            # rebuilds DailySeveritySummary
        aggregate_finding_group_summaries_task.si(...),   # rebuilds FindingGroupDailySummary
    ),
)
  • DailySeveritySummary depends on ScanSummary, so the inner pipeline is a chain with a parallel fan-out for the two leaf aggregators.
  • Per-scan pipelines are still dispatched through an outer Celery group so wall-clock scales with the worker pool.
  • Celery task name is intentionally preserved (reaggregate-all-finding-group-summaries) so tasks already enqueued during a deploy keep resolving. Docstring and log message updated to describe the broader scope.

Unit tests for the task were updated to assert that all three sub-tasks (perform_scan_summary_task, aggregate_daily_severity_task, aggregate_finding_group_summaries_task) are dispatched for each de-duplicated (provider, day) scan.

Steps to review

  1. Run a scan that produces several FAIL findings for a provider.
  2. GET /api/v1/overviews/findings and note fail / muted totals.
  3. POST /api/v1/mute-rules selecting some of those finding IDs.
  4. Wait for the Celery overview queue to drain (or docker compose logs worker | grep reaggregate).
  5. GET /api/v1/overviews/findings again → muted should increase and fail decrease by the same amount.
  6. GET /api/v1/overviews/findings-severity and /api/v1/overviews/services → severity / service buckets should reflect the mute.
  7. GET /api/v1/finding-groupsfail_count / muted_count for affected groups should continue to reflect the mute (unchanged behavior).

Automated:

cd api && poetry run pytest src/backend/tasks/tests/test_tasks.py::TestReaggregateAllFindingGroupSummaries -x

Checklist

Community Checklist
  • This feature/issue is listed in here or roadmap.prowler.com
  • Is it assigned to me, if not, request it via the issue/feature in here or Prowler Community Slack

API

  • All issue/task requirements work as expected on the API
  • Endpoint response output (if applicable)
  • EXPLAIN ANALYZE output for new/modified queries or indexes (if applicable)
  • Performance test results (if applicable)
  • Any other relevant evidence of the implementation (if applicable)
  • Verify if API specs need to be regenerated.
  • Check if version updates are required.
  • Ensure new entries are added to api/CHANGELOG.md

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Extend `reaggregate_all_finding_group_summaries_task` to re-run the full
scan-completion aggregation pipeline (`ScanSummary`,
`DailySeveritySummary`, `FindingGroupDailySummary`) on the latest scan
per (provider, day) pair, so `/overviews/findings`,
`/overviews/findings-severity`, `/overviews/services` and the finding
groups endpoints all reflect the newly-muted findings without waiting
for the next scan.
@AdriiiPRodri AdriiiPRodri requested a review from a team as a code owner April 21, 2026 15:42
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 21, 2026

Conflict Markers Resolved

All conflict markers have been successfully resolved in this pull request.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 21, 2026

⚠️ Changes detected in the following folders without a corresponding update to the CHANGELOG.md:

  • mcp_server

Please add an entry to the corresponding CHANGELOG.md file to maintain a clear history of changes.

@AdriiiPRodri AdriiiPRodri added the backport-to-v5.24 Backport PR to the v5.24 branch label Apr 21, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 21, 2026

🔒 Container Security Scan

Image: prowler-api:25dea95
Last scan: 2026-04-22 08:39:15 UTC

📊 Vulnerability Summary

Severity Count
🔴 Critical 5
Total 5

4 package(s) affected

⚠️ Action Required

Critical severity vulnerabilities detected. These should be addressed before merging:

  • Review the detailed scan results
  • Update affected packages to patched versions
  • Consider using a different base image if updates are unavailable

📋 Resources:

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 21, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.51%. Comparing base (61a62fd) to head (685f09e).
⚠️ Report is 10 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10827      +/-   ##
==========================================
+ Coverage   93.48%   93.51%   +0.03%     
==========================================
  Files         228      228              
  Lines       32081    32230     +149     
==========================================
+ Hits        29990    30139     +149     
  Misses       2091     2091              
Flag Coverage Δ
api 93.51% <100.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
prowler ∅ <ø> (∅)
api 93.51% <100.00%> (+0.03%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

`aggregate_findings` now deletes the scan's existing `ScanSummary` rows
before `bulk_create`, so re-running against a scan that already has
summaries no longer violates the `unique_scan_summary` constraint. The
post-mute reaggregation pipeline dispatches this task against scans
whose summaries were written at scan completion; the prior plain INSERT
aborted the transaction and stopped the downstream chain from
recomputing `DailySeveritySummary` and `FindingGroupDailySummary`.
Copy link
Copy Markdown
Member

@jfagoagas jfagoagas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AdriiiPRodri AdriiiPRodri merged commit 1456def into master Apr 22, 2026
37 checks passed
@AdriiiPRodri AdriiiPRodri deleted the fix/api-overview-reaggregate-on-mute branch April 22, 2026 08:44
@prowler-bot prowler-bot added the was-backported The PR was successfully backported to the target branch label Apr 22, 2026
@prowler-bot
Copy link
Copy Markdown
Collaborator

💚 All backports created successfully

Status Branch Result
v5.24

Questions ?

Please refer to the Backport tool documentation and see the Github Action logs for details

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-v5.24 Backport PR to the v5.24 branch component/api was-backported The PR was successfully backported to the target branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants