fix(api): reaggregate overview summaries after muting findings#10827
Conversation
Extend `reaggregate_all_finding_group_summaries_task` to re-run the full scan-completion aggregation pipeline (`ScanSummary`, `DailySeveritySummary`, `FindingGroupDailySummary`) on the latest scan per (provider, day) pair, so `/overviews/findings`, `/overviews/findings-severity`, `/overviews/services` and the finding groups endpoints all reflect the newly-muted findings without waiting for the next scan.
|
✅ Conflict Markers Resolved All conflict markers have been successfully resolved in this pull request. |
|
Please add an entry to the corresponding |
🔒 Container Security ScanImage: 📊 Vulnerability Summary
4 package(s) affected
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #10827 +/- ##
==========================================
+ Coverage 93.48% 93.51% +0.03%
==========================================
Files 228 228
Lines 32081 32230 +149
==========================================
+ Hits 29990 30139 +149
Misses 2091 2091
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
`aggregate_findings` now deletes the scan's existing `ScanSummary` rows before `bulk_create`, so re-running against a scan that already has summaries no longer violates the `unique_scan_summary` constraint. The post-mute reaggregation pipeline dispatches this task against scans whose summaries were written at scan completion; the prior plain INSERT aborted the transaction and stopped the downstream chain from recomputing `DailySeveritySummary` and `FindingGroupDailySummary`.
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation and see the Github Action logs for details |

Context
After creating a mute rule,
/finding-groupsalready reflected the newly-muted findings, but the Overview endpoints (/overviews/findings,/overviews/findings-severity,/overviews/services) kept showing the pre-mute totals. Those endpoints read from the pre-aggregatedScanSummary/DailySeveritySummarytables, which were not being recomputed aftermute_historical_findings_taskupdatedFinding.muted.reaggregate_all_finding_group_summaries_taskonly re-ran the finding-group aggregator, leaving the rest of the summaries stale until the next scan completed.Description
Extend
reaggregate_all_finding_group_summaries_task(already chained aftermute_historical_findings_taskinMuteRuleViewSet.create) to re-run the same per-scan pipeline that the scan-completion flow executes, on the latest completed scan per(provider, day)pair:DailySeveritySummarydepends onScanSummary, so the inner pipeline is a chain with a parallel fan-out for the two leaf aggregators.groupso wall-clock scales with the worker pool.reaggregate-all-finding-group-summaries) so tasks already enqueued during a deploy keep resolving. Docstring and log message updated to describe the broader scope.Unit tests for the task were updated to assert that all three sub-tasks (
perform_scan_summary_task,aggregate_daily_severity_task,aggregate_finding_group_summaries_task) are dispatched for each de-duplicated(provider, day)scan.Steps to review
GET /api/v1/overviews/findingsand notefail/mutedtotals.POST /api/v1/mute-rulesselecting some of those finding IDs.overviewqueue to drain (ordocker compose logs worker | grep reaggregate).GET /api/v1/overviews/findingsagain →mutedshould increase andfaildecrease by the same amount.GET /api/v1/overviews/findings-severityand/api/v1/overviews/services→ severity / service buckets should reflect the mute.GET /api/v1/finding-groups→fail_count/muted_countfor affected groups should continue to reflect the mute (unchanged behavior).Automated:
Checklist
Community Checklist
API
License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.