Skip to content

perf(local): avoid loading the entire advisory unless it will actually be used#2450

Merged
another-rex merged 4 commits into
google:mainfrom
ackama:perf/optimize-check
Jan 9, 2026
Merged

perf(local): avoid loading the entire advisory unless it will actually be used#2450
another-rex merged 4 commits into
google:mainfrom
ackama:perf/optimize-check

Conversation

@G-Rath
Copy link
Copy Markdown
Collaborator

@G-Rath G-Rath commented Jan 8, 2026

Right now we always parse advisories even if we don't end up loading them into the database, which is relatively expensive at scale so now we use gjson to extract the subset of data we need from the raw bytes to determine if the advisory is relevant before we do the advisory parsing.

This is especially useful for databases with a high amount of MAL advisories since their packages are very rare, such as the NPM database (which has 209647 MAL advisories out of a total of 214057 advisories) - before this it takes about 10 seconds to do a scan, whereas after this optimization it takes about 3 seconds

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jan 8, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.80%. Comparing base (25eccbf) to head (b6cb3e2).
⚠️ Report is 33 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2450   +/-   ##
=======================================
  Coverage   67.79%   67.80%           
=======================================
  Files         172      172           
  Lines       13301    13305    +4     
=======================================
+ Hits         9018     9022    +4     
  Misses       3573     3573           
  Partials      710      710           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@G-Rath G-Rath force-pushed the perf/optimize-check branch from 720b348 to 4671f6e Compare January 9, 2026 00:54
@G-Rath G-Rath force-pushed the perf/optimize-check branch from 4671f6e to b6cb3e2 Compare January 9, 2026 01:20
@G-Rath G-Rath marked this pull request as ready for review January 9, 2026 01:21
Copy link
Copy Markdown
Collaborator

@another-rex another-rex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

@another-rex another-rex merged commit 9c36a12 into google:main Jan 9, 2026
17 checks passed
@another-rex another-rex deleted the perf/optimize-check branch January 9, 2026 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants