Skip to content

feat: merge extractor and detector flags into plugins #2146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
another-rex merged 11 commits into from
Aug 7, 2025
Merged

feat: merge extractor and detector flags into plugins #2146

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
480ebee
refactor: use `plugin.Plugin` instead of `extractor.Extractor`
G-Rath Jul 23, 2025
e076cca
refactor: use `plugin.Plugin` instead of `filesystem.Extractor`
G-Rath Aug 6, 2025
059cc89
refactor: move extractor building into `scalibrplugin` package
G-Rath Aug 6, 2025
ae5b0af
refactor: move tests into their own package
G-Rath Jul 22, 2025
e576a40
refactor: merge extractor and detector resolvers
G-Rath Aug 6, 2025
0a93c30
refactor: remove old parser functions
G-Rath Aug 6, 2025
9c54258
feat: resolve extractors and detectors at the same time
G-Rath Aug 6, 2025
2b15315
feat: merge extractors and detectors flags
G-Rath Aug 6, 2025
ad6168b
refactor: clean up references to plugins
G-Rath Aug 6, 2025
15b89db
refactor: remove `BuildExtractors` and such
G-Rath Aug 6, 2025
677cb71
test: add snapshot case for detectors preset
G-Rath Aug 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 4 additions & 12 deletions cmd/osv-scanner/internal/helper/flags.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,21 +168,13 @@ func BuildCommonScanFlags(defaultExtractors []string) []cli.Flag {
Value: &allowedLicencesFlag{},
},
&cli.StringSliceFlag{
Name: "experimental-extractors",
Usage: "list of specific extractors and presets of extractors to use",
Name: "experimental-plugins",
Usage: "list of specific plugins and presets of plugins to use",
Value: defaultExtractors,
},
&cli.StringSliceFlag{
Name: "experimental-disable-extractors",
Usage: "list of specific extractors and presets of extractors to not use",
},
&cli.StringSliceFlag{
Name: "experimental-detectors",
Usage: "list of specific detectors and presets of detectors to use",
},
&cli.StringSliceFlag{
Name: "experimental-disable-detectors",
Usage: "list of specific detectors and presets of detectors to not use",
Name: "experimental-disable-plugins",
Usage: "list of specific plugins and presets of plugins to not use",
},
}
}
7 changes: 2 additions & 5 deletions cmd/osv-scanner/internal/helper/getters.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,7 @@ func GetCommonScannerActions(cmd *cli.Command, scanLicensesAllowlist []string) o

func GetExperimentalScannerActions(cmd *cli.Command) osvscanner.ExperimentalScannerActions {
return osvscanner.ExperimentalScannerActions{
ExtractorsEnabled: cmd.StringSlice("experimental-extractors"),
ExtractorsDisabled: cmd.StringSlice("experimental-disable-extractors"),

DetectorsEnabled: cmd.StringSlice("experimental-detectors"),
DetectorsDisabled: cmd.StringSlice("experimental-disable-detectors"),
PluginsEnabled: cmd.StringSlice("experimental-plugins"),
PluginsDisabled: cmd.StringSlice("experimental-disable-plugins"),
}
}
33 changes: 19 additions & 14 deletions cmd/osv-scanner/scan/image/command_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "extractors_cancelled_out",
Args: []string{
"", "image",
"--experimental-extractors=sbom/spdx",
"--experimental-extractors=sbom/cdx",
"--experimental-disable-extractors=sbom",
"--experimental-plugins=sbom/spdx",
"--experimental-plugins=sbom/cdx",
"--experimental-disable-plugins=sbom",
"alpine:non-existent-tag",
},
Exit: 127,
Expand All @@ -30,8 +30,8 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "extractors_cancelled_out_with_presets",
Args: []string{
"", "image",
"--experimental-extractors=sbom",
"--experimental-disable-extractors=sbom",
"--experimental-plugins=sbom",
"--experimental-disable-plugins=sbom",
"alpine:non-existent-tag",
},
Exit: 127,
Expand All @@ -40,8 +40,8 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "extractors_cancelled_out",
Args: []string{
"", "image",
"--experimental-extractors=sbom/spdx,sbom/cdx",
"--experimental-disable-extractors=sbom",
"--experimental-plugins=sbom/spdx,sbom/cdx",
"--experimental-disable-plugins=sbom",
"alpine:non-existent-tag",
},
Exit: 127,
Expand Down Expand Up @@ -101,7 +101,7 @@ func TestCommand_Docker(t *testing.T) {
// since we've requested the os/apk extractor disabled, and there's nothing else
// in the image that we support extracting
Name: "real_alpine_image_without_apk_extractor_enabled",
Args: []string{"", "image", "--experimental-disable-extractors=os/apk", "alpine:3.18.9"},
Args: []string{"", "image", "--experimental-disable-plugins=os/apk", "alpine:3.18.9"},
Exit: 128,
},
}
Expand Down Expand Up @@ -221,7 +221,8 @@ func TestCommand_OCIImage(t *testing.T) {
Name: "scanning_insecure_alpine_image_with_specific_detector_enabled",
Args: []string{
"", "image",
"--experimental-detectors", "weakcredentials/etcshadow",
"--experimental-plugins", "os/apk",
"--experimental-plugins", "weakcredentials/etcshadow",
"--archive", "../../../../internal/image/fixtures/test-alpine-etcshadow.tar",
},
Exit: 1,
Expand All @@ -230,8 +231,9 @@ func TestCommand_OCIImage(t *testing.T) {
Name: "scanning_insecure_alpine_image_with_specific_detector_disabled",
Args: []string{
"", "image",
"--experimental-detectors", "weakcreds",
"--experimental-disable-detectors", "weakcredentials/etcshadow",
"--experimental-plugins", "os/apk",
"--experimental-plugins", "weakcreds",
"--experimental-disable-plugins", "weakcredentials/etcshadow",
"--archive", "../../../../internal/image/fixtures/test-alpine-etcshadow.tar",
},
Exit: 1,
Expand All @@ -240,7 +242,8 @@ func TestCommand_OCIImage(t *testing.T) {
Name: "scanning_insecure_alpine_image_with_detector_preset",
Args: []string{
"", "image",
"--experimental-detectors", "weakcreds",
"--experimental-plugins", "os/apk",
"--experimental-plugins", "weakcreds",
"--archive", "../../../../internal/image/fixtures/test-alpine-etcshadow.tar",
},
Exit: 1,
Expand Down Expand Up @@ -323,7 +326,8 @@ func TestCommand_OCIImage_JSONFormat(t *testing.T) {
Name: "scanning_insecure_alpine_image_with_specific_detector_enabled",
Args: []string{
"", "image", "--format=json",
"--experimental-detectors", "weakcredentials/etcshadow",
"--experimental-plugins", "os/apk",
"--experimental-plugins", "weakcredentials/etcshadow",
"--archive", "../../../../internal/image/fixtures/test-alpine-etcshadow.tar",
},
Exit: 1,
Expand All @@ -338,7 +342,8 @@ func TestCommand_OCIImage_JSONFormat(t *testing.T) {
Name: "scanning_insecure_alpine_image_with_detector_preset",
Args: []string{
"", "image", "--format=json",
"--experimental-detectors", "weakcreds",
"--experimental-plugins", "os/apk",
"--experimental-plugins", "weakcreds",
"--archive", "../../../../internal/image/fixtures/test-alpine-etcshadow.tar",
},
Exit: 1,
Expand Down
10 changes: 5 additions & 5 deletions cmd/osv-scanner/scan/source/__snapshots__/command_test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1399,14 +1399,14 @@ Total 0 packages affected by 0 known vulnerabilities (0 Critical, 0 High, 0 Medi

---

[TestCommand_ExplicitExtractors/empty_extractors_flag_does_nothing - 1]
[TestCommand_ExplicitExtractors/empty_plugins_flag_does_nothing - 1]

---

[TestCommand_ExplicitExtractors/empty_extractors_flag_does_nothing - 2]
Incorrect Usage: flag needs an argument: --experimental-extractors=
[TestCommand_ExplicitExtractors/empty_plugins_flag_does_nothing - 2]
Incorrect Usage: flag needs an argument: --experimental-plugins=

flag needs an argument: --experimental-extractors=
flag needs an argument: --experimental-plugins=

---

Expand Down Expand Up @@ -1481,7 +1481,7 @@ Total 0 packages affected by 0 known vulnerabilities (0 Critical, 0 High, 0 Medi
---

[TestCommand_ExplicitExtractors/scanning_directory_with_an_extractor_that_does_not_exist - 2]
unknown extractor "custom/extractor"
not an exact name for a plugin: "custom/extractor"

---

Expand Down
58 changes: 32 additions & 26 deletions cmd/osv-scanner/scan/source/command_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -303,35 +303,35 @@ func TestCommand_ExplicitExtractors(t *testing.T) {

tests := []testcmd.Case{
{
Name: "empty_extractors_flag_does_nothing",
Args: []string{"", "source", "--experimental-extractors="},
Name: "empty_plugins_flag_does_nothing",
Args: []string{"", "source", "--experimental-plugins="},
Exit: 127,
},
{
Name: "extractors_cancelled_out_specified_individually",
Args: []string{
"", "source",
"--experimental-extractors=sbom/spdx",
"--experimental-extractors=sbom/cdx",
"--experimental-disable-extractors=sbom",
"--experimental-plugins=sbom/spdx",
"--experimental-plugins=sbom/cdx",
"--experimental-disable-plugins=sbom",
},
Exit: 127,
},
{
Name: "extractors_cancelled_out_specified_together",
Args: []string{
"", "source",
"--experimental-extractors=sbom/spdx,sbom/cdx",
"--experimental-disable-extractors=sbom",
"--experimental-plugins=sbom/spdx,sbom/cdx",
"--experimental-disable-plugins=sbom",
},
Exit: 127,
},
{
Name: "extractors_cancelled_out_with_presets",
Args: []string{
"", "source",
"--experimental-extractors=sbom",
"--experimental-disable-extractors=sbom",
"--experimental-plugins=sbom",
"--experimental-disable-plugins=sbom",
},
Exit: 127,
},
Expand All @@ -341,7 +341,7 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "scanning_directory_with_one_specific_extractor_enabled",
Args: []string{
"", "source",
"--experimental-extractors=javascript/packagelockjson",
"--experimental-plugins=javascript/packagelockjson",
"./fixtures/locks-many",
},
Exit: 0,
Expand All @@ -350,9 +350,9 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "scanning_directory_with_an_extractor_that_does_not_exist",
Args: []string{
"", "source",
"--experimental-extractors=javascript/packagelockjson",
"--experimental-extractors=custom/extractor",
"--experimental-disable-extractors=custom/anotherextractor",
"--experimental-plugins=javascript/packagelockjson",
"--experimental-plugins=custom/extractor",
"--experimental-disable-plugins=custom/anotherextractor",
"./fixtures/locks-many",
},
Exit: 127,
Expand All @@ -363,8 +363,8 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "scanning_directory_with_a_couple_of_specific_extractors_enabled_individually",
Args: []string{
"", "source",
"--experimental-extractors=javascript/packagelockjson",
"--experimental-extractors=php/composerlock",
"--experimental-plugins=javascript/packagelockjson",
"--experimental-plugins=php/composerlock",
"./fixtures/locks-many",
},
Exit: 0,
Expand All @@ -375,7 +375,7 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "scanning_directory_with_a_couple_of_specific_extractors_enabled_specified_together",
Args: []string{
"", "source",
"--experimental-extractors=javascript/packagelockjson,php/composerlock",
"--experimental-plugins=javascript/packagelockjson,php/composerlock",
"./fixtures/locks-many",
},
Exit: 0,
Expand All @@ -386,7 +386,7 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "scanning_directory_with_one_specific_extractor_disabled",
Args: []string{
"", "source",
"--experimental-disable-extractors=javascript/packagelockjson",
"--experimental-disable-plugins=javascript/packagelockjson",
"./fixtures/locks-many",
},
Exit: 0,
Expand All @@ -397,7 +397,7 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "scanning_file_with_one_specific_extractor_enabled",
Args: []string{
"", "source",
"--experimental-extractors=javascript/packagelockjson",
"--experimental-plugins=javascript/packagelockjson",
"./fixtures/locks-many/package-lock.json",
},
Exit: 0,
Expand All @@ -408,7 +408,7 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "scanning_file_with_one_different_extractor_enabled",
Args: []string{
"", "source",
"--experimental-extractors=javascript/packagelockjson",
"--experimental-plugins=javascript/packagelockjson",
"./fixtures/locks-many/composer.lock",
},
Exit: 128,
Expand All @@ -420,7 +420,7 @@ func TestCommand_ExplicitExtractors(t *testing.T) {
Name: "scanning_file_with_parse_as_but_specific_extractor_disabled",
Args: []string{
"", "source",
"--experimental-disable-extractors=javascript/packagelockjson",
"--experimental-disable-plugins=javascript/packagelockjson",
"-L", "package-lock.json:./fixtures/locks-many/composer.lock",
},
Exit: 127,
Expand Down Expand Up @@ -1050,7 +1050,8 @@ func TestCommand_WithDetector_OnLinux(t *testing.T) {
Name: "ssh_version_is_before_first_vuln_version",
Args: []string{
"", "source",
"--experimental-detectors", "cve/cve-2023-38408",
"--experimental-plugins", "php/composerlock",
"--experimental-plugins", "cve/cve-2023-38408",
filepath.Join(testDir, "composer.lock"),
},
Exit: 0,
Expand All @@ -1060,7 +1061,8 @@ func TestCommand_WithDetector_OnLinux(t *testing.T) {
Name: "ssh_version_is_after_last_vuln_version",
Args: []string{
"", "source",
"--experimental-detectors", "cve/cve-2023-38408",
"--experimental-plugins", "php/composerlock",
"--experimental-plugins", "cve/cve-2023-38408",
filepath.Join(testDir, "composer.lock"),
},
Exit: 0,
Expand All @@ -1070,7 +1072,8 @@ func TestCommand_WithDetector_OnLinux(t *testing.T) {
Name: "ssh_version_errors",
Args: []string{
"", "source",
"--experimental-detectors", "cve/cve-2023-38408",
"--experimental-plugins", "php/composerlock",
"--experimental-plugins", "cve/cve-2023-38408",
filepath.Join(testDir, "composer.lock"),
},
Exit: 0,
Expand Down Expand Up @@ -1119,7 +1122,8 @@ func TestCommand_WithDetector_OffLinux(t *testing.T) {
Name: "ssh_version_is_before_first_vuln_version",
Args: []string{
"", "source",
"--experimental-detectors", "cve/cve-2023-38408",
"--experimental-plugins", "php/composerlock",
"--experimental-plugins", "cve/cve-2023-38408",
filepath.Join(testDir, "composer.lock"),
},
Exit: 0,
Expand All @@ -1129,7 +1133,8 @@ func TestCommand_WithDetector_OffLinux(t *testing.T) {
Name: "ssh_version_is_after_last_vuln_version",
Args: []string{
"", "source",
"--experimental-detectors", "cve/cve-2023-38408",
"--experimental-plugins", "php/composerlock",
"--experimental-plugins", "cve/cve-2023-38408",
filepath.Join(testDir, "composer.lock"),
},
Exit: 0,
Expand All @@ -1139,7 +1144,8 @@ func TestCommand_WithDetector_OffLinux(t *testing.T) {
Name: "ssh_version_errors",
Args: []string{
"", "source",
"--experimental-detectors", "cve/cve-2023-38408",
"--experimental-plugins", "php/composerlock",
"--experimental-plugins", "cve/cve-2023-38408",
filepath.Join(testDir, "composer.lock"),
},
Exit: 0,
Expand Down
48 changes: 0 additions & 48 deletions internal/builders/extractors.go
View file
Open in desktop

This file was deleted.

Loading
Loading