perf(detector): derive verified-product suppression from the detection result by MaineK00n · Pull Request #2592 · future-architect/vuls

MaineK00n · 2026-07-01T04:02:24Z

Stacked on #2590 (depends on collectVerifiedProducts, introduced there). Base will retarget to master once #2590 merges.

Problem

collectVerifiedProducts (the verified-source suppression mirroring go-cve-dictionary's isCpeURIAlsoDefinedInVerifiedDataSource) re-fetched each detected VulnCheck/JVN CVE's verified-source data with a per-CVE GetVulnerabilityDataByVulnerabilityID. On a pathological kernel-as-CPE scan (cpe:/o:linux:linux_kernel:6.1 → 7268 CVEs) this dominates DetectCPEs — ~36s of a ~42s warm run (did not finish within 2 min cold).

Why the re-fetch is redundant

cpe.Detect → util.Detect finds candidate roots through a part:vendor:product index (version-agnostic) and returns every source's full criteria for each such root unconditionally (FilteredCriterion keeps the underlying Criterion, so non-matching definitions survive), so the verified sources' defined products are already present in the in-memory detection result. A verified root absent from the result can only define products that were not scanned (else the index would have surfaced it), and suppress() only looks up scanned products — so the derived set is identical for suppression purposes.

Change

collectVerifiedProducts drops its *session.Session and derives the set from detected in two passes (union verified-source defined products per CVE, then attach to suppressed roots).
collectDefinedCPEProducts now walks a FilteredCriteria via FilteredCriterion.Criterion.
new isVerifiedCPESource predicate.

Result (measured, same DB)

phase	before	after
`collectVerifiedProducts`	~36s	~1.3s (I/O-free, cache-independent)
`DetectCPEs` total (warm)	~42s	~8.6s
`DetectCPEs` (cold)	>120s	~21s
detection output	7268 CVEs / 1162 VulnCheck / 109 JVN	identical

Verification

build (normal / -tags scanner ./detector/... / GOOS=windows), go vet, gofmt, go test ./detector/..., golangci-lint (0 issues) green.
Detection output identical before/after on the kernel-as-CPE smoke (same CVE / VulnCheck / JVN counts).

🤖 Generated with Claude Code

…n result collectVerifiedProducts re-fetched each detected VulnCheck/JVN CVE's verified-source data with a per-CVE GetVulnerabilityDataByVulnerabilityID. On a pathological kernel-as-CPE scan (7268 CVEs) this dominated DetectCPEs at ~36s of a ~42s warm run (did not finish within 2 min cold). The re-fetch is redundant: cpe.Detect finds candidate roots through a part:vendor:product index (version-agnostic) and util.Detect returns every source's full criteria for each such root unconditionally, so the verified sources' defined products are already in the in-memory detection result. A verified root absent from the result can only define non-scanned products (else the index would have surfaced it), and suppress() only looks up scanned products, so the derived set is identical for suppression purposes. Build the set from detected instead: collectVerifiedProducts drops its *session.Session, unions each detected root's verified-source defined products per CVE (collectDefinedCPEProducts now walks a FilteredCriteria via FilteredCriterion.Criterion, which cond.Accept keeps in full), then attaches them to the suppressed roots. collectVerifiedProducts ~36s -> ~1.3s (I/O-free, cache-independent); DetectCPEs ~42s -> ~8.6s; detection output unchanged (kernel:6.1 -> 7268 CVEs, 1162 VulnCheck / 109 JVN, identical before/after). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This PR removes per-CVE DB re-fetching for verified-product suppression during vuls2 CPE detection by deriving the verified-source defined product set directly from the in-memory detection result, substantially reducing runtime overhead in pathological high-CVE scans.

Changes:

Reworked collectVerifiedProducts to be I/O-free by deriving verified-source defined products from detectTypes.DetectResult.
Added isVerifiedCPESource to centralize the “verified CPE source” predicate.
Updated collectDefinedCPEProducts to traverse FilteredCriteria via FilteredCriterion.Criterion.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ment The old header still described a raw, unfiltered criteria tree after the signature moved to FilteredCriteria, contradicting the accurate block directly below it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

This branch removed the go-cve-dictionary dependency from go.mod, so its entries in the vuls group and others exclude-patterns are now stale (same cleanup already applied to go-cti). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

…on result Add focused table-driven tests for the I/O-free verified-product derivation: same-root suppression, cross-root aliasing (NVD under the CVE root feeding JVN under a JVNDB-* root), per-CVE separation on a multi-CVE suppressed root, empty result when no verified product exists, CPEMatches folding to the product key with non-CPE ecosystems ignored, and that a suppressed source's own products are not treated as verified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Drop the cpeCond/cpeCondMatches/cpeDetection/vulns/set helper closures and spell out each DetectResult literal in full, so individual cases can be tuned independently without threading changes through shared builders. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Pass 1 allocated a products map for every detected root, including the majority that carry no verified CPE source. Allocate only once a verified CPE source is actually encountered, cutting avoidable allocations/GC on large detection results. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Single-use one-liner over slices.Contains(verifiedCPESources, ...); the verifiedCPESources var already documents the set, so the call site reads clearly inline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Revert the lazy allocation: the avoided per-root heap header is negligible (empty maps allocate no buckets, and this is dwarfed by the DB-I/O win the rewrite delivered), so favor the simpler unconditional make. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Reference fcn.Criterion directly instead of aliasing it, removing the intermediate binding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

postConvert already has the DetectResult, so compute the verified-product sets from it directly (inlined into the walkVulnerabilityDetections call) instead of threading a separate parameter through detectWith. Removes a redundant argument that had to be kept consistent with detected. Test_postConvert no longer injects an independent verified map; the multi-CVE JVN case now carries a real (unmatched) NVD root so the product set is derived, exercising the production path end-to-end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

postConvert threaded noJVNCPEs through but every Test_postConvert case passed nil, so the parameter never rode the execution path. Add a case where the scanned CPE is in noJVNCPEs and the sole (JVN) detection is therefore suppressed, asserting the CVE drops from the result — locking the wiring end-to-end (per-CPE behaviour stays unit-tested in Test_walkCPECriteria). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…cts case VulnCheck (like NVD) is rooted per CVE ID, so a single VulnCheck root carrying two CVEs is unrealistic. Switch the multi-CVE suppressed root to JVN's JVNDB-* form, which genuinely groups multiple CVEs under one root. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

VulnCheck is rooted per CVE ID, so the no-verified-product case's root is renamed VC-only -> CVE-2024-0004. Also switch the placeholder CVE-2024-000A/ 000B (letters are not a valid CVE ID) to numeric CVE-2024-0011/0012. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

MaineK00n force-pushed the MaineK00n/vulncheck-jvn-vuls2 branch from adaa539 to 28d61cb Compare July 1, 2026 04:13

MaineK00n force-pushed the MaineK00n/collect-verified-from-detected branch from 37c078a to 7f6053b Compare July 1, 2026 04:14

Base automatically changed from MaineK00n/vulncheck-jvn-vuls2 to master July 1, 2026 04:32

MaineK00n self-assigned this Jul 1, 2026

MaineK00n force-pushed the MaineK00n/collect-verified-from-detected branch from 7f6053b to 11d7dac Compare July 1, 2026 04:35

MaineK00n requested a review from Copilot July 1, 2026 04:37

Copilot started reviewing on behalf of MaineK00n July 1, 2026 04:37 View session

Copilot AI reviewed Jul 1, 2026

View reviewed changes

Comment thread detector/vuls2/vuls2.go Outdated

MaineK00n and others added 2 commits July 1, 2026 13:43

MaineK00n requested a review from Copilot July 1, 2026 04:48

Copilot started reviewing on behalf of MaineK00n July 1, 2026 04:48 View session

Copilot AI reviewed Jul 1, 2026

View reviewed changes

Comment thread detector/vuls2/vuls2.go

MaineK00n requested a review from Copilot July 1, 2026 05:48

Copilot started reviewing on behalf of MaineK00n July 1, 2026 05:48 View session

Copilot AI reviewed Jul 1, 2026

View reviewed changes

Comment thread detector/vuls2/vuls2.go

MaineK00n requested a review from Copilot July 1, 2026 16:54

Copilot started reviewing on behalf of MaineK00n July 1, 2026 16:55 View session

Copilot AI reviewed Jul 1, 2026

View reviewed changes

MaineK00n and others added 7 commits July 3, 2026 15:01

refactor(detector): inline isVerifiedCPESource

6580e21

Single-use one-liner over slices.Contains(verifiedCPESources, ...); the verifiedCPESources var already documents the set, so the call site reads clearly inline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

refactor(detector): drop cn alias in collectDefinedCPEProducts

65b6edd

Reference fcn.Criterion directly instead of aliasing it, removing the intermediate binding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

MaineK00n requested review from Copilot and shino July 3, 2026 08:55

Copilot started reviewing on behalf of MaineK00n July 3, 2026 08:56 View session

Copilot AI reviewed Jul 3, 2026

View reviewed changes

Comment thread detector/vuls2/vuls2.go

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

perf(detector): derive verified-product suppression from the detection result#2592

perf(detector): derive verified-product suppression from the detection result#2592
MaineK00n wants to merge 13 commits into
masterfrom
MaineK00n/collect-verified-from-detected

MaineK00n commented Jul 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

MaineK00n commented Jul 1, 2026

Problem

Why the re-fetch is redundant

Change

Result (measured, same DB)

Verification

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants