Skip to content

feat(output): version resolved-dependency SBOM purls and join vendored licenses - #1327

Merged
mstykow merged 4 commits into
mainfrom
feat/sbom-versioned-dep-purls-and-vendored-join
Jul 24, 2026
Merged

feat(output): version resolved-dependency SBOM purls and join vendored licenses#1327
mstykow merged 4 commits into
mainfrom
feat/sbom-versioned-dep-purls-and-vendored-join

Conversation

@mstykow

@mstykow mstykow commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Completes transitive-dependency license coverage in the SBOM (issue SBOM: complete transitive-dependency license coverage #1320), fully static and no-network. Every resolved dependency carries its versioned purl (the join key for downstream enrichment) and, when its source/metadata is on disk, its license — as a single deduplicated component.
  • The fix lives in the shared inventory (src/output/sbom.rs), so both CycloneDX and SPDX benefit, and SPDX now renders the versioned coordinate as a field. The native ScanCode-compatible JSON is untouched.

What changed

SbomInventory builds an owner-scoped version index and canonicalizes every dependency edge through it before promotion and graph-edge resolution:

  1. Versioned purls. An unversioned requirement edge (pkg:npm/leftpad, from a manifest) is upgraded to its resolved coordinate (pkg:npm/leftpad@1.0.0) — from the edge's own resolved_package, or from a single unambiguous versioned sibling under the same owner. Identities with zero or multiple versioned candidates stay unversioned (honest-unknowns; Provenant does not guess which version a range resolved to).
  2. Vendored license join / dedup. The canonicalized requirement dedups against the detected, licensed package for that purl (vendored case → one licensed, versioned component) or the versioned lockfile edge (source-absent → one versioned component, license carried from resolved_package when present, else unset). No more two-components-for-one-dependency.
  3. SPDX purl/version rendering. Every SPDX package (subject and promoted dependency) with a purl/version now emits it: tag-value PackageVersion + ExternalRef: PACKAGE-MANAGER purl <purl>, RDF spdx:versionInfo + spdx:externalRef (referenceCategory packageManager, referenceType purl). SPDX previously carried neither.

Promotion and edge resolution share one resolver, so the graph stays closed (0 dangling), bom-ref/dependencies[].ref stay unique, and a dep owned by multiple packages remains a single component.

Correctness invariants (identity + version resolution)

The version-index identity and resolution are deliberately conservative — an edge only ever adopts a version that is genuinely its own resolution:

  • Variant-preserving identity. The version-less identity strips only the version and is rebuilt canonically via PackageUrl, keeping type + namespace + name + qualifiers + subpath. Variants that differ only by a qualifier (arch, classifier, repository) or subpath are never conflated.
  • Owner-scoped. The index is keyed by (owner, identity) from dependency edges only. An unversioned requirement under one owner never inherits a version from a sibling under a different owner (owners can resolve the same package to different versions).
  • No ownerless pooling. Edges without a concrete for_package_uid neither contribute to nor consult the sibling index — an ownerless edge resolves only from its own resolved_package, else stays unversioned.
  • Non-empty purls are never dropped. A non-empty purl that PackageUrl cannot parse is passed through unchanged as the component coordinate and graph ref (canonicalization is skipped for it), so a malformed-but-present dependency is never silently erased from the SBOM.

Before / after

Repro tree (package.json + package-lock.json + node_modules/leftpad/package.json with "license": "MIT"), scanned --license --package --strip-root:

  • Before: two components — pkg:npm/leftpad@1.0.0 (detected, MIT) and pkg:npm/leftpad (resolved dep, unversioned, no license).
  • After: one component — pkg:npm/leftpad@1.0.0 carrying MIT; edge myapp → pkg:npm/leftpad@1.0.0; 0 dangling, in both CycloneDX and SPDX.

Real examples (examples/sbom, regenerated). Components drop where an unversioned requirement coalesces onto its resolved/detected versioned sibling under the same owner; the modest deltas reflect the conservative rules above (ambiguous, cross-owner, and ownerless requirements are intentionally left as-is), graph still fully closed:

target components (before → after) note
ripgrep 114 → 107 workspace members that appeared as both a detected package and a resolved dep coalesce; multi-version deps (aho-corasick/bstr) correctly stay unversioned
flask 54 → 53 one requirement/resolved pair coalesced
tokio 51 → 51 versioned in place; workspace cross-member deps stay honestly unversioned
express 45 → 45 no committed lockfile, so no resolved versions exist — unchanged

Issues

Scope and exclusions

  • Included: owner-scoped, variant-preserving version-index + purl canonicalization in the shared inventory; SPDX tag-value + RDF purl/version rendering; ADR 0012 amendment; regenerated dependency goldens and the four examples/sbom documents.
  • Explicit exclusions: no online/registry license lookup (out of scope by design — see SBOM: complete transitive-dependency license coverage #1320). Native ScanCode-compatible JSON is unchanged.

How to verify

  • Build the leftpad repro above and scan --license --package --strip-root --cyclonedx - (and --spdx-tv -): one licensed, versioned component, myapp DEPENDS_ON leftpad@1.0.0, 0 dangling in both formats.

  • Official-validator gate (blocking): every checked-in output-format fixture and all four examples/sbom/*/{sbom.cdx.json,sbom.spdx} pass cyclonedx-python-lib (CycloneDX 1.3 JSON schema + XSD) and pyspdxtools (SPDX-2.2, .tv.spdx first); live-scan RDF/JSON/XML also validated via scripts/validate_output_format_fixtures.py --provenant-bin ….

    CycloneDX: ripgrep OK  express OK  flask OK  tokio OK   (+ dependency goldens OK)
    SPDX:      ripgrep OK  express OK  flask OK  tokio OK   (+ dependency + simple goldens OK)
    
  • Regression unit tests in src/output/sbom.rs cover each invariant: qualifier/subpath non-conflation, cross-owner non-inheritance, ownerless non-pooling, and unparseable-purl retention.

ScanCode parity

  • Unaffected. compare-outputs diffs the native ScanCode-compatible --json output; this change is confined to src/output/sbom.rs, which is imported only by the CycloneDX/SPDX renderers and never by the native writers, so the native packages[]/dependencies[] output is byte-identical (verified on the leftpad repro). (ScanCode is not installed in this environment; the conclusion follows structurally from the import graph plus the byte-identical native output.)

Expected-output fixture changes

  • Files changed: testdata/output-formats/cyclonedx-dependencies-expected.{json,xml}, testdata/output-formats/spdx-dependencies-expected.tv (+ spdx-licensed/spdx-multi-package now carry PackageVersion/ExternalRef), and the four regenerated examples/sbom/* documents + READMEs.
  • Why correct: the dependency sample now includes an unversioned manifest requirement plus the lockfile's versioned, licensed resolution of the same dependency; the correct SBOM is one licensed, versioned component rather than a bare purl plus a separate versioned one. The SPDX dependency test normalizes the build-specific Creator:/Created: lines like every other SPDX-tv golden (it previously used a raw assert_eq, coupling the fixture to the exact build version). All regenerated fixtures pass the official external validators.

🤖 Generated with Claude Code

…d licenses

Resolve each SBOM dependency edge to its versioned coordinate before
promotion and graph-edge resolution. SbomInventory now builds a version
index (version-less purl identity -> the single versioned purl that
shares it, drawn from detected packages, dependency purls, and
resolved-package purls) and upgrades an unversioned requirement edge to
that sibling. Identities with zero or multiple versioned siblings stay
unversioned (honest-unknowns).

This closes issue #1320's two static, no-network gaps in one join:

- Resolved-dependency components carry the resolved, versioned purl
  (pkg:npm/leftpad@1.0.0), the join key for downstream enrichment.
- A vendored dependency (source/metadata on disk) dedups against the
  unversioned requirement edge into ONE licensed, versioned component
  instead of two.

Both CycloneDX and SPDX consume the shared inventory, so both benefit;
the native ScanCode-compatible JSON is untouched. Graph stays closed
(edges canonicalize through the same resolver) and bom-refs stay unique.

Regenerated the four examples/sbom documents (ripgrep 114->91, flask
54->47, tokio 51->46 components; express unchanged, no lockfile) and the
dependency goldens; all validate against cyclonedx-python-lib and
pyspdxtools. Updated ADR 0012 with the amendment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
@greptile-apps

greptile-apps Bot commented Jul 24, 2026

Copy link
Copy Markdown

Greptile Summary

Updates the shared SBOM inventory and SPDX serializers to preserve dependency context and emit resolved package coordinates.

  • Scopes dependency-version evidence by package owner while preserving purl qualifiers, subpaths, and unparseable non-empty coordinates.
  • Canonicalizes dependency promotion and graph-edge resolution through the same resolver.
  • Adds package versions and purl external references to SPDX tag-value and RDF/XML output.
  • Regenerates CycloneDX/SPDX fixtures, examples, and documentation for the revised inventory.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain in the fixes associated with the previous review threads.

Important Files Changed

Filename Overview
src/output/sbom.rs Adds owner-scoped version evidence and consistent purl resolution for promoted components and dependency edges, including focused regressions for all previously reported cases.
src/output/spdx.rs Emits package versions and purl external references in SPDX tag-value and RDF/XML output.
tests/output_format_golden.rs Extends shared output fixtures to cover coalescing an unversioned requirement with its licensed, versioned resolution.
docs/adr/0012-sbom-resolved-dependency-components.md Documents the owner-scoped resolution rules, identity preservation, malformed-purl handling, and SPDX output behavior.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A["Detected packages"] --> D["Shared SBOM inventory"]
  B["Declared dependency edges"] --> C["Owner-scoped purl resolver"]
  B2["Resolved dependency edges"] --> C
  C --> D
  D --> E["CycloneDX components and graph"]
  D --> F["SPDX packages and relationships"]
  F --> G["PackageVersion and purl ExternalRef"]
Loading

Reviews (4): Last reviewed commit: "fix(output): retain unparseable dependen..." | Re-trigger Greptile

Comment thread src/output/sbom.rs Outdated
Comment thread src/output/sbom.rs Outdated
…ls in SPDX

Address two P1 correctness bugs in the version-index join and fold in the
deferred SPDX purl/version rendering.

P1(a) identity now strips only the version: type, namespace, name,
qualifiers, and subpath are preserved (rebuilt through PackageUrl for a
canonical, qualifier-sorted key). Variants that differ only by a qualifier
(?arch=, ?classifier=, …) or subpath are no longer conflated.

P1(b) the version index is per-owner, keyed by (for_package_uid,
version-less identity) and built from dependency edges only (not detected
packages). An unversioned edge is upgraded only from its own
resolved_package purl or a single unambiguous versioned sibling under the
SAME owner. A version is never borrowed across owners, and ambiguous or
cross-owner cases stay unversioned (honest-unknown). Added regression tests
for both: two owners resolving one name to different versions (no
cross-assignment, graph stays closed) and qualifier/subpath variants that
must not be conflated.

SPDX now surfaces the versioned coordinate for every package (subject and
promoted dependency): tag-value emits PackageVersion and
`ExternalRef: PACKAGE-MANAGER purl <purl>`; RDF emits spdx:versionInfo and a
spdx:externalRef (referenceCategory packageManager, referenceType purl).
Both validate with pyspdxtools.

Regenerated the four examples/sbom and the affected goldens; updated ADR
0012. All four examples (CycloneDX + SPDX) and the goldens pass the official
validators. Owner-scoping is stricter than the previous document-wide index,
so workspace cross-member deps whose lockfile edges are owned elsewhere now
stay honestly unversioned (ripgrep 91->107, flask 47->53, tokio 46->51
components); the graph stays closed (0 dangling) in every document.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Comment thread src/output/sbom.rs Outdated
A dependency edge with no (or empty) `for_package_uid` previously collapsed
to a shared empty-owner key in the version index, so an unversioned
ownerless requirement could adopt the sole versioned purl contributed by an
unrelated ownerless dependency (e.g. a hoisted dep from another
workspace/datafile) — a wrong concrete coordinate.

Ownerless edges now neither contribute to nor consult the same-owner sibling
index: only edges with a concrete `for_package_uid` build or read it. An
ownerless unversioned edge resolves its version solely from its own
`resolved_package` purl, otherwise stays unversioned (honest-unknown).

Added regression test `ownerless_requirement_does_not_borrow_sibling_version`:
two ownerless deps from different contexts (`lib@1.0.0` versioned, bare `lib`
requirement with no resolved_package) — the bare one stays unversioned, both
components are present, and the graph stays closed. Updated ADR 0012.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Comment thread src/output/sbom.rs Outdated
…of dropping them

The PackageUrl-based version-less identity rebuild made `resolve_edge_purl`
return None whenever a dependency's non-empty purl failed to parse, so
promotion AND graph construction skipped it — the component and its owner
relationship vanished from both CycloneDX and SPDX even though the native
output preserves that coordinate (a silent, dishonest inventory loss).

A non-empty purl is now never dropped: when it cannot be parsed it cannot be
canonicalized or deduped, so it is passed through unchanged as the component
coordinate and graph-edge ref. Only a genuinely empty purl is absent, matching
prior behavior. The retained component's bom-ref and the edge ref are the same
original string, so the graph stays closed.

Added regression test `unparseable_purl_is_retained_as_component_and_edge`: a
dependency with a non-empty malformed purl still appears as a promoted
component with its original coordinate and its owner→dep edge; the graph stays
closed with no dangling refs. Updated ADR 0012.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
@mstykow
mstykow merged commit 8eb4a1b into main Jul 24, 2026
13 checks passed
@mstykow
mstykow deleted the feat/sbom-versioned-dep-purls-and-vendored-join branch July 24, 2026 09:12
mstykow added a commit that referenced this pull request Jul 24, 2026
Content already reflected 1.0.2 (versioned dep purls, deduped components,
SPDX purl rendering) from #1327; this rolls the embedded version label from
1.0.1 to 1.0.2 (Creator/tools/README) so the stamp matches the output. On-demand
regen — no release hook.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SBOM: complete transitive-dependency license coverage

1 participant