Skip to content

fix(output): emit valid SPDX tag-value for resolved deps and exceptions - #1325

Merged
mstykow merged 1 commit into
mainfrom
fix/spdx-tagvalue-sbom-validity
Jul 23, 2026
Merged

fix(output): emit valid SPDX tag-value for resolved deps and exceptions#1325
mstykow merged 1 commit into
mainfrom
fix/spdx-tagvalue-sbom-validity

Conversation

@mstykow

@mstykow mstykow commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

The checked-in SBOM examples (and any real scan with resolved dependencies or an unattached license exception) produced SPDX tag-value documents that fail official SPDX validation (spdx-tools / pyspdxtools). CycloneDX was and is valid; this fixes two SPDX tag-value renderer bugs and adds the regression coverage that would have caught them.

Both were surfaced by validating the committed examples/sbom/*/sbom.spdx against the official validator — something CI did not do (its testdata/smoke scan is too small to have resolved-dependency packages or exception detections).

Bug 1 — promoted-dependency packages placed before the file section

SPDX tag-value associates a file with the most recently declared package (positional). The writer emitted every package — including promoted resolved dependencies, which are FilesAnalyzed: false and own no files — before the ## File Information section, so the scanned files positionally bound to a FilesAnalyzed: false package. spdx-tools rejects this: "package must contain no elements if files_analyzed is False."

Fix (src/output/spdx.rs): emit scanned-subject packages (FilesAnalyzed: true) before the file section and promoted-dependency packages after it. (Before #1321 there was only the subject package, so this was latent.)

Bug 2 — license exception symbols used outside a WITH

A detected exception (e.g. ScanCode's LicenseRef-scancode-generic-exception, or a listed LLVM-exception) reached SPDX license fields as a standalone id (LicenseInfoInFile, PackageLicenseInfoFromFiles) and inside an AND-joined expression (LicenseConcluded: MIT AND Unlicense AND LicenseRef-scancode-generic-exception). SPDX only permits an exception on the right of a WITH, so spdx-tools rejects it.

Fix:

  • The token after WITH is an exception, not a license id — spdx_ids_from_expression no longer emits it as a standalone id.
  • strip_unattached_exceptions drops floating (non-WITH) exception operands from license expressions while preserving a valid license WITH exception; applied to file and package LicenseConcluded/LicenseDeclared in both the tag-value and RDF writers.
  • Exception symbols are filtered out of the id-list fields and their now-unreferenced ExtractedLicensingInfo declarations.
  • Exceptions are identified with the existing license_detection::spdx_lid::is_spdx_exception (promoted to pub(crate)) — a pure function of the token, since output.license_references is empty by default (ScanCode-compatible; only populated with --license-references).

Regression coverage

  • New golden fixture testdata/output-formats/spdx-dependencies-expected.tv and test test_spdx_tag_value_emits_promoted_dependency_packages_after_file_section (tests/output_format_golden.rs): a scanned subject package that owns files plus a promoted dependency (the existing CycloneDX dependency sample has no files). The test asserts the subject package precedes, and the dependency package follows, the file section.
  • The new fixture is added to the official-validator set in scripts/validate_output_format_fixtures.py (SPDX_TAGVALUE_FIXTURES), so CI now validates a deps-bearing SPDX document against pyspdxtools on every run — closing the gap that let both bugs ship.
  • Unit tests for the exception handling (spdx_ids_from_expression_drops_the_with_exception_symbol, strip_unattached_exceptions_drops_floating_exceptions_but_keeps_with).

Examples regenerated

All four examples/sbom/ regenerated (stamped 1.0.1). Validated locally against the official validators: CycloneDX valid ×4, SPDX valid ×4.

Release note

This is the fix that the cancelled 1.0.1 was blocked on. main is already at the 1.0.1 version bump; nothing was published (no crate, image, release, or Homebrew bump). After merge, re-tag v1.0.1 on the new HEAD to re-trigger the release.

@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown

Greptile Summary

Fixes SPDX tag-value generation and expands regression validation.

  • Places promoted dependency packages after the positional file section.
  • Removes unattached license exceptions while preserving valid WITH expressions in tag-value and RDF output.
  • Adds dependency-bearing golden coverage and official SPDX fixture validation.
  • Regenerates the checked-in SBOM examples.

Confidence Score: 5/5

The PR appears safe to merge with no eligible blocking follow-up failures identified.

No blocking failure remains within the scope of the prior review threads.

Important Files Changed

Filename Overview
src/output/spdx.rs Reorders tag-value package sections and normalizes exception handling across tag-value and RDF serializers.
src/license_detection/spdx_lid/mod.rs Exposes the existing SPDX-exception classifier within the crate for serializer use.
tests/output_format_golden.rs Adds a dependency-bearing SPDX regression fixture and verifies package/file section ordering.
scripts/validate_output_format_fixtures.py Adds the dependency-bearing tag-value fixture to official SPDX validation.
testdata/output-formats/spdx-dependencies-expected.tv Pins the corrected tag-value structure for an analyzed package with a promoted dependency.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A["Shared output schema"] --> B["Plan SPDX packages"]
  B --> C["Analyzed subject packages"]
  C --> D["File information"]
  D --> E["Promoted dependencies"]
  E --> F["Relationships"]
  A --> G["Normalize license expressions"]
  G --> H["Tag-value output"]
  G --> I["RDF/XML output"]
Loading

Reviews (2): Last reviewed commit: "fix(output): emit valid SPDX tag-value f..." | Re-trigger Greptile

The checked-in SBOM examples produced SPDX tag-value documents that failed
official SPDX validation (spdx-tools/pyspdxtools). Two renderer bugs, both
surfaced only by a real scan with resolved dependencies or an unattached
license exception (CI's testdata/smoke scan has neither):

1. Promoted-dependency packages (FilesAnalyzed: false) were emitted before the
   file section. SPDX tag-value binds a file to the most recently declared
   package, so scanned files bound to a FilesAnalyzed: false package. Emit
   subject packages before, and dependency packages after, the file section.
2. License exception symbols (e.g. LicenseRef-scancode-generic-exception,
   LLVM-exception) reached SPDX license fields standalone or AND-joined; SPDX
   only allows an exception on the right of WITH. Drop the post-WITH token from
   id lists, strip floating (non-WITH) exceptions from expressions while keeping
   'license WITH exception', and drop their unreferenced ExtractedLicensingInfo.
   Exceptions are identified via license_detection::spdx_lid::is_spdx_exception
   (now pub(crate)); output.license_references is empty by default.

Regression coverage: new golden fixture spdx-dependencies-expected.tv + a
golden test asserting dependency-package placement, wired into the official
validator set (scripts/validate_output_format_fixtures.py) so CI validates a
deps-bearing SPDX document every run; plus unit tests for the exception
handling. All four examples/sbom regenerated (1.0.1) and validate clean against
the official CycloneDX and SPDX validators.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Maxim Stykow <maxim.stykow@gmail.com>
@mstykow
mstykow force-pushed the fix/spdx-tagvalue-sbom-validity branch from b141519 to aed009e Compare July 23, 2026 22:18
@mstykow
mstykow merged commit be8aeae into main Jul 23, 2026
12 of 13 checks passed
@mstykow
mstykow deleted the fix/spdx-tagvalue-sbom-validity branch July 23, 2026 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant