This directory documents stable, user-visible differences and improvements between Provenant and the Python ScanCode Toolkit reference implementation. It includes parser improvements and text-detection subsystem improvements.
This directory records cases where Provenant intentionally differs from or improves on the Python reference implementation. We:
- Fix bugs present in the Python implementation
- Implement features marked as TODO in Python
- Add missing functionality where it improves user-visible behavior
- Improve data quality and extraction accuracy
Python implementation has incorrect behavior, we fix it.
Python has TODO comments or placeholders, we implement the feature.
Python extracts some data, we extract more (additional fields, better parsing).
Python has unsafe patterns (code execution, DoS vulnerabilities), we use safe alternatives.
| Area | Improvement Type | Python Reference Limitation | Rust Improvement | Impact |
|---|---|---|---|---|
| RPM | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Truncated installed EVR, extension-only source RPM detection, missing YumDB enrichment, and thin archive metadata | Full dependency extraction, EVR preservation, hash-based source RPM recognition, richer archive metadata, and YumDB merge support | Better RPM/rootfs parity, cleaner package identity, and richer system-package provenance |
| Debian | ✨ New Feature + 🔍 Enhanced + 🐛 Bug Fix + ⚡ Performance | No direct .deb archive introspection, incomplete DEP-5 copyright detections, missing installed-package sidecar integration, and avoidable repeated paragraph parsing work |
Direct .deb control/archive introspection, package-matching embedded copyright recovery, coherent DEP-5 primary/paragraph/fallback detections, installed Debian sidecar integration for rootfs/container scans, and a Debian-local single-pass paragraph finalizer |
Better Debian archive metadata accuracy, improved Debian copyright visibility, stronger Debian rootfs/container package inventory, and faster large DEP-5 parsing |
| Conan | ✨ New Feature | No conanfile.txt or conan.lock parser | Full conanfile.txt + conan.lock extraction | C/C++ dependency visibility |
| BitBake | ✨ New Feature + 🔍 Enhanced | No shipped Python BitBake parser, no .bbappend packaged metadata coverage, and no parser-owned recovery of local LIC_FILES_CHKSUM / SRC_URI file references |
Added bounded static .bb + .bbappend parsing, override-style dependency handling, raw declared-license preservation, local file-reference extraction, and sibling assembly for recipe-plus-append metadata |
Better Yocto/OpenEmbedded package visibility and stronger recipe-local provenance |
| Carthage | ✨ New Feature + 🔍 Enhanced | No shipped Python Carthage parser | Added static Cartfile, Cartfile.private, and Cartfile.resolved parsing with dependency origin preservation and sibling assembly for declared plus pinned state |
Better Apple dependency visibility for Carthage-managed projects |
| CPAN | ✨ New Feature + 🐛 Bug Fix | Stub-only handlers (no parse method), and malformed inputs in Rust fallback paths could lose parser identity | Full META.json, META.yml, MANIFEST parsing plus datasource-preserving fallback package identity for malformed/unreadable CPAN inputs | Perl metadata extraction with safer assembly/scanner behavior |
| RPM Specfile | ✨ New Feature | Stub with TODO comment | Full preamble parsing | RPM spec metadata extraction |
| CPAN Makefile.PL | ✨ New Feature + 🔍 Enhanced | Stub-only handler (no parse method) | WriteMakefile metadata extraction plus bounded VERSION_FROM and ABSTRACT_FROM recovery from sibling module files |
Better Perl build metadata and package identity |
| OSGi Manifest | ✨ New Feature | Empty path_patterns (assembly only) | Full OSGi metadata extraction | Java bundle dependencies |
| Gradle | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Wrong compileOnly runtime classification, missing Gradle SBOM license propagation, incomplete version-catalog alias identifiers, and only-first-block dependency extraction |
Fixed scope classification, extracted Gradle POM license metadata, parsed all discovered Gradle dependency blocks, resolved TOML-backed libs.versions.toml aliases, and preserved nested project-path identifiers |
Better Gradle dependency semantics, better SBOM metadata, and more accurate package identifiers |
| Gradle Module Metadata | ✨ New Feature + 🔍 Enhanced | No merged .module metadata handler despite upstream attempts |
Added direct Gradle .module parsing with published artifact checksums, dependency and constraint extraction, variant metadata preservation, and file-reference recovery |
Better JVM publication metadata, stronger artifact provenance, and better coverage of modern Gradle-native modules |
| Gradle Lockfile | ✨ New Feature | No gradle.lockfile parser | Full lockfile dependency extraction | Pinned dependency auditing |
| vcpkg | ✨ New Feature + 🔍 Enhanced | No modern vcpkg.json manifest parser and no preservation of manifest-mode dependency/configuration metadata |
Added strict-JSON vcpkg.json parsing for project and port manifests, direct dependency extraction, versioning metadata preservation, and embedded/sibling configuration capture |
Better modern C/C++ dependency visibility and stronger manifest-mode provenance |
| Bun Lockfile | ✨ New Feature + 🔍 Enhanced | No Bun lockfile parser, no Bun binary lockfile compatibility, and no Bun lockfile sibling/workspace assembly coverage | Added JSONC-aware bun.lock parsing, static legacy bun.lockb v2 compatibility, workspace-aware dependency extraction, and Bun sibling/workspace assembly support |
Better Bun dependency visibility, safer legacy Bun compatibility, cleaner npm-family assembly, and stronger modern JS lockfile coverage |
| npm Workspace | ✨ New Feature | NonAssemblable stub + basic assembly | Workspace extraction + improved assembly | Monorepo structure visibility + correct package counts |
| npm + Yarn | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Missing overrides, dropped package.json platform metadata, anonymous hard-failure fallback rows for malformed package.json, incomplete modern lockfile/root handling, lossy Yarn Berry non-@npm: resolution detail, no Yarn manifest-based scope inference, and package-root ownership gaps |
Preserved overrides, package.json platform metadata, identified malformed-manifest fallback rows, richer Yarn Berry resolution metadata, fixed scoped npm fallback URLs and metadata normalization, modern lockfile/link handling, Yarn scope inference, and deterministic nested/workspace ownership |
Better npm/Yarn parity, clearer malformed-manifest output, cleaner lockfile metadata, better manifest compatibility visibility, and correct bundled/workspace package attribution |
| Structured metadata | ✨ New Feature + 🔍 Enhanced | CITATION.cff and publiccode.yml were only tracked as planned metadata surfaces, not shipped parsers |
Added bounded standalone parsers for CITATION.cff and publiccode.yml, including SPDX-aware declared-license normalization, party extraction, parser goldens, and datasource accounting |
Better project provenance and descriptive metadata without evaluating ecosystem tooling |
| Composer | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Alternate file names unsupported, nested package files not assigned, and large lockfiles still too eager in memory behavior | Added alternate-name support, nested package resource assignment, lighter large-lock extraction, and richer manifest provenance/party typing | Better Composer package detection, better nested file attribution, safer large-lock handling, and richer manifest provenance |
| Conda | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Installed package files from conda-meta not assigned, ambiguity between symbolic channel namespace and channel URL prefixes, and parser/assembly/docs drift around real environment filename aliases |
Added conda-meta file assignment via merged rootfs metadata, explicit channel-vs-channel_url disambiguation, and consistent .yml / .yaml / hyphenated environment alias handling across parsing, assembly, and supported-format metadata |
Better Conda rootfs package assignment, cleaner package identity/origin semantics, and more reliable environment-file detection |
| Alpine | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Missing APKBUILD recipe parsing, no APKBUILD dependency extraction, uneven license normalization across Alpine surfaces, incorrect raw matched text for custom:multiple, fileless package confusion, and no HTTPS Alpine VCS URL |
Added static APKBUILD parsing with dependency families, preserved raw custom:multiple matched text, normalized trustworthy licenses across APKBUILD / installed-db / .apk, retained fileless packages, and emitted HTTPS Alpine commit vcs_url |
Better Alpine recipe parity, safer package evidence, cleaner Alpine provenance, and more consistent declared license metadata |
| Arch | ✨ New Feature + 🔍 Enhanced | Upstream work is currently .SRCINFO-only, with no .PKGINFO parser, no legacy .AURINFO compatibility, and no Rust implementation |
Added .SRCINFO, legacy .AURINFO, and .PKGINFO parsing, real alpm package typing/purls, split-package inheritance, and arch-specific dependency scope preservation |
Better Arch/AUR package visibility, proper alpm identities, and fuller source/binary metadata coverage |
| Ruby | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | External gemspec constants unresolved, Gemfile manifest provenance was dropped, extracted gem duplication, missing nested key-file tagging, and no Ruby-driven license clarity score | Added required-file constant resolution, Gemfile + Gemfile.lock provenance preservation, extracted-gem merge dedupe, nested Ruby file-to-package assignment, key-file tagging, package metadata promotion, and summary license clarity scoring | Better Ruby gemspec parity, stronger Bundler manifest/lockfile provenance, cleaner extracted-gem results, and correct nested key-file clarity |
| SBT | ✨ New Feature + 🔍 Enhanced | No shipped Python SBT parser | Added bounded static build.sbt parsing for top-level literal metadata, literal license/homepage forms, direct/config-prefixed libraryDependencies, root-safe .settings(...), same-file literal Seq(...) bundle reuse, and explicit non-evaluating guardrails |
Better JVM/Scala package visibility without executing Scala |
| Meson | ✨ New Feature + 🔍 Enhanced | No shipped Python Meson parser | Added bounded static meson.build parsing for literal project() metadata, top-level dependency() declarations, official license_files/meson_version capture, and explicit no-evaluation guardrails |
Better native-project package visibility without executing Meson |
| Nix | ✨ New Feature + 🔍 Enhanced + 🛡️ Security | No shipped Python Nix package parser | Added static flake.nix metadata extraction, flake.lock root-input parsing, bounded default.nix mkDerivation support, and flake sibling assembly without Nix evaluation |
Better reproducible-build package visibility with deterministic flake metadata and safe bounded derivation parsing |
| Helm | ✨ New Feature + 🔍 Enhanced | No shipped Python Helm parser | Added static Chart.yaml and Chart.lock parsing, chart metadata extraction, maintainer/dependency preservation, and sibling assembly for declared plus locked Helm dependencies |
Better Kubernetes chart package visibility with declared and pinned dependency state |
| Pixi | ✨ New Feature + 🔍 Enhanced | No shipped Python Pixi parser | Added static pixi.toml and version-gated pixi.lock parsing, mixed Conda/PyPI dependency extraction, environment metadata preservation, and sibling assembly for manifest plus lock state |
Better reproducible Pixi workspace visibility with direct and locked dependency state |
| Clojure | ✨ New Feature + 🔍 Enhanced | No shipped Python Clojure parser | Added bounded static deps.edn and project.clj parsing, alias/profile dependency extraction, and literal-only guardrails for Leiningen metadata without code evaluation |
Better JVM/Clojure package visibility without evaluating project code |
| Dart | 🔍 Enhanced | Scope always null, YAML lossy, lockfile intent flattened, dependency source descriptors dropped, and publish_to / extra pub metadata under-preserved |
Proper scope and YAML preservation, lockfile direct/dev/transitive classification, hosted/git/path/sdk descriptor preservation, legacy sdk: compatibility, publish_to: none privacy mapping, archive_url download handling, and richer pubspec metadata capture |
Correct dependency classification, better lockfile provenance, and more truthful Dart package metadata |
| OS Release | 🐛 Bug Fix + 🔍 Enhanced | Debian name logic bug + no URL extraction | Fixed name logic + HOME/SUPPORT/BUG URLs | Correct distro identification + richer metadata |
| Conan Data | 🔍 Enhanced | Only extracts primary source URL | Patches metadata + mirror/fallback URLs | Complete source provenance tracking |
| Docker | ✨ New Feature + 🔍 Enhanced | No packagedcode Docker parser for Dockerfile/Containerfile package metadata | Added Dockerfile/Containerfile recognition, OCI label extraction, and non-assembled package data handling | Container metadata visibility and alternative Dockerfile-name support |
| Deno | ✨ New Feature | No Deno parser | Added direct deno.json / deno.jsonc / deno.lock parsing with import-map dependency extraction, v5 lockfile support, and sibling assembly |
Better JS/TS dependency visibility for Deno projects |
| pylock.toml | ✨ New Feature | No pylock.toml parser |
Added direct PEP 751 / PyPA pylock.toml parsing with dependency graph recovery, provenance extraction, and sibling assembly support |
Better Python dependency visibility for standards-based lockfiles |
| UV Lock | ✨ New Feature | No uv.lock parser |
Added direct uv lockfile parsing with root-package recovery, dependency groups, resolved packages, and sibling assembly support | Better Python dependency visibility for uv-managed projects |
| Python | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Rich setup.cfg metadata, unresolved imported-module setup.py dunder patterns, private-package classifiers, installed/source metadata file references, installed wheel provenance, exact-filename pypi.json parsing, pip cache provenance, RFC822 dependency extraction, direct source-distribution archive support, archive-read hardening for suspicious metadata entries, anonymous degraded rows for malformed Python package inputs, and unowned requirements/ subdirectory dependency evidence were incomplete or missing |
Added richer setup.cfg metadata, setup.py OrderedDict URL handling, imported-module dunder fallback, private-package detection, installed metadata file-reference collection/assignment, sibling WHEEL enrichment, project-root attachment for requirements/ subdirectory dependency files, direct sdist archive parsing, bound zip validation to actual archive reads, tar/zip archive hardening, datasource-preserving fallback identity for malformed Python package inputs, pip cache origin.json parsing/merge, source SOURCES.txt collection, saved pypi.json parsing, and RFC822 dependency extraction |
Better Python manifest fidelity, clearer degraded-input output, safer metadata recovery, richer installed-package provenance, broader PyPI package-input coverage, and stronger archive safety |
| Swift | 🐛 Bug Fix + 🔍 Enhanced | Root-package identity depended too heavily on whichever Swift artifact was available, malformed manifest fallback rows were anonymous, lockfile fallback overstated direct/runtime intent, and local path dependencies were dropped from dumped manifests | Added Swift-specific manifest/show-dependencies/resolved precedence, identified malformed-manifest fallback rows, resolved-only package emission, local path dependency preservation, less-assertive resolved/show-dependencies intent flags, and resolved fallback that enriches manifest-known dependencies instead of flattening every pin into a direct root edge | Correct top-level Swift packages, clearer malformed-manifest output, better dependency fidelity, safer file ownership, and more truthful manifest-vs-lock intent handling |
| CPAN dist.ini | ✨ New Feature + 🔍 Enhanced | Stub-only handler (returns empty) | Full INI parsing with runtime/test/build/configure dependency scopes plus declared-license normalization for Dist::Zilla metadata | Perl Dist::Zilla metadata extraction with correct configure-time dependency semantics and SPDX-aware declared license output |
| Swift Dependencies | 🔍 Enhanced | Only extracts root package name | Full dependency graph with versions + direct/transitive | Complete Swift dependency visibility |
| Maven | 🐛 Bug Fix + ✨ Feature + 🔍 Enhanced | Flattened licenses, missing dependencyManagement/relocation/qualifier support, older POM assumptions | Structured licenses + dependencyManagement/relocation/qualifier support + Maven 4.1.0 support | Better Maven parity, richer SBOM metadata, and modern POM compatibility |
| NuGet | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Missing project-style manifests, bounded CPM property propagation, version backfill for versionless PackageReference dependencies, empty party types, and weak modern nuspec/archive license handling | Added project.json/project.lock.json/PackageReference support, standalone Directory.Packages.props and Directory.Build.props parsing, nearest-ancestor plus bounded parent-import CPM backfill, bounded build-props participation, literal property-backed VersionOverride support, typed parties, modern license hints, and archive-backed license-file extraction |
Better .NET package coverage, stronger bounded CPM recovery, and stronger modern NuGet metadata |
| Cargo | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Missing crate/member file ownership, manifest-declared file references, lockfile dependency provenance, and smaller parser parity gaps around lowercase files plus readme/publish metadata | Added crate/workspace file assignment, explicit Cargo.toml file_references for readme/license-file, preserved Cargo.lock source/checksum provenance, lowercase Cargo filename matching, and readme/publish extraction |
Better Rust crate ownership, better manifest/lockfile provenance, better workspace attribution, and tighter Cargo parser parity |
| Compiled Binary | ✨ New Feature + 🔍 Enhanced + 🛡️ Security | Compiled Go and Rust package extraction depends on optional external inspectors in Python core, Python's Windows PE support is a separate narrow handler, and cargo-auditable dependency recovery is not built into the default packagedcode path | Added native core support for Go build-info, Rust cargo-auditable compiled binaries, and bounded Windows PE VERSIONINFO package identity extraction, recovered Rust dependency edges with dedicated binary datasources/PURLs, and bounded oversized Rust audit decompression |
Better out-of-the-box compiled-artifact coverage, stronger Rust dependency visibility, narrower Windows executable parity, and safer compiled-binary parsing |
| ABOUT | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Falls back to invalid pkg:about/... PURLs, misses download_url-based ecosystem inference, and couples file-reference promotion to reopening ABOUT resources |
Added real PURL/type inference from download_url, suppressed invalid pkg:about synthesis, preserved ABOUT identity on partial files, and resolved ABOUT file references during scan-time promotion |
Better ABOUT package identity, safer fallback behavior, and more reliable referenced-resource promotion |
| CocoaPods | 🐛 Bug Fix + 🔍 Enhanced | Scope semantics remain muddy across pod manifests/lockfiles, and historical duplicate-output regressions need explicit guardrails | Refined scope/runtime semantics across .podspec, .podspec.json, Podfile, and Podfile.lock, and bounded duplicate-output protection for RxDataSources.podspec |
More honest dependency semantics and safer protection against explosive duplicate package output |
| Go | 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced | Missing graph support, thin directive fidelity, and Go test/build file categorization gaps | Added go.mod graph parsing, replace/retract/toolchain directive fidelity, fallback datasource fixes, and build-tag-aware Go source categorization |
Better module graph fidelity, safer assembly accounting, and more accurate Go source heuristics |
| Hackage | ✨ New Feature + 🔍 Enhanced | No Haskell package parser for *.cabal, cabal.project, or stack.yaml |
Added static parsing for package metadata, component build-depends, project/workspace surfaces, config preservation, and sibling assembly across the three Hackage inputs |
Better Haskell package visibility, cleaner project provenance, and assembled multi-surface Hackage results |
| Hex Lockfile | ✨ New Feature | No Hex lockfile parser and no safe lockfile-only Elixir dependency extraction | Added static mix.lock parsing with locked package versions, nested dependency reconstruction, repo/checksum preservation, and explicit no-code-execution scope |
Better Elixir/Hex dependency visibility without executable manifest parsing |
| npm Git URLs | 🐛 Bug Fix | Git URLs treated as pinned versions | Correct is_pinned=false for non-version deps | Valid PURLs + correct dependency resolution status |
| Gitmodules | ✨ New Feature | No .gitmodules parser | Full submodule dependency extraction | Complete dependency graphs for projects using submodules |
| Copyright Detection | 🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security | Year range stops at 2039, short-year typo, French/Spanish case bugs, string-based POS tags, global mutable singleton, and more aggressive emitted copyright normalization | Year range 2099, all regex bugs fixed, type-safe enum POS tags, thread-safe LazyLock, shared media metadata clues from supported images and fonts, and source-faithful file-level copyright output by default with an opt-in ScanCode compatibility mode |
Correct year detection, reliable i18n, compile-time safety, better compliance fidelity, and parallel scanning |
| Email/URL Detection | 🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security | TLD length too strict, IPv6/private-IP issues, less explicit URL handling | Extended TLD support, robust host/IP filtering, credential stripping, and shared metadata clues from supported images and fonts | Better extraction correctness and safer metadata handling |
| License Detection | 🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security + ⚡ Performance | SPDX-LID fan-out duplicates, stale whole-query cache after SPDX subtraction, bounds-only qcontains/qoverlap, zero-overlap surround merges, per-candidate rule cloning, and global mutable state |
SPDX-LID deduplication, immutable whole-query snapshot for AHO, position-set-aware containment/overlap, explicit AHO extra-matchables tracking, unified PositionSpan enum, borrowed rule references in candidates, and thread-safe Arc<LicenseDetectionEngine> |
Correct SPDX match counts, clean architectural separation from stale-cache coupling, correct positional overlap semantics, reduced allocation overhead, and safe parallel scanning |
| CLI Workflows | ✨ New Feature + 🐛 Bug Fix | Selected-file and PR-changed-file scans still depend on argv expansion, include-filter approximations, or cwd-sensitive multi-input behavior, and repeated full rescans still pay the full cost on unchanged trees | Added rooted --paths-file native-scan selection with stdin support, root-relative entries, recoverable missing-entry warnings, cwd-independent explicit-root workflows, and opt-in --incremental unchanged-file reuse under a shared cache root |
Better PR/changed-file scanning, cleaner CI/container ergonomics, safer selected-file scope control, and faster repeated native rescans |
| Cross-cutting (All Parsers) | 🛡️ Security | No DoS limits | File size + iteration limits | Protection against resource exhaustion |
Most areas with improvements have a dedicated document. Cross-cutting security hardening and broader infrastructure improvements are documented elsewhere when they span multiple subsystems:
- rpm-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: EVR preservation, hash-based source RPM detection, richer archive metadata, and YumDB enrichment
- debian-parser.md — ✨ New Feature + 🔍 Enhanced Extraction + 🐛 Bug Fix + ⚡ Performance: direct
.debintrospection, embedded copyright recovery, coherent DEP-5 detections, installed Debian sidecar integration, and a Debian-local paragraph parsing optimization - conan-parser.md — ✨ New Feature: conanfile.txt and conan.lock parsers (Python has neither)
- bitbake-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: bounded
.bb+.bbappendparsing, override-style dependency handling, raw declared-license preservation, local file-reference recovery, and sibling assembly support - carthage-parser.md — ✨ New Feature + 🔍 Enhanced Extraction:
Cartfile,Cartfile.private, andCartfile.resolvedparsing with origin-aware dependency metadata and sibling assembly for declared plus pinned state - cpan-parser.md — ✨ New Feature + 🐛 Bug Fix: Full META.json, META.yml, MANIFEST parsing plus datasource-preserving fallback identity for malformed/unreadable CPAN inputs
- gradle-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: correct Gradle scope classification, all discovered dependency-block parsing, version-catalog alias resolution, Gradle POM license extraction, and nested project-path identifiers
- gradle-module-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: direct Gradle
.moduleparsing with artifact checksums, deduplicated dependency extraction, preserved variant metadata, and file references for published artifacts - gradle-lockfile-parser.md — ✨ New Feature: gradle.lockfile dependency extraction (Python has no equivalent)
- vcpkg-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: strict-JSON
vcpkg.jsonparsing, project/port manifest support, direct dependency extraction, and embedded/sibling configuration metadata preservation - bun-lock-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: JSONC-aware
bun.lockparsing, static legacybun.lockbv2 compatibility, and sibling/workspace assembly integration - arch-parser.md — ✨ New Feature + 🔍 Enhanced Extraction:
.SRCINFO, legacy.AURINFO, and.PKGINFOparsing withalpmpurls, split-package inheritance, and arch-specific dependency scope preservation - npm-workspace-parser.md — ✨ New Feature: pnpm-workspace.yaml metadata extraction (Python has stub only)
- npm-yarn-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: npm overrides plus platform metadata preservation, scoped URL/metadata normalization fixes, richer Yarn Berry resolution detail, modern lockfile/link handling, Yarn scope inference, and nested/workspace ownership fixes
- structured-metadata-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: bounded
CITATION.cffandpubliccode.ymlparsers for standalone project metadata, SPDX-aware declared-license recovery, and maintainer/author metadata extraction - composer-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: alternate Composer file names, nested package file assignment, lighter lockfile handling, richer manifest provenance, and declared-license normalization for trustworthy
composer.jsonlicense metadata - ruby-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: required-file constant resolution, Gemfile + Gemfile.lock provenance preservation, extracted-gem merge/resource assignment, and nested key-file / license-clarity propagation
- dart-parser.md — 🔍 Enhanced Extraction: proper scope handling, YAML trailing newline preservation, lockfile direct/dev/transitive classification, dependency descriptor preservation, and richer pubspec metadata
- os-release-parser.md — 🐛 Bug Fix + 🔍 Enhanced: Debian name logic fix + URL extraction (HOME, SUPPORT, BUG)
- conan-data-parser.md — 🔍 Enhanced Extraction: Patches metadata + mirror/fallback URL extraction
- docker-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: Dockerfile/Containerfile recognition, OCI label extraction, and non-assembled package data handling
- deno-parser.md — ✨ New Feature:
deno.json(c)identity/import parsing,deno.lockv5 extraction, and sibling assembly support - bazel-module-parser.md — ✨ New Feature:
MODULE.bazelparsing withbazel_depextraction, dev-scope support, and override metadata preservation - go-work-parser.md — ✨ New Feature:
go.workparsing with workspace-member recovery, replace metadata extraction, and root-module assembly support - pylock-toml-parser.md — ✨ New Feature: standardized
pylock.tomlparsing, dependency graph recovery, provenance extraction, and Python sibling assembly support - uv-lock-parser.md — ✨ New Feature: uv.lock root-package recovery, dependency groups, resolved package extraction, and Python sibling assembly support
- python-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: richer
setup.cfgmetadata, setup.py imported-dunder/project-URL fixes, private classifier support, direct sdist archive parsing plus archive hardening, installed/source metadata file-reference handling, siblingWHEELenrichment, pip cacheorigin.jsonprovenance, exact-filenamepypi.jsonparsing, and RFC822 dependency extraction - swift-parser.md — 🐛 Bug Fix + 🔍 Enhanced Extraction: Swift-specific root-package assembly precedence, local path manifest dependency preservation, less-assertive lock/show intent flags, resolved fallback enrichment, and nested-root-safe file ownership
- cpan-dist-ini-parser.md — ✨ New Feature + 🔍 Enhanced: Full dist.ini parsing with runtime/test/build/configure dependency extraction (Python has stub only)
- swift-show-dependencies-parser.md — 🔍 Enhanced Extraction: Full dependency graph with versions, direct/transitive marking, and less-assertive runtime intent
- rpm-specfile-parser.md — ✨ New Feature: Full RPM spec preamble parsing (Python is stub with TODO)
- cpan-makefile-pl-parser.md — ✨ New Feature + 🔍 Enhanced: Makefile.PL WriteMakefile extraction plus bounded VERSION_FROM / ABSTRACT_FROM recovery (Python has no parse method)
- osgi-manifest-parser.md — ✨ New Feature: OSGi bundle metadata extraction with mixed-manifest Maven identity fallback when strong Maven coordinates are present (Python has empty patterns)
- maven-parser.md - 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: structured licenses, dependencyManagement/relocation/qualifier support, Maven 4.1.0 support, and richer normalized metadata
- nuget-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: nearest-ancestor NuGet CPM backfill plus bounded parent-import/build-props participation, literal
VersionOverridesupport for PackageReference dependencies in project files, standaloneDirectory.Packages.props/Directory.Build.propsparsing, modern nuspec license hints,.nupkglicense-file extraction,.deps.jsonruntime graph parsing, and support for legacy/project-style NuGet manifests - cargo-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: crate/workspace file ownership, Cargo.toml file references, Cargo.lock provenance preservation, lowercase Cargo file matching, and
readme/publishmanifest parity - compiled-binary-parser.md — ✨ New Feature + 🔍 Enhanced Extraction + 🛡️ Security: native Go build-info, Rust cargo-auditable, and bounded Windows PE version-resource extraction in core scanning, Rust dependency-edge recovery, and bounded Rust audit decompression
- go-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction:
go.mod graphsupport,replace/retract/toolchaindirective fidelity, and Go test/build source categorization - hackage-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: static
*.cabal/cabal.project/stack.yamlparsing with component dependency recovery, project-config preservation, and sibling assembly support - sbt-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: bounded static
build.sbtparsing for top-level literal metadata, literal license/homepage forms, literallibraryDependencies, limited string alias reuse, and explicit no-evaluation guardrails - meson-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: bounded static
meson.buildparsing for literalproject()metadata, top-leveldependency()declarations, officiallicense_files/meson_versioncapture, and explicit no-evaluation guardrails - nix-parser.md — ✨ New Feature + 🔍 Enhanced Extraction + 🛡️ Security: static
flake.nix/flake.locksupport, boundeddefault.nixmkDerivationextraction, and flake sibling assembly without Nix evaluation - helm-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: static
Chart.yaml/Chart.lockparsing with chart metadata recovery, declared dependency extraction, pinned lock metadata, and sibling assembly support - pixi-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: static
pixi.toml/ version-gatedpixi.lockparsing with workspace metadata recovery, mixed Conda/PyPI dependency extraction, environment metadata preservation, and sibling assembly support - clojure-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: bounded static
deps.edn/project.cljparsing with Maven/Clojars-style dependency extraction, alias/profile support, and explicit no-evaluation guardrails - hex-lock-parser.md — ✨ New Feature: static
mix.lockparsing with locked Hex dependency extraction and nested dependency reconstruction under a no-code-execution scope - conda-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction:
conda-metafile assignment and channel-vs-channel_url disambiguation - alpine-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: APKBUILD recipe parsing, raw matched-text preservation for
custom:multiple, retained fileless packages, and HTTPS Alpine VCS URLs - about-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: PURL/type inference from
download_url, invalidpkg:aboutsuppression, graceful partial ABOUT handling, and scan-time file-reference resolution - cocoapods-parser.md — 🐛 Bug Fix + 🔍 Enhanced Extraction: refined dependency scope semantics and bounded duplicate-output protection for
RxDataSources.podspec - npm-git-url-dependencies.md — 🐛 Bug Fix: Correct handling of Git URLs, GitHub shortcuts, and local paths (Python treats them as pinned versions)
- gitmodules-parser.md — ✨ New Feature: Git submodule dependency extraction (Python has no equivalent parser)
- copyright-detection.md — 🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security: Year range fix, regex typo fixes, type-safe POS tags, thread-safe design, and source-faithful copyright output by default
- email-url-detection.md — 🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security: Email/URL extraction hardening, stronger filtering, and EXIF/XMP-backed metadata detection
- license-detection.md — 🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security + ⚡ Performance: SPDX-LID deduplication, position-set-aware containment/overlap, immutable whole-query snapshot for AHO, explicit extra-matchables tracking, unified PositionSpan, and borrowed rule references in candidates
- cli-workflows.md — ✨ New Feature + 🐛 Bug Fix: rooted
--paths-fileselected-file scanning plus opt-in--incrementalrescans with cache-root controls for repeated native workflows
Related broader architecture/security docs:
- Cross-cutting parser security hardening - File size limits, iteration limits, and archive-safety policy are documented in ADR 0004: Security-First Parsing.
- Incremental rescans - The implemented unchanged-file reuse workflow is documented in ARCHITECTURE.md.
- Scan-result shaping parity - The shaping pipeline and landed behavior are documented in ARCHITECTURE.md#post-processing-pipeline.
For the general contributor workflow, local setup, testing guidance, and pull request conventions, start with ../CONTRIBUTING.md.
When implementing a parser or detection subsystem, if you discover:
- A bug in Python: Fix it in Rust, document here
- A TODO in Python: Implement it in Rust, document here
- Missing extraction: Add it in Rust, document here
Template for new improvement docs: See existing files for structure.
- ADR 0002: Extraction vs Detection Separation - Why we separate concerns better than Python
- ADR 0004: Security-First Parsing - Our security advantages over Python
- SUPPORTED_FORMATS.md - Auto-generated list of all supported formats