diff --git a/CHANGELOG.md b/CHANGELOG.md index 4b7210d..e9a73d6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,12 @@ # Changelog -## Unreleased +## [0.1.5] - 2026-07-03 + +### Added + +- Added `capture_figma_metadata_shards.py` to stage sharded Figma `get_metadata` captures into canonical raw metadata shards, index, and node inventory artifacts without passing large provider responses through agent context. +- Added IA-matrix preview readiness checks that require page visual-state enumerations, page IA matrices, component visual-state enumerations, component IA matrices with interaction refs, and coverage evidence conclusions. +- Added structured UI/visual asset source-of-truth checks that block evidence packets, preview HTML, screenshots, and visual diffs from being used as `source_refs` or `evidence_refs`. ## [0.1.4] - 2026-07-01 diff --git a/README.md b/README.md index 428005b..2b804cd 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,8 @@ # Spec Kit Intake Extension -Extract, normalize, and validate SDD-ready intake artifacts from PRDs, visual designs, visual requirements/spec structured asset packages, optional preview evidence, test cases, and other software sources before downstream Spec Kit workflows project them into requirements. +Extract, normalize, and validate SDD-ready intake artifacts from PRDs, visual designs, structured UI/visual assets, optional preview evidence, test cases, and other software sources before downstream Spec Kit workflows project them into requirements. + +For UI intake, the structured UI/visual asset is the source of truth. Evidence packets support confidence, and preview HTML supports human reading of page-component IA; neither may create, override, or replace structured asset records. The first goal of intake is not to generate requirements. It is to preserve as much input information as possible and turn it into structured material that SDD `specify` can consume accurately. @@ -15,7 +17,7 @@ Intake artifacts are validated in two layers: JSON Schema checks enforce the req - Static images such as PNG, JPG, WebP, and exported screens - PDF design packs and annotated review documents - Figma files, pages, frames, nodes, components, variables, and exported screenshots -- Optional Figma-derived component matrix preview and coverage review helpers with traceable component, state, variant, viewport, and screenshot coverage +- Optional Figma-derived IA matrix preview and coverage review helpers with traceable page, component, visual-state, interaction, variant, viewport, and screenshot coverage - Visual requirements/spec structured asset packages with CI-friendly DOM, ARIA, token, state, locator, relation, and assertion facts - Existing test cases, Gherkin files, QA exports, and test management spreadsheets @@ -27,7 +29,7 @@ Intake commands are organized by vertical source domain. Each domain uses the sa | --- | --- | --- | --- | | PRD | product briefs, Markdown PRDs, exported docs, PDFs, issue or epic links, mixed stakeholder notes | `prd-intake.yaml` | source identity, product intent traceability, scope boundaries, acceptance evidence, clarification gaps | | Visual design | static images, wireframes, PDF design packs, Markdown design briefs, Figma files or selected nodes | `visual-requirements.yaml` and `visual-spec-package/` | source integrity, Figma-backed resource traceability, fidelity rules, visual requirement traceability, structured visual spec readiness, CI-low-cost assertion readiness | -| Figma preview helper | Figma files or selected nodes projected into component matrix review surfaces | `previews/component-matrix-preview.html` plus coverage YAML | Figma component coverage, component-state coverage, variant coverage, viewport screenshots, known gaps | +| Figma preview helper | Figma files or selected nodes projected into IA matrix review surfaces | `previews/component-matrix-preview.html` plus coverage YAML | Figma page coverage, component coverage, visual-state coverage, fused interaction coverage, variant coverage, viewport screenshots, known gaps | | Test cases | automated tests, Gherkin files, manual QA cases, spreadsheets, test management exports, bug or issue repro steps | `test-case-intake.yaml` | scenario traceability, assertion extraction, fixture evidence, coverage gaps, flaky or skipped case reporting | Vertical instructions should never convert source evidence directly into downstream-owned requirement IDs, implementation tasks, or code component names. They produce provider-neutral intake facts that downstream workflows can consume with source refs intact. @@ -77,6 +79,9 @@ Figma metadata artifacts are required for Figma visual-design sources. Image, PD Machine-readable JSON Schemas live under `templates/schemas/` and are used by the validators before readiness rules run. Preview helpers are defined by `templates/intake-visual-previews-contract.md` and use `component-coverage.schema.json` and `viewport-coverage.schema.json`. Visual spec packages use `visual-spec-package.schema.json` and `visual-spec-assertions.schema.json`. All intake commands provide capture instructions, evidence contracts, and readiness validation. Visual design validation additionally checks visual fidelity and Figma metadata parity. +Visual validators reject evidence packets, preview HTML, screenshots, and visual diffs when they appear in source-of-truth fields such as `source_refs` or `evidence_refs`; use preview-specific helper refs for those supporting artifacts. +Figma metadata capture should use bounded shards: stage raw `get_metadata` response files with `scripts/python/capture_figma_metadata_shards.py` so large provider responses are written to disk and only shard paths, hashes, completeness, and inventory parity flow into the intake index. +Keep raw capture files outside the target `visual-design/` directory before staging, and pass one `--node-id` per selected root when a capture spans multiple roots. Component matrix preview validation is owned by `scripts/python/validate_visual_previews.py`, including cross-file checks for preview refs, visual spec refs, screenshots, component coverage, viewport coverage, and known gaps. Visual spec package validation is owned by `scripts/python/validate_visual_spec_package.py`, including source readiness, schema, cross-reference, locator, downstream-ownership, provider-evidence, product-ambiguity, design-source resource traceability, and CI assertion checks. @@ -99,7 +104,7 @@ specify extension add --dev C:/Users/24598/Documents/github/spec-kit-intake From a Spec Kit project: ```bash -specify extension add intake --from https://github.com/bigsmartben/spec-kit-intake/archive/refs/tags/v0.1.4.zip +specify extension add intake --from https://github.com/bigsmartben/spec-kit-intake/archive/refs/tags/v0.1.5.zip ``` Release artifacts must include source-backed provenance for the `bigsmartben/spec-kit` integration fork. The release workflow uploads `release-provenance.json` with: @@ -150,18 +155,19 @@ For irregular Figma sources, intake may generate bounded candidate completions, ## Figma Preview Helper Readiness Gate -Figma-derived component matrix preview evidence passes only when: +Figma-derived IA matrix preview evidence passes only when: - upstream Figma visual-design intake is ready -- every required component set, component instance, variant prop, state, size, density, theme, content sample, resource, and viewport is covered or recorded as a blocking missing record -- the minimum acceptance grain is covered: component instance, state, content sample, container constraint, and viewport -- required preview cells link back to Figma refs, `visual-spec.yaml` items, `component-coverage.yaml` records, and screenshot or diff evidence when available +- `component-matrix-preview.html` is structured as: IA matrix overview; for each page, visual state enumeration plus page IA matrix; for each component, visual state enumeration plus component IA matrix with event interaction information; coverage evidence conclusion +- every required page, component set, component instance, variant prop, state, size, density, theme, content sample, interaction, resource, and viewport is covered or recorded as a blocking missing record +- the minimum acceptance grain is covered: page state, page IA row, component instance, component state, component IA row, interaction event, content sample, container constraint, and viewport +- required preview cells and interaction rows link back to Figma refs, `visual-spec.yaml` items, `component-coverage.yaml` records, and screenshot or diff evidence when available - component coverage is expressed in `previews/component-coverage.yaml` - viewport coverage is expressed in `previews/viewport-coverage.yaml` - screenshot coverage and visual-diff status are recorded - known gaps are explicit and no blocking gap remains unresolved -The preview validator emits blocker codes such as `VISUAL_PREVIEW_SOURCE_INTAKE_BLOCKED`, `VISUAL_PREVIEW_REQUIRED_ARTIFACT_MISSING`, `VISUAL_PREVIEW_SCHEMA_INVALID`, `VISUAL_PREVIEW_FIGMA_NODE_COVERAGE_INCOMPLETE`, `VISUAL_PREVIEW_COMPONENT_STATE_COVERAGE_INCOMPLETE`, `VISUAL_PREVIEW_PAGE_COVERAGE_INCOMPLETE`, `VISUAL_PREVIEW_ASSET_TRACEABILITY_INCOMPLETE`, `VISUAL_PREVIEW_VIEWPORT_CAPTURE_INCOMPLETE`, `VISUAL_PREVIEW_VISUAL_DIFF_BLOCKED`, and `VISUAL_PREVIEW_KNOWN_GAP_UNRESOLVED`. +The preview validator emits blocker codes such as `VISUAL_PREVIEW_SOURCE_INTAKE_BLOCKED`, `VISUAL_PREVIEW_REQUIRED_ARTIFACT_MISSING`, `VISUAL_PREVIEW_SCHEMA_INVALID`, `VISUAL_PREVIEW_FIGMA_NODE_COVERAGE_INCOMPLETE`, `VISUAL_PREVIEW_COMPONENT_STATE_COVERAGE_INCOMPLETE`, `VISUAL_PREVIEW_IA_MATRIX_INCOMPLETE`, `VISUAL_PREVIEW_PAGE_COVERAGE_INCOMPLETE`, `VISUAL_PREVIEW_ASSET_TRACEABILITY_INCOMPLETE`, `VISUAL_PREVIEW_VIEWPORT_CAPTURE_INCOMPLETE`, `VISUAL_PREVIEW_VISUAL_DIFF_BLOCKED`, and `VISUAL_PREVIEW_KNOWN_GAP_UNRESOLVED`. ## Visual Spec Package Readiness Gate @@ -174,7 +180,7 @@ The visual requirements/spec structured asset package passes only when: - assertions reference existing visual spec items and include ready `ci_low_cost` checks - missing provider evidence and product ambiguity are represented with distinct blocker paths - locator strategies avoid implementation-owned CSS selectors, XPath, generated class names, downstream test IDs, code component names, tasks, or requirement IDs -- component matrix previews, screenshots, and visual diffs remain helper evidence rather than the target deliverable +- IA matrix previews, screenshots, and visual diffs remain helper evidence rather than the target deliverable The visual spec package validator emits blocker codes such as `VISUAL_SPEC_SOURCE_INTAKE_BLOCKED`, `VISUAL_SPEC_REQUIRED_ARTIFACT_MISSING`, `VISUAL_SPEC_SCHEMA_INVALID`, `VISUAL_SPEC_INTAKE_INCOMPLETE`, `VISUAL_SPEC_PROVIDER_EVIDENCE_MISSING`, `VISUAL_SPEC_PRODUCT_AMBIGUITY_UNRESOLVED`, `VISUAL_SPEC_ASSERTION_COVERAGE_INCOMPLETE`, `VISUAL_SPEC_LOCATOR_STRATEGY_INVALID`, `VISUAL_SPEC_DOWNSTREAM_OWNERSHIP_LEAK`, and `VISUAL_SPEC_READY_WITHOUT_EVIDENCE`. @@ -192,7 +198,19 @@ Validate visual-design artifacts: python scripts/python/validate_visual_design_intake.py specs//intake/visual-design ``` -Validate component matrix preview helpers: +Stage sharded Figma metadata captures before validation: + +```bash +python scripts/python/capture_figma_metadata_shards.py specs//intake/visual-design \ + --metadata-source ./figma-metadata-raw \ + --file-url \ + --file-key \ + --page-id \ + --node-id \ + --overwrite +``` + +Validate IA matrix preview helpers: ```bash python scripts/python/validate_visual_previews.py specs//intake/visual-design/previews diff --git a/commands/speckit.intake.visual-design.md b/commands/speckit.intake.visual-design.md index 712d299..828912f 100644 --- a/commands/speckit.intake.visual-design.md +++ b/commands/speckit.intake.visual-design.md @@ -12,12 +12,15 @@ Classify the input before proceeding: - `source`: image, PDF, Markdown design brief, Figma URL, file, page, frame, node, or exported design asset - `intake_dir`: existing visual-design intake artifact directory -- `validation_request`: validate, check, gate, readiness, build-spec-package, validate-spec-package, build-previews, validate-previews, or CI-friendly assertion request +- `validation_request`: validate, check, gate, readiness, validate-spec-package, or validate-previews +- `asset_request`: build-spec-package, build-previews, downstream delivery asset request, preview coverage asset request, or CI-low-cost assertion asset request - `review_guidance`: target platform, required fidelity, capture scope, source precedence, Figma-backed resource requirements, or reviewer instructions ## Goal -Create, update, or validate provider-neutral visual design intake artifacts for the active Spec Kit feature. Intake preserves reachable design sources, raw provider evidence, stable source refs, checksums or retrieval metadata, schema-required visual facts, and the visual requirements/spec structured asset package so downstream SDD workflows can implement high-fidelity UI with traceability. +Create, update, or validate high-confidence provider-neutral structured UI/visual assets for the active Spec Kit feature. The structured UI/visual asset is the source of truth for this intake: it records pages, regions, components, states, styles, resources, and interaction cues from high-fidelity UI or visual design sources so downstream product specification, implementation, and acceptance workflows can consume source-backed visual facts. + +Evidence chain and preview artifacts are supporting artifacts only: evidence supports confidence; preview supports human reading of page-component IA. Neither may create, override, or replace structured asset records. Default artifact directory: @@ -28,9 +31,10 @@ specs//intake/visual-design/ Normative authority: - `templates/schemas/*.json` defines machine-readable structure, required fields, types, and enums. -- `scripts/python/validate_visual_design_intake.py` defines readiness evaluation and blocker emission. -- `scripts/python/validate_visual_spec_package.py` defines structured visual spec package readiness. -- `scripts/python/validate_visual_previews.py` defines component matrix preview and coverage readiness. +- `scripts/python/validate_visual_design_intake.py` defines readiness evaluation and blocker emission for the core structured UI/visual asset. +- `scripts/python/capture_figma_metadata_shards.py` stages already-sharded Figma metadata captures into raw shard, index, and inventory artifacts. +- `scripts/python/validate_visual_spec_package.py` defines downstream structured visual spec package readiness after the core asset passes. +- `scripts/python/validate_visual_previews.py` defines supporting IA matrix preview and coverage readiness only for preview artifacts. - `templates/intake-visual-design-contract.md` defines semantic extraction policy, fidelity policy, and provider evidence policy. - `templates/intake-visual-spec-package-contract.md` defines the visual requirements/spec structured asset package. - `templates/intake-visual-previews-contract.md` defines preview coverage helper artifact structure, boundaries, and blocker semantics. @@ -40,15 +44,17 @@ Normative authority: - Preserve original design sources and record checksums before extraction. - For Figma sources, implementation resources, images, exported assets, and token refs must trace back to Figma source refs. -- Extract visual requirements as traceable engineering input, not as unsupported prose summaries or downstream-specific schema projections. -- Treat `visual-spec-package/` as the target structured visual requirements/spec asset package for downstream delivery and CI-friendly checks. +- Extract structured UI/visual asset records as traceable engineering input, not as unsupported prose summaries or downstream-specific schema projections. +- Treat `visual-spec-package/` as the downstream structured UI/visual asset package for delivery and CI-low-cost checks. - Treat `previews/component-matrix-preview.html`, screenshots, and visual diffs as optional human-review helper evidence only, not the target deliverable. - Treat `previews/component-coverage.yaml` and `previews/viewport-coverage.yaml` as structured coverage evidence for reviewer completeness checks; they may support readiness but do not replace `visual-spec-package/`. +- Do not place `visual-evidence-packet.md`, `visual-spec-evidence-packet.md`, `component-matrix-preview.html`, screenshots, visual diffs, or other preview artifacts in `source_refs` or `evidence_refs` as source-of-truth records. Use preview-specific helper fields such as `preview_refs` instead. - Use bounded inference for dirty or incomplete design sources: observed claims are source-backed facts; inferred claims require explicit rules and high confidence; candidate claims are reference-only; unsupported claims must remain blocked. - Mark low, medium, or high fidelity explicitly and apply the matching extraction rules. - Use stable provider-neutral evidence IDs and source refs. Do not invent downstream-owned item IDs, requirement IDs, schema fields, code component names, or product semantics. -- Do not mark intake ready unless source integrity, requirement traceability, fidelity proof, and intake parity planning pass. +- Do not mark intake ready unless source integrity, source refs, fidelity rules, bounded inference checks, and intake parity plan pass the validator readiness gates. - Preserve raw Figma metadata exactly in `figma-metadata.part-*.xml` for Figma sources. +- Do not request complete metadata for a broad Figma page, canvas, or board in one MCP response. Split large Figma scopes into smaller frame/component/node captures and stage them with `capture_figma_metadata_shards.py`. - Do not modify application source, tests, package manifests, feature implementation files, or existing Spec Kit core templates. - If required tooling is unavailable, create a blocked evidence packet that records the missing tool and stop before claiming readiness. @@ -58,7 +64,7 @@ Normative authority: 2. Identify the active feature: - Prefer `SPECIFY_FEATURE` when set. - Otherwise use the current Git branch name when it matches a directory under `specs/`. - - Otherwise inspect `specs/` and choose the most recent feature directory only if there is a single clear candidate. + - Otherwise inspect `specs/` and choose the most recently modified feature directory only when exactly one feature directory exists. - If the feature cannot be identified and no standalone artifact directory was provided, stop and ask the user to set `SPECIFY_FEATURE` or run from the feature branch. 3. Read `.specify/extensions/intake/intake-config.yml` when present. 4. Read `templates/intake-visual-design-contract.md` and the referenced JSON Schemas from this extension before creating or validating artifacts. @@ -66,10 +72,16 @@ Normative authority: ## Mode Routing +Apply routing precedence before executing a mode: + +- If both a `source` and `build-spec-package` intent are present, run capture then validate first. Continue to Build spec package mode only when the updated visual-design intake validator returns `PASS`; otherwise stop and report the visual-design blockers. +- If both a `source` and `build-previews` intent are present, run capture then validate first. Continue to Build previews mode only when the updated visual-design intake validator returns `PASS`; otherwise stop and report the visual-design blockers. +- If build-spec-package, validate-spec-package, build-previews, or validate-previews intent is present without a resolvable source or existing intake directory, stop and ask for the visual-design intake directory. + - Capture mode: use when `$ARGUMENTS` names an image, PDF, Markdown design brief, Figma URL, frame, node, platform, fidelity level, or asks to capture, ingest, update, or recapture visual evidence. -- Build spec package mode: use when `$ARGUMENTS` includes `build-spec-package`, `with spec package`, `structured visual spec`, `CI assertions`, or asks for downstream delivery/acceptance assets. +- Build spec package mode: use when `$ARGUMENTS` includes `build-spec-package`, `with spec package`, `structured visual spec`, `CI-low-cost assertions`, or asks for downstream delivery/acceptance assets. - Validate spec package mode: use when `$ARGUMENTS` includes `validate-spec-package`, `check spec package`, `visual spec readiness`, or only names an existing `visual-spec-package` directory. -- Build previews mode: use when `$ARGUMENTS` includes `build-previews`, `component matrix`, `preview coverage`, `coverage review`, `component-coverage`, or `viewport-coverage`. +- Build previews mode: use when `$ARGUMENTS` includes `build-previews`, `IA matrix`, `preview coverage`, `coverage review`, `component-coverage`, or `viewport-coverage`. - Validate previews mode: use when `$ARGUMENTS` includes `validate-previews`, `check previews`, `preview readiness`, or only names an existing `previews` directory. - Validate mode: use when `$ARGUMENTS` includes `validate`, `check`, `gate`, `readiness`, or only names an existing visual-design intake directory. - Capture then validate: use when both a source and validation intent are present, or after capture artifacts are updated. @@ -84,23 +96,39 @@ Normative authority: 2. Create `design-source-manifest.yaml` with contract-required source identity, integrity, coverage, capture method, and fidelity fields. 3. Preserve file-based originals under `source-files/`; for remote or Figma sources, record stable URLs and exported screenshots or assets, or record a structured gap/blocker when unavailable. 4. For Figma sources, preserve raw provider evidence before deriving normalized requirements: + - capture metadata in bounded node batches instead of one broad page/canvas response + - store raw responses outside the target intake directory before staging; do not point `--metadata-source` at the target `visual-design/` directory + - for multiple selected roots, pass `--node-id` once per root and avoid overlapping node scopes because duplicate or missing node parity blocks readiness + - stage raw metadata files with: + +```bash +python .specify/extensions/intake/scripts/python/capture_figma_metadata_shards.py \ + --metadata-source \ + --file-url \ + --file-key \ + --page-id \ + --node-id \ + --overwrite +``` + - write raw metadata shards as `figma-metadata.part-NNN.xml` - build `figma-metadata.index.yaml` - build `figma-node-inventory.yaml` - validate metadata and inventory parity before deriving visual requirements + - if any shard is truncated or lacks detectable node ids, keep the intake `BLOCKED` and retry with a smaller node scope or a direct file-based provider export 5. Extract source-specific evidence: - image: dimensions, regions, OCR status, visual hierarchy, assets, and region coverage - pdf: original file hash, page count, rendered page refs, text extraction status, and page coverage - markdown: heading structure, design notes, embedded or linked assets, and visual requirement mappings - figma: complete descendant metadata, node inventory, variables/styles/components, screenshots, and assets -6. Classify source-domain scenario coverage using `templates/intake-visual-design-contract.md`; do not define additional scenario categories in this command. +6. Record source coverage and extraction gaps using `design-source-manifest.yaml`, `visual-requirements.yaml`, and `templates/intake-visual-design-contract.md`; do not define scenario categories in this command. 7. Build `visual-requirements.yaml` according to `templates/schemas/visual-requirements.schema.json` and the semantic policies in `templates/intake-visual-design-contract.md`. - Record direct facts as `evidence_type: observed`. - Promote only rule-backed, high-confidence derived claims to `evidence_type: inferred` with `inference_rule`, `confidence_method`, `score_breakdown`, `downstream_use: accepted_claim`, and `blocking_conditions`. - Keep low- or medium-confidence completions as `evidence_type: candidate` with `downstream_use: reference_only` and `missing_evidence`. - Record unsupported or conflicting claims as `evidence_type: unsupported` with `blocker_code`, `reason`, `missing_evidence`, and `blockers`. 8. For unavailable required evidence, record a structured gap or blocker instead of omitting the field. Do not infer business rules, permissions, form validation, error copy, dynamic states, data sources, analytics, security, or compliance behavior from visual appearance alone. -9. Create or update `visual-evidence-packet.md` from `templates/intake-visual-design-evidence-packet-template.md` with readiness front matter and human-readable evidence notes; keep structured records in `visual-requirements.yaml`. Preserve an existing `figma-evidence-packet.md` only as a legacy compatibility alias when already configured by the host project. +9. Create or update `visual-evidence-packet.md` from `templates/intake-visual-design-evidence-packet-template.md` with readiness front matter and human-readable evidence notes; keep structured records in `visual-requirements.yaml`. The evidence packet must summarize validator-backed confidence only and must not create, override, or replace structured asset records. Preserve an existing `figma-evidence-packet.md` only as a legacy compatibility alias when already configured by the host project. 10. Add an intake parity plan that records source-side comparison targets, methods, thresholds, accepted exceptions, and blocking difference categories without defining implementation capture artifacts or downstream delivery approval. 11. Run validation before reporting readiness. @@ -113,17 +141,10 @@ Normative authority: python .specify/extensions/intake/scripts/python/validate_visual_design_intake.py ``` -3. Create or update: - - `visual-spec.yaml`: structured visual requirements/spec facts for pages, regions, roles, states, viewports, locators, expectations, resources, tokens, and blockers. - - `visual-spec-assertions.yaml`: low-cost assertions over visual spec items. - - `visual-spec-evidence-packet.md`: readiness summary, blocker separation, resource traceability, and next corrective action. -4. For Figma sources, every implementation resource, image, exported asset, icon, font, color token, spacing token, radius token, typography token, and component-state ref must trace to Figma metadata, node, variable, style, component, or exported asset refs. -5. When preview artifacts exist, use them only as helper evidence: - - `previews/component-matrix-preview.html` is a human review mirror for component sets, instances, variant props, states, sizes, density, theme, content samples, and viewports. - - `previews/component-coverage.yaml` is the machine-readable component coverage record. - - `previews/viewport-coverage.yaml` is the machine-readable viewport coverage record. -6. Do not use `component-matrix-preview.html` or preview rendering output as the source of truth for assets, tokens, product behavior, or requirements. Use Figma/source refs and visual-design intake evidence as authority. -7. Validate before reporting readiness: +3. Create or update the `visual-spec-package/` artifact family according to `templates/intake-visual-spec-package-contract.md`, `templates/schemas/visual-spec-package.schema.json`, and `templates/schemas/visual-spec-assertions.schema.json`. +4. Keep provider/source traceability, downstream-ownership exclusions, assertion coverage, and optional preview helper refs aligned with the visual spec package contract. Do not restate or invent package fields in this command. +5. Do not use `component-matrix-preview.html` or preview rendering output as the source of truth for assets, tokens, product behavior, or requirements. Use source refs and structured visual-design intake records as authority. +6. Validate before reporting readiness: ```bash python .specify/extensions/intake/scripts/python/validate_visual_spec_package.py @@ -138,16 +159,10 @@ python .specify/extensions/intake/scripts/python/validate_visual_spec_package.py python .specify/extensions/intake/scripts/python/validate_visual_design_intake.py ``` -3. Create or update according to `templates/intake-visual-previews-contract.md`: - - `component-matrix-preview.html`: human-review panel that exhaustively displays component sets, component instances, variant props, states, sizes, density, theme, content samples, and viewports. - - `component-coverage.yaml`: machine-readable coverage records for each required component dimension, covered cell, missing cell, blocker, visual spec ref, preview ref, and Figma source ref. - - `viewport-coverage.yaml`: machine-readable viewport coverage records with source refs, visual spec refs, page refs, screenshots, and visual diff status. - - `known-gaps.md`: accepted exceptions, missing evidence, blocked captures, and owner or next action. - - `screenshots/`: Figma source screenshots, preview screenshots, and diff outputs when tooling is available. -4. Link every preview cell back to its Figma node, component, variable, or style ref, its `visual-spec.yaml` item, its `component-coverage.yaml` record, and screenshot or diff evidence when available. +3. Create or update the `previews/` artifact family according to `templates/intake-visual-previews-contract.md`, `templates/schemas/component-coverage.schema.json`, and `templates/schemas/viewport-coverage.schema.json`. +4. Keep IA matrix structure, preview refs, interaction refs, source refs, screenshot/diff refs, missing coverage records, and known-gap semantics aligned with the preview contract. Do not restate or invent preview fields in this command. 5. Do not use preview HTML as a requirements source, implementation HTML, product semantic source, token source, or replacement for `visual-spec.yaml`. -6. Do not silently complete missing Figma states, variants, viewports, resources, or images. Record them in `component-coverage.yaml`, `viewport-coverage.yaml`, or `known-gaps.md`. -7. Validate before reporting readiness: +6. Validate before reporting readiness: ```bash python .specify/extensions/intake/scripts/python/validate_visual_previews.py @@ -163,7 +178,7 @@ python .specify/extensions/intake/scripts/python/validate_visual_design_intake.p ``` 3. Prefer `--json` when a machine-readable result is needed. Report the validator result exactly: - - `PASS` means the evidence passed JSON Schema structure checks and is ready for downstream projection as traceable visual engineering input. + - `PASS` means the structured UI/visual asset passed JSON Schema structure checks and validator readiness checks, with evidence chain used only to support confidence. - `BLOCKED` means downstream workflows must keep design-derived requirements blocked, unresolved, or marked `[NEEDS CLARIFICATION]` instead of promoting unsupported design facts. ## Readiness Authority @@ -171,9 +186,9 @@ python .specify/extensions/intake/scripts/python/validate_visual_design_intake.p Use this precedence when sources disagree: 1. JSON Schemas are canonical for structural validity in all modes. -2. `validate_visual_design_intake.py` is canonical for visual-design intake readiness status and blocker codes. -3. `validate_visual_spec_package.py` is canonical for visual spec package readiness status and blocker codes. -4. `validate_visual_previews.py` is canonical for preview coverage readiness status and blocker codes. +2. `validate_visual_design_intake.py` is canonical for core structured UI/visual asset readiness status and blocker codes. +3. `validate_visual_spec_package.py` is canonical only for downstream visual spec package readiness status and blocker codes. +4. `validate_visual_previews.py` is canonical only for supporting preview coverage readiness status and blocker codes. 5. `templates/intake-visual-design-contract.md` is canonical for semantic extraction, fidelity, and provider evidence policy. 6. `templates/intake-visual-spec-package-contract.md` and `templates/intake-visual-previews-contract.md` are canonical for their artifact families. @@ -183,7 +198,7 @@ Do not restate, reinterpret, or override blocker codes in this command. Return: -- mode executed: capture, validate, capture_then_validate, build_spec_package, validate_spec_package, build_previews, or validate_previews +- mode executed or sequence executed: capture, validate, capture_then_validate, build_spec_package, validate_spec_package, build_previews, validate_previews, or an ordered combination when source capture precedes build - output or validated directory - source type and source refs captured, or the recorded gap/blocker - required fidelity, or the recorded gap/blocker diff --git a/config-template.yml b/config-template.yml index a2cff98..54bb40d 100644 --- a/config-template.yml +++ b/config-template.yml @@ -19,6 +19,7 @@ artifacts: viewport_coverage: "viewport-coverage.yaml" visual_previews_known_gaps: "known-gaps.md" visual_previews_screenshots_dir: "screenshots" + figma_metadata_capture_script: "scripts/python/capture_figma_metadata_shards.py" visual_previews_validator: "scripts/python/validate_visual_previews.py" visual_spec_package: "visual-spec.yaml" visual_spec_assertions: "visual-spec-assertions.yaml" @@ -78,6 +79,7 @@ readiness: require_visual_spec_package_provider_evidence: true require_visual_spec_package_ci_assertions: true require_visual_spec_package_provider_product_gap_separation: true + require_visual_preview_ia_matrix_interactions: true require_visual_source_refs: true require_fidelity_rules_applied: true require_visual_parity_plan: true @@ -97,6 +99,8 @@ capture: visual_requirements_file: "visual-requirements.yaml" test_case_intake_file: "test-case-intake.yaml" shard_prefix: "figma-metadata.part-" + metadata_capture_mode: "sharded_file_staging" preserve_raw_metadata: true + reject_truncated_metadata_shards: true allow_screenshot_only_intake: true default_screenshot_level: "L0" diff --git a/extension.yml b/extension.yml index 645d0d4..68c3dcd 100644 --- a/extension.yml +++ b/extension.yml @@ -3,7 +3,7 @@ extension: id: intake name: "Intake" - version: "0.1.4" + version: "0.1.5" description: "Normalize PRD, visual design/spec packages, preview evidence, and test-case evidence into SDD-ready intake artifacts" author: "bigsmartben" repository: "https://github.com/bigsmartben/spec-kit-intake" @@ -70,6 +70,7 @@ defaults: viewport_coverage: "viewport-coverage.yaml" visual_previews_known_gaps: "known-gaps.md" visual_previews_screenshots_dir: "screenshots" + figma_metadata_capture_script: "scripts/python/capture_figma_metadata_shards.py" visual_previews_validator: "scripts/python/validate_visual_previews.py" visual_spec_package: "visual-spec.yaml" visual_spec_assertions: "visual-spec-assertions.yaml" @@ -128,6 +129,7 @@ defaults: require_visual_spec_package_provider_evidence: true require_visual_spec_package_ci_assertions: true require_visual_spec_package_provider_product_gap_separation: true + require_visual_preview_ia_matrix_interactions: true require_visual_source_refs: true require_fidelity_rules_applied: true require_visual_parity_plan: true @@ -146,6 +148,8 @@ defaults: visual_requirements_file: "visual-requirements.yaml" test_case_intake_file: "test-case-intake.yaml" shard_prefix: "figma-metadata.part-" + metadata_capture_mode: "sharded_file_staging" preserve_raw_metadata: true + reject_truncated_metadata_shards: true allow_screenshot_only_intake: true default_screenshot_level: "L0" diff --git a/scripts/python/capture_figma_metadata_shards.py b/scripts/python/capture_figma_metadata_shards.py new file mode 100644 index 0000000..0a36589 --- /dev/null +++ b/scripts/python/capture_figma_metadata_shards.py @@ -0,0 +1,394 @@ +"""Stage sharded Figma metadata captures into visual-design intake artifacts.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import shutil +import sys +from datetime import datetime, timezone +from pathlib import Path +from typing import Any +from xml.etree import ElementTree + + +SHARD_PREFIX = "figma-metadata.part-" +INDEX_NAME = "figma-metadata.index.yaml" +INVENTORY_NAME = "figma-node-inventory.yaml" + + +def sha256_bytes(data: bytes) -> str: + return hashlib.sha256(data).hexdigest() + + +def utc_now() -> str: + return datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z") + + +def yaml_scalar(value: Any) -> str: + if isinstance(value, bool): + return "true" if value else "false" + if isinstance(value, int): + return str(value) + if value is None: + return "null" + return json.dumps(str(value), ensure_ascii=False) + + +def write_mapping(path: Path, rows: list[tuple[str, Any]]) -> None: + lines: list[str] = [] + for key, value in rows: + if isinstance(value, list): + lines.append(f"{key}:") + if not value: + lines[-1] = f"{key}: []" + else: + for item in value: + if isinstance(item, dict): + lines.append(" - " + next(iter(item.keys())) + f": {yaml_scalar(next(iter(item.values())))}") + for sub_key, sub_value in list(item.items())[1:]: + write_nested(lines, sub_key, sub_value, indent=" ") + else: + lines.append(f" - {yaml_scalar(item)}") + else: + lines.append(f"{key}: {yaml_scalar(value)}") + path.write_text("\n".join(lines) + "\n", encoding="utf-8") + + +def write_nested(lines: list[str], key: str, value: Any, *, indent: str) -> None: + if isinstance(value, list): + if not value: + lines.append(f"{indent}{key}: []") + return + lines.append(f"{indent}{key}:") + for item in value: + if isinstance(item, dict): + lines.append(f"{indent} - " + next(iter(item.keys())) + f": {yaml_scalar(next(iter(item.values())))}") + for sub_key, sub_value in list(item.items())[1:]: + write_nested(lines, sub_key, sub_value, indent=indent + " ") + else: + lines.append(f"{indent} - {yaml_scalar(item)}") + return + lines.append(f"{indent}{key}: {yaml_scalar(value)}") + + +def is_metadata_source(path: Path) -> bool: + return path.suffix.lower() in {".xml", ".json", ".txt"} + + +def is_canonical_artifact(path: Path, intake_dir: Path) -> bool: + try: + path.relative_to(intake_dir.resolve()) + except ValueError: + return False + return path.name in {INDEX_NAME, INVENTORY_NAME} or ( + path.name.startswith(SHARD_PREFIX) and path.suffix.lower() == ".xml" + ) + + +def expand_sources(sources: list[Path], intake_dir: Path) -> list[Path]: + expanded: list[Path] = [] + for source in sources: + if source.is_dir(): + for path in sorted(source.rglob("*")): + if path.is_file() and is_metadata_source(path) and not is_canonical_artifact(path.resolve(), intake_dir): + expanded.append(path) + elif source.is_file(): + if not is_metadata_source(source): + raise SystemExit(f"metadata source must be .xml, .json, or .txt: {source}") + if is_canonical_artifact(source.resolve(), intake_dir): + raise SystemExit(f"metadata source must not be an existing canonical intake artifact: {source}") + expanded.append(source) + else: + raise SystemExit(f"metadata source does not exist: {source}") + if not expanded: + raise SystemExit("at least one metadata source file is required") + return expanded + + +def looks_truncated(text: str) -> bool: + lower = text.lower() + if "truncated" not in lower: + return False + allowed = [ + 'truncated="false"', + "truncated='false'", + '"truncated": false', + "'truncated': false", + "truncated: false", + ] + return not any(marker in lower for marker in allowed) + + +def extract_xml_node_ids(text: str) -> tuple[list[str], list[str], str | None]: + try: + root = ElementTree.fromstring(text) + except ElementTree.ParseError as exc: + return [], [], f"XML_PARSE_ERROR: {exc}" + + all_ids: list[str] = [] + for element in root.iter(): + node_id = element.attrib.get("id") + if node_id: + all_ids.append(str(node_id)) + + root_ids: list[str] = [] + root_id = root.attrib.get("id") + if root_id: + root_ids.append(str(root_id)) + else: + for child in list(root): + child_id = child.attrib.get("id") + if child_id: + root_ids.append(str(child_id)) + if not root_ids and all_ids: + root_ids.append(all_ids[0]) + return root_ids, all_ids, None + + +def extract_json_node_ids(text: str) -> tuple[list[str], list[str], str | None]: + try: + data = json.loads(text) + except json.JSONDecodeError as exc: + return [], [], f"JSON_PARSE_ERROR: {exc}" + + all_ids: list[str] = [] + + def walk(value: Any) -> None: + if isinstance(value, dict): + for key in ("id", "nodeId", "node_id"): + node_id = value.get(key) + if isinstance(node_id, (str, int)): + all_ids.append(str(node_id)) + break + for child in value.values(): + walk(child) + elif isinstance(value, list): + for item in value: + walk(item) + + walk(data) + root_ids: list[str] = [] + if isinstance(data, dict): + for key in ("id", "nodeId", "node_id"): + node_id = data.get(key) + if isinstance(node_id, (str, int)): + root_ids.append(str(node_id)) + break + if not root_ids: + document = data.get("document") + if isinstance(document, dict): + node_id = document.get("id") + if isinstance(node_id, (str, int)): + root_ids.append(str(node_id)) + if not root_ids and all_ids: + root_ids.append(all_ids[0]) + return root_ids, all_ids, None + + +def extract_node_ids(raw: bytes, source: Path) -> tuple[list[str], list[str], str | None]: + text = raw.decode("utf-8-sig", errors="replace") + suffix = source.suffix.lower() + if suffix == ".json": + return extract_json_node_ids(text) + if suffix in {".xml", ".txt", ""}: + root_ids, all_ids, error = extract_xml_node_ids(text) + if error and suffix == ".txt": + return extract_json_node_ids(text) + return root_ids, all_ids, error + return extract_xml_node_ids(text) + + +def sorted_unique(values: list[str]) -> list[str]: + return sorted(set(values)) + + +def build_artifacts(args: argparse.Namespace) -> dict[str, Any]: + intake_dir = args.intake_dir + intake_dir.mkdir(parents=True, exist_ok=True) + sources = expand_sources(args.metadata_source, intake_dir) + if args.node_id and len(args.node_id) not in {1, len(sources)}: + raise SystemExit("--node-id must be supplied once for all shards or once per metadata source") + + if args.overwrite: + for old in intake_dir.glob(f"{SHARD_PREFIX}*.xml"): + old.unlink() + for old_name in [INDEX_NAME, INVENTORY_NAME]: + old = intake_dir / old_name + if old.exists(): + old.unlink() + + captured_at = args.captured_at or utc_now() + design_version = args.design_version or captured_at + shard_rows: list[dict[str, Any]] = [] + all_node_ids: list[str] = [] + all_root_ids: list[str] = [] + gaps: list[dict[str, Any]] = [] + any_truncated = False + + for index, source in enumerate(sources, start=1): + raw = source.read_bytes() + text = raw.decode("utf-8-sig", errors="replace") + root_ids, node_ids, parse_error = extract_node_ids(raw, source) + if args.node_id: + root_ids = [args.node_id[index - 1] if len(args.node_id) == len(sources) else args.node_id[0]] + + truncated = looks_truncated(text) + any_truncated = any_truncated or truncated + if truncated: + gaps.append( + { + "code": "FIGMA_RAW_METADATA_TRUNCATED", + "source": str(source), + "reason": "metadata source contains a truncation marker", + } + ) + if parse_error: + gaps.append( + { + "code": "FIGMA_METADATA_PARITY_FAILED", + "source": str(source), + "reason": parse_error, + } + ) + if not node_ids: + gaps.append( + { + "code": "FIGMA_METADATA_PARITY_FAILED", + "source": str(source), + "reason": "no node ids were found in the metadata source", + } + ) + + shard_name = f"{SHARD_PREFIX}{index:03d}.xml" + shard_path = intake_dir / shard_name + if shard_path.exists() and not args.overwrite: + raise SystemExit(f"refusing to overwrite existing shard without --overwrite: {shard_path}") + shutil.copyfile(source, shard_path) + all_node_ids.extend(node_ids) + all_root_ids.extend(root_ids) + shard_rows.append( + { + "path": shard_name, + "byte_size": len(raw), + "sha256": sha256_bytes(raw), + "root_node_ids": root_ids, + "node_count": len(sorted_unique(node_ids)), + "truncated": truncated, + "source_path": str(source), + } + ) + + expected_roots = args.node_id or sorted_unique(all_root_ids) + captured_roots = sorted_unique(all_root_ids) + missing_roots = sorted(set(expected_roots) - set(captured_roots)) + duplicate_node_count = len(all_node_ids) - len(set(all_node_ids)) + raw_node_count = len(set(all_node_ids)) + metadata_complete = not any_truncated and not missing_roots and raw_node_count > 0 and not any( + gap["code"] == "FIGMA_METADATA_PARITY_FAILED" for gap in gaps + ) + selected_subtree_complete = metadata_complete + parity_passed = metadata_complete and duplicate_node_count == 0 + + index_rows: list[tuple[str, Any]] = [ + ("file_url", args.file_url), + ("file_key", args.file_key), + ("page_id", args.page_id), + ("selected_node_ids", expected_roots), + ("captured_at", captured_at), + ("mcp_tool", "get_metadata"), + ("design_version_or_timestamp", design_version), + ("selected_subtree_complete", selected_subtree_complete), + ("raw_metadata_complete", metadata_complete), + ("expected_root_node_ids", expected_roots), + ("captured_root_node_ids", captured_roots), + ("missing_root_node_ids", missing_roots), + ("gap_count", len(gaps)), + ("gaps", gaps), + ("shards", shard_rows), + ] + write_mapping(intake_dir / INDEX_NAME, index_rows) + + inventory_rows: list[tuple[str, Any]] = [ + ("raw_node_count", raw_node_count), + ("inventory_node_count", raw_node_count), + ("excluded_node_count", 0), + ("missing_node_count", len(missing_roots)), + ("duplicate_node_count", duplicate_node_count), + ("truncated_raw_evidence", any_truncated), + ("node_inventory_coverage", "100%" if parity_passed else "incomplete"), + ("parity_passed", parity_passed), + ("captured_root_node_ids", captured_roots), + ("missing_root_node_ids", missing_roots), + ] + write_mapping(intake_dir / INVENTORY_NAME, inventory_rows) + + return { + "status": "PASS" if metadata_complete and parity_passed else "BLOCKED", + "intake_dir": str(intake_dir), + "metadata_shards": [row["path"] for row in shard_rows], + "index": INDEX_NAME, + "inventory": INVENTORY_NAME, + "raw_node_count": raw_node_count, + "captured_root_node_ids": captured_roots, + "missing_root_node_ids": missing_roots, + "gaps": gaps, + "blockers": sorted( + { + gap["code"] + for gap in gaps + } + | ({"FIGMA_METADATA_PARITY_FAILED"} if not parity_passed else set()) + ), + } + + +def parse_args(argv: list[str]) -> argparse.Namespace: + parser = argparse.ArgumentParser( + description=( + "Copy already-sharded Figma get_metadata outputs into an intake directory, " + "then write figma-metadata.index.yaml and figma-node-inventory.yaml." + ) + ) + parser.add_argument("intake_dir", type=Path, help="visual-design intake directory") + parser.add_argument( + "--metadata-source", + type=Path, + action="append", + required=True, + help="Raw get_metadata response file or directory. Repeat for each shard.", + ) + parser.add_argument("--file-url", required=True, help="Stable Figma file URL") + parser.add_argument("--file-key", required=True, help="Figma file key") + parser.add_argument("--page-id", required=True, help="Figma page or source page id") + parser.add_argument( + "--node-id", + action="append", + help="Expected selected root node id. Repeat once per metadata source for exact root mapping.", + ) + parser.add_argument("--captured-at", help="Capture timestamp. Defaults to current UTC time.") + parser.add_argument("--design-version", help="Design version or timestamp. Defaults to captured-at.") + parser.add_argument("--overwrite", action="store_true", help="Replace existing canonical metadata artifacts.") + parser.add_argument("--json", action="store_true", help="Emit machine-readable JSON result.") + return parser.parse_args(argv) + + +def main(argv: list[str] | None = None) -> int: + args = parse_args(argv or sys.argv[1:]) + result = build_artifacts(args) + if args.json: + print(json.dumps(result, indent=2, ensure_ascii=False, sort_keys=True)) + else: + print(f"Figma metadata shard capture: {result['status']}") + print(f"Output directory: {result['intake_dir']}") + print(f"Shards: {', '.join(result['metadata_shards'])}") + if result["blockers"]: + print("Blockers:") + for blocker in result["blockers"]: + print(f"- {blocker}") + return 0 if result["status"] == "PASS" else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/python/intake_validator_common.py b/scripts/python/intake_validator_common.py index 65460aa..3706e2d 100644 --- a/scripts/python/intake_validator_common.py +++ b/scripts/python/intake_validator_common.py @@ -42,6 +42,32 @@ def is_remote_ref(value: str) -> bool: return value.startswith(("http://", "https://", "figma://")) +def is_supporting_visual_artifact_ref(value: Any) -> bool: + """Return true when a ref points at helper evidence, not a source-of-truth record.""" + ref = str(value or "").strip().replace("\\", "/").lower() + if not ref: + return False + supporting_markers = ( + "visual-evidence-packet.md", + "figma-evidence-packet.md", + "visual-spec-evidence-packet.md", + "component-matrix-preview.html", + "/previews/", + "previews/", + "/screenshots/", + "screenshots/", + "visual-diff", + "diff-output", + ) + return any(marker in ref for marker in supporting_markers) + + +def supporting_visual_artifact_refs(value: Any) -> list[str]: + if not isinstance(value, list): + return [] + return [str(ref).strip() for ref in value if is_supporting_visual_artifact_ref(ref)] + + def has_needs_clarification(value: Any) -> bool: if isinstance(value, str): return "[NEEDS CLARIFICATION]" in value diff --git a/scripts/python/validate_visual_design_intake.py b/scripts/python/validate_visual_design_intake.py index ab0515a..3959927 100644 --- a/scripts/python/validate_visual_design_intake.py +++ b/scripts/python/validate_visual_design_intake.py @@ -17,7 +17,11 @@ except ImportError: # pragma: no cover - exercised in user environments yaml = None -from intake_validator_common import parse_evidence_packet_status, validate_json_schema +from intake_validator_common import ( + parse_evidence_packet_status, + supporting_visual_artifact_refs, + validate_json_schema, +) ALLOWED_SOURCE_TYPES = {"image", "pdf", "markdown", "figma"} @@ -393,6 +397,7 @@ def validate_visual_requirements( has_blocker_lint = isinstance(blocker_lint_errors, list) and len(blocker_lint_errors) > 0 has_inference_contract_error = False evidence_type_counts: dict[str, int] = {} + supporting_source_refs: list[dict[str, Any]] = [] for index, item in enumerate(requirements): if not isinstance(item, dict): @@ -408,6 +413,12 @@ def validate_visual_requirements( source_refs = item.get("source_refs") if not isinstance(source_refs, list) or not source_refs or any(not str(ref).strip() for ref in source_refs): has_untraceable = True + else: + helper_refs = supporting_visual_artifact_refs(source_refs) + if helper_refs: + requirement_errors.append({"index": index, "supporting_artifact_source_refs": helper_refs}) + supporting_source_refs.append({"index": index, "refs": helper_refs}) + has_untraceable = True evidence_type = str(item.get("evidence_type") or "").strip().lower() if evidence_type: @@ -437,6 +448,7 @@ def validate_visual_requirements( details["visual_requirements"]["requirement_errors"] = requirement_errors details["visual_requirements"]["evidence_type_counts"] = evidence_type_counts + details["visual_requirements"]["supporting_artifact_source_refs"] = supporting_source_refs details["visual_requirements"]["count_matches_requirements"] = count_matches if has_missing_required: diff --git a/scripts/python/validate_visual_previews.py b/scripts/python/validate_visual_previews.py index 8cd4f26..eb28bd3 100644 --- a/scripts/python/validate_visual_previews.py +++ b/scripts/python/validate_visual_previews.py @@ -1,11 +1,12 @@ #!/usr/bin/env python3 -"""Validate Spec Kit Figma-derived component matrix preview bundles.""" +"""Validate Spec Kit provider-neutral IA matrix preview bundles.""" from __future__ import annotations import argparse import json import re +import subprocess import sys from pathlib import Path from typing import Any @@ -15,7 +16,7 @@ except ImportError: # pragma: no cover - exercised in user environments yaml = None -from intake_validator_common import validate_json_schema +from intake_validator_common import supporting_visual_artifact_refs, validate_json_schema BLOCKERS = { @@ -23,6 +24,7 @@ "REQUIRED_ARTIFACT_MISSING": "VISUAL_PREVIEW_REQUIRED_ARTIFACT_MISSING", "FIGMA_NODE_COVERAGE_INCOMPLETE": "VISUAL_PREVIEW_FIGMA_NODE_COVERAGE_INCOMPLETE", "COMPONENT_STATE_COVERAGE_INCOMPLETE": "VISUAL_PREVIEW_COMPONENT_STATE_COVERAGE_INCOMPLETE", + "IA_MATRIX_INCOMPLETE": "VISUAL_PREVIEW_IA_MATRIX_INCOMPLETE", "PAGE_COVERAGE_INCOMPLETE": "VISUAL_PREVIEW_PAGE_COVERAGE_INCOMPLETE", "ASSET_TRACEABILITY_INCOMPLETE": "VISUAL_PREVIEW_ASSET_TRACEABILITY_INCOMPLETE", "VIEWPORT_CAPTURE_INCOMPLETE": "VISUAL_PREVIEW_VIEWPORT_CAPTURE_INCOMPLETE", @@ -31,6 +33,42 @@ "SCHEMA_INVALID": "VISUAL_PREVIEW_SCHEMA_INVALID", } +REQUIRED_PREVIEW_SECTIONS = { + "ia-matrix-overview", + "page-state-enumeration", + "page-ia-matrix", + "component-state-enumeration", + "component-ia-matrix", + "coverage-evidence-conclusion", +} + +REQUIRED_PAGE_IA_FIELDS = { + "page_region", + "visual_state", + "user_event", + "precondition", + "system_response", + "state_change", + "transition_or_overlay", + "exception_branch", + "evidence_ref", + "coverage_status", +} + +REQUIRED_COMPONENT_IA_FIELDS = { + "component_state", + "visible_elements", + "action_target", + "user_event", + "precondition", + "immediate_feedback", + "state_change", + "affected_surface", + "disabled_or_error_rule", + "evidence_ref", + "coverage_status", +} + def main() -> int: parser = argparse.ArgumentParser(description=__doc__) @@ -82,8 +120,9 @@ def main() -> int: if required_files["component_matrix_preview"].exists(): html_text = required_files["component_matrix_preview"].read_text(encoding="utf-8", errors="replace") + validate_preview_html_structure(html_text, details, blocker_codes) validate_component_coverage(component_coverage, html_text, details, blocker_codes) - validate_viewport_coverage(html_dir, viewport_coverage, details, blocker_codes) + validate_viewport_coverage(html_dir, viewport_coverage, html_text, details, blocker_codes) validate_known_gaps(required_files["known_gaps"], details, blocker_codes) return emit(args.json, details, sorted(set(blocker_codes)), warnings) @@ -95,26 +134,23 @@ def validate_source_intake( blocker_codes: list[str], ) -> None: upstream = html_dir.parent - packet = upstream / "visual-evidence-packet.md" - if not packet.exists(): - details["source_intake"] = {"missing": True} - blocker_codes.append(BLOCKERS["SOURCE_INTAKE_BLOCKED"]) - return - - text = packet.read_text(encoding="utf-8", errors="replace").lstrip("\ufeff") - match = re.match(r"\A---\s*\r?\n(.*?)\r?\n---", text, re.DOTALL) - metadata: dict[str, Any] = {} - if match and yaml is not None: - loaded = yaml.safe_load(match.group(1)) or {} - metadata = loaded if isinstance(loaded, dict) else {} - ready_gate = str(metadata.get("ready_gate") or "").strip().upper() - blockers = metadata.get("blockers") + validator = Path(__file__).resolve().with_name("validate_visual_design_intake.py") + result = subprocess.run( + [sys.executable, str(validator), str(upstream), "--json"], + text=True, + capture_output=True, + ) + try: + payload = json.loads(result.stdout or "{}") + except json.JSONDecodeError: + payload = {} details["source_intake"] = { - "path": str(packet), - "ready_gate": ready_gate, - "blockers": blockers, + "path": str(upstream), + "validator": str(validator), + "status": payload.get("status"), + "blockers": payload.get("blockers"), } - if ready_gate != "PASS" or (isinstance(blockers, list) and blockers): + if result.returncode != 0 or payload.get("status") != "PASS": blocker_codes.append(BLOCKERS["SOURCE_INTAKE_BLOCKED"]) @@ -158,9 +194,56 @@ def preview_ref_in_html(preview_ref: str, html_text: str) -> bool: or f"id='{fragment}'" in html_text or f'data-preview-id="{fragment}"' in html_text or f"data-preview-id='{fragment}'" in html_text + or f'data-interaction-id="{fragment}"' in html_text + or f"data-interaction-id='{fragment}'" in html_text ) +def html_attr_value_exists(html_text: str, attr: str, value: str) -> bool: + return ( + f'{attr}="{value}"' in html_text + or f"{attr}='{value}'" in html_text + ) + + +def validate_preview_html_structure( + html_text: str, + details: dict[str, Any], + blocker_codes: list[str], +) -> None: + if not html_text: + details["preview_html_structure"] = { + "missing_sections": sorted(REQUIRED_PREVIEW_SECTIONS), + "missing_page_ia_fields": sorted(REQUIRED_PAGE_IA_FIELDS), + "missing_component_ia_fields": sorted(REQUIRED_COMPONENT_IA_FIELDS), + } + blocker_codes.append(BLOCKERS["IA_MATRIX_INCOMPLETE"]) + return + + missing_sections = [ + section + for section in sorted(REQUIRED_PREVIEW_SECTIONS) + if not html_attr_value_exists(html_text, "data-preview-section", section) + ] + missing_page_ia_fields = [ + field + for field in sorted(REQUIRED_PAGE_IA_FIELDS) + if not html_attr_value_exists(html_text, "data-page-ia-field", field) + ] + missing_component_ia_fields = [ + field + for field in sorted(REQUIRED_COMPONENT_IA_FIELDS) + if not html_attr_value_exists(html_text, "data-component-ia-field", field) + ] + details["preview_html_structure"] = { + "missing_sections": missing_sections, + "missing_page_ia_fields": missing_page_ia_fields, + "missing_component_ia_fields": missing_component_ia_fields, + } + if missing_sections or missing_page_ia_fields or missing_component_ia_fields: + blocker_codes.append(BLOCKERS["IA_MATRIX_INCOMPLETE"]) + + def validate_component_coverage( component_coverage: dict[str, Any], html_text: str, @@ -169,9 +252,11 @@ def validate_component_coverage( ) -> None: components = component_coverage.get("components", []) missing_preview_refs: list[str] = [] + missing_interaction_refs: list[str] = [] missing_visual_spec_refs: list[str] = [] missing_records: list[str] = [] asset_or_resource_gaps: list[str] = [] + supporting_source_refs: list[dict[str, Any]] = [] covered_count = 0 missing_count = 0 @@ -179,6 +264,9 @@ def validate_component_coverage( if not isinstance(component, dict): continue component_id = str(component.get("id") or "") + component_helper_ref = supporting_visual_artifact_refs([component.get("source_ref")]) + if component_helper_ref: + supporting_source_refs.append({"id": component_id, "refs": component_helper_ref}) covered = component.get("covered", []) missing = component.get("missing", []) if isinstance(covered, list): @@ -187,11 +275,17 @@ def validate_component_coverage( if not isinstance(record, dict): continue preview_ref = str(record.get("preview_ref") or "") + interaction_ref = str(record.get("interaction_ref") or "") visual_spec_ref = str(record.get("visual_spec_ref") or "") + record_helper_refs = supporting_visual_artifact_refs([record.get("source_ref")]) + if record_helper_refs: + supporting_source_refs.append({"id": component_id, "refs": record_helper_refs}) if not visual_spec_ref: missing_visual_spec_refs.append(component_id) if not preview_ref or not preview_ref_in_html(preview_ref, html_text): missing_preview_refs.append(preview_ref or component_id) + if not interaction_ref or not preview_ref_in_html(interaction_ref, html_text): + missing_interaction_refs.append(interaction_ref or component_id) if isinstance(missing, list): missing_count += len(missing) for record in missing: @@ -208,17 +302,21 @@ def validate_component_coverage( "covered_count": covered_count, "missing_count": missing_count, "missing_preview_refs": missing_preview_refs, + "missing_interaction_refs": missing_interaction_refs, "missing_visual_spec_refs": missing_visual_spec_refs, "missing_records": missing_records, "asset_or_resource_gaps": asset_or_resource_gaps, + "supporting_artifact_source_refs": supporting_source_refs, "ready_gate": component_coverage.get("ready_gate"), "blockers": blockers, } if not components or missing_preview_refs: blocker_codes.append(BLOCKERS["FIGMA_NODE_COVERAGE_INCOMPLETE"]) + if missing_interaction_refs: + blocker_codes.append(BLOCKERS["IA_MATRIX_INCOMPLETE"]) if missing_records or missing_visual_spec_refs: blocker_codes.append(BLOCKERS["COMPONENT_STATE_COVERAGE_INCOMPLETE"]) - if asset_or_resource_gaps: + if asset_or_resource_gaps or supporting_source_refs: blocker_codes.append(BLOCKERS["ASSET_TRACEABILITY_INCOMPLETE"]) if component_coverage.get("ready_gate") != "PASS" or (isinstance(blockers, list) and blockers): blocker_codes.append(BLOCKERS["KNOWN_GAP_UNRESOLVED"]) @@ -227,13 +325,16 @@ def validate_component_coverage( def validate_viewport_coverage( html_dir: Path, viewport_coverage: dict[str, Any], + html_text: str, details: dict[str, Any], blocker_codes: list[str], ) -> None: viewports = viewport_coverage.get("viewports", []) missing_screenshots: list[str] = [] + missing_page_refs: list[str] = [] uncovered_viewports: list[str] = [] visual_diff_blocked: list[str] = [] + supporting_source_refs: list[dict[str, Any]] = [] page_refs = 0 for viewport in viewports if isinstance(viewports, list) else []: @@ -242,6 +343,9 @@ def validate_viewport_coverage( viewport_id = str(viewport.get("id") or "") if viewport.get("covered") is not True: uncovered_viewports.append(viewport_id) + helper_refs = supporting_visual_artifact_refs(viewport.get("source_refs")) + if helper_refs: + supporting_source_refs.append({"id": viewport_id, "refs": helper_refs}) refs = viewport.get("screenshot_refs", []) if not refs: missing_screenshots.append(viewport_id) @@ -250,6 +354,10 @@ def validate_viewport_coverage( if ref_path and not (html_dir / ref_path).exists(): missing_screenshots.append(ref_path) page_refs += len(viewport.get("page_refs", []) or []) + for ref in viewport.get("page_refs", []) or []: + page_ref = str(ref) + if page_ref and not preview_ref_in_html(page_ref, html_text): + missing_page_refs.append(page_ref) if viewport.get("visual_diff_status") == "blocked": visual_diff_blocked.append(viewport_id) @@ -260,15 +368,19 @@ def validate_viewport_coverage( "uncovered_viewports": uncovered_viewports, "visual_diff_blocked": visual_diff_blocked, "page_ref_count": page_refs, + "missing_page_refs": missing_page_refs, + "supporting_artifact_source_refs": supporting_source_refs, "ready_gate": viewport_coverage.get("ready_gate"), "blockers": blockers, } if not viewports or uncovered_viewports or missing_screenshots: blocker_codes.append(BLOCKERS["VIEWPORT_CAPTURE_INCOMPLETE"]) - if page_refs == 0: + if page_refs == 0 or missing_page_refs: blocker_codes.append(BLOCKERS["PAGE_COVERAGE_INCOMPLETE"]) if visual_diff_blocked: blocker_codes.append(BLOCKERS["VISUAL_DIFF_BLOCKED"]) + if supporting_source_refs: + blocker_codes.append(BLOCKERS["ASSET_TRACEABILITY_INCOMPLETE"]) if viewport_coverage.get("ready_gate") != "PASS" or (isinstance(blockers, list) and blockers): blocker_codes.append(BLOCKERS["KNOWN_GAP_UNRESOLVED"]) diff --git a/scripts/python/validate_visual_spec_package.py b/scripts/python/validate_visual_spec_package.py index 84f70d6..ee53088 100644 --- a/scripts/python/validate_visual_spec_package.py +++ b/scripts/python/validate_visual_spec_package.py @@ -4,7 +4,9 @@ from __future__ import annotations import argparse +import json import re +import subprocess import sys from pathlib import Path from typing import Any @@ -15,6 +17,7 @@ load_yaml, non_empty, parse_evidence_packet_status, + supporting_visual_artifact_refs, validate_json_schema, ) @@ -115,26 +118,24 @@ def validate_source_intake( blocker_codes: list[str], ) -> None: upstream = package_dir.parent - packet = upstream / "visual-evidence-packet.md" - if not packet.exists(): - details["source_intake"] = {"missing": True} - blocker_codes.append(BLOCKERS["SOURCE_INTAKE_BLOCKED"]) - return - - packet_status = parse_evidence_packet_status(packet.read_text(encoding="utf-8", errors="replace")) - metadata = packet_status["metadata"] + validator = Path(__file__).resolve().with_name("validate_visual_design_intake.py") + result = subprocess.run( + [sys.executable, str(validator), str(upstream), "--json"], + text=True, + capture_output=True, + ) + try: + payload = json.loads(result.stdout or "{}") + except json.JSONDecodeError: + payload = {} details["source_intake"] = { - "path": str(packet), - "ready_gate": packet_status["ready_gate"], - "blockers": metadata.get("blockers"), - "errors": packet_status["errors"], + "path": str(upstream), + "validator": str(validator), + "status": payload.get("status"), + "blockers": payload.get("blockers"), + "errors": payload.get("details", {}).get("evidence_packet_metadata", {}).get("errors"), } - source_blockers = metadata.get("blockers") - if ( - packet_status["ready_gate"] != "PASS" - or packet_status["errors"] - or (isinstance(source_blockers, list) and source_blockers) - ): + if result.returncode != 0 or payload.get("status") != "PASS": blocker_codes.append(BLOCKERS["SOURCE_INTAKE_BLOCKED"]) @@ -170,6 +171,7 @@ def validate_visual_spec_package( provider_evidence_gaps: list[str] = [] product_ambiguity_gaps: list[str] = [] blocker_lint_items: list[str] = [] + supporting_source_refs: list[dict[str, Any]] = [] has_ready_item = False details["visual_spec_package"] = { @@ -237,8 +239,11 @@ def validate_visual_spec_package( if missing: item_errors.append({"id": item_id, "missing_fields": missing}) - if not valid_source_refs(item.get("source_refs")): + helper_refs = supporting_visual_artifact_refs(item.get("source_refs")) + if not valid_source_refs(item.get("source_refs")) or helper_refs: provider_evidence_gaps.append(item_id) + if helper_refs: + supporting_source_refs.append({"id": item_id, "refs": helper_refs}) evidence_type = str(item.get("evidence_type") or "") if evidence_type in {"missing", "unsupported"} or non_empty(item.get("missing_evidence")): @@ -268,6 +273,7 @@ def validate_visual_spec_package( details["visual_spec_package"]["invalid_locators"] = sorted(set(invalid_locators)) details["visual_spec_package"]["ownership_leaks"] = sorted(set(ownership_leaks)) details["visual_spec_package"]["blocker_lint_items"] = sorted(set(blocker_lint_items)) + details["visual_spec_package"]["supporting_artifact_source_refs"] = supporting_source_refs if item_errors or blocker_lint_items or not has_ready_item: blocker_codes.append(BLOCKERS["INTAKE_INCOMPLETE"]) @@ -314,6 +320,7 @@ def validate_visual_spec_assertions( product_ambiguity_gaps: list[str] = [] provider_evidence_gaps: list[str] = [] blocker_lint_assertions: list[str] = [] + supporting_evidence_refs: list[dict[str, Any]] = [] ready_ci_count = 0 details["visual_spec_assertions"] = { @@ -366,8 +373,11 @@ def validate_visual_spec_assertions( elif status == "ready": non_ci_assertions.append(assertion_id) - if not valid_source_refs(assertion.get("evidence_refs")): + helper_refs = supporting_visual_artifact_refs(assertion.get("evidence_refs")) + if not valid_source_refs(assertion.get("evidence_refs")) or helper_refs: provider_evidence_gaps.append(assertion_id) + if helper_refs: + supporting_evidence_refs.append({"id": assertion_id, "refs": helper_refs}) blockers = as_string_set(assertion.get("blockers")) if blockers: @@ -389,6 +399,7 @@ def validate_visual_spec_assertions( details["visual_spec_assertions"]["provider_evidence_gaps"] = sorted(set(provider_evidence_gaps)) details["visual_spec_assertions"]["product_ambiguity_gaps"] = sorted(set(product_ambiguity_gaps)) details["visual_spec_assertions"]["blocker_lint_assertions"] = sorted(set(blocker_lint_assertions)) + details["visual_spec_assertions"]["supporting_artifact_evidence_refs"] = supporting_evidence_refs if assertion_errors or blocker_lint_assertions or non_ci_assertions or ready_ci_count == 0: blocker_codes.append(BLOCKERS["ASSERTION_COVERAGE_INCOMPLETE"]) diff --git a/templates/intake-visual-design-contract.md b/templates/intake-visual-design-contract.md index 528c201..434b651 100644 --- a/templates/intake-visual-design-contract.md +++ b/templates/intake-visual-design-contract.md @@ -1,8 +1,10 @@ # Visual Design Intake Contract -Required visual design intake artifacts and readiness gates. Runtime agents or external intake tools extract traceable, provider-neutral visual facts before downstream SDD workflows project evidence into requirements or visual verification criteria. +Required visual design intake artifacts and readiness gates. Runtime agents or external intake tools extract high-confidence, traceable, provider-neutral structured UI/visual assets before downstream SDD workflows project evidence into requirements or visual verification criteria. -Intake does not generate requirements. It preserves all reachable design sources, raw provider evidence, stable source refs, checksums or retrieval metadata, and schema-required visual facts that SDD `specify` can consume accurately. +Intake does not generate requirements. The structured UI/visual asset is the source of truth for this intake: it preserves all reachable design sources, raw provider evidence, stable source refs, checksums or retrieval metadata, and schema-required visual facts that SDD `specify` can consume accurately. + +Evidence chain artifacts are supporting artifacts only. They support confidence and human review, but they must not create, override, replace, or act as `source_refs` for structured UI/visual asset records. The machine-readable structures in this contract are enforced by JSON Schemas under `templates/schemas/` before readiness-specific validation runs. Field lists in this document are semantic summaries; the JSON Schemas are canonical for required fields, types, and enums. @@ -52,6 +54,8 @@ The intake must record `fidelity_rules_applied: true` and explain any accepted g `visual-requirements.yaml` must normalize extracted visual facts into engineering input. The evidence packet may summarize the same records for human review, but readiness validation uses the standalone machine-readable file. +`source_refs` must point to original design sources, provider metadata, source files, or structured intake records. They must not point to `visual-evidence-packet.md`, `figma-evidence-packet.md`, `visual-spec-evidence-packet.md`, preview HTML, screenshots, visual diffs, or other helper artifacts as source-of-truth records. + Visual design intake uses a bounded inference model. Intake may preserve direct observations, rule-backed derived claims, and candidate completions, but it must not smooth missing or contradictory evidence into a confirmed requirement. Every non-observed claim must remain auditable through source refs, inference rules, confidence method, score breakdown, downstream use, and blocking conditions. Each requirement must include: @@ -256,3 +260,5 @@ Record a gap instead of passing silently when source evidence is missing, summar - generated_at: Human-readable sections may summarize the same records, but readiness metadata is validated from the front matter when present. + +The evidence packet is not a source-of-truth record. It must summarize structured asset readiness and confidence only; downstream workflows must not derive requirements from the packet when the corresponding structured record is absent or blocked. diff --git a/templates/intake-visual-previews-contract.md b/templates/intake-visual-previews-contract.md index d8641b7..c00927c 100644 --- a/templates/intake-visual-previews-contract.md +++ b/templates/intake-visual-previews-contract.md @@ -1,9 +1,11 @@ # Visual Preview Coverage Contract -Required component matrix preview helper artifacts and readiness gates. Preview coverage artifacts help reviewers inspect whether design-source components, states, variants, resources, content samples, and viewports were enumerated before downstream implementation, but they are not the target visual requirements/spec asset package. +Required IA-matrix preview helper artifacts and readiness gates. Preview coverage artifacts help reviewers inspect whether design-source pages, components, visual states, component states, interaction events, resources, content samples, and viewports were enumerated before downstream implementation, but they are not the target visual requirements/spec asset package. Preview coverage does not generate requirements, implementation HTML, product semantics, downstream-owned selectors, tasks, code component names, or design tokens. It preserves source-backed coverage evidence that points back to design-source refs and forward to `visual-spec-package/` records. +Preview coverage is assembled from the structured UI/visual asset and source-backed records. It must not create, override, replace, or backfill `visual-requirements.yaml`, `visual-spec.yaml`, or `visual-spec-assertions.yaml`; do not infer specifications from `component-matrix-preview.html`. + ## Artifact Family Default directory: @@ -34,11 +36,67 @@ If Figma or design-source evidence is missing, truncated, contradictory, or bloc The file is a human-review panel only. Each preview cell should expose stable anchors such as `id` or `data-preview-id` so `component-coverage.yaml` can reference the cell. +The preview panel must use an IA matrix structure that fuses interaction evidence into page and component review surfaces. Do not create a standalone interaction matrix that is disconnected from the visual states it exercises. + +Required top-level order: + +1. IA matrix overview for fused interactions. +2. For each required page: + - page visual state enumeration + - page IA matrix +3. For each required component: + - component visual state enumeration + - component IA matrix with event interaction information +4. Coverage evidence conclusion + +The HTML must expose these stable section anchors so readiness can be checked: + +- `data-preview-section="ia-matrix-overview"` +- `data-preview-section="page-state-enumeration"` +- `data-preview-section="page-ia-matrix"` +- `data-preview-section="component-state-enumeration"` +- `data-preview-section="component-ia-matrix"` +- `data-preview-section="coverage-evidence-conclusion"` + +Each visual-state enumeration cell must render the state visually or point to source-backed screenshot evidence. A prose-only state row is a missing coverage cell unless the state is explicitly blocked or out of scope. + +Each page IA matrix row must fuse the current interaction matrix information into the page state that owns it. Required IA fields: + +- `page_region` +- `visual_state` +- `user_event` +- `precondition` +- `system_response` +- `state_change` +- `transition_or_overlay` +- `exception_branch` +- `evidence_ref` +- `coverage_status` + +Each component IA matrix row must include event interaction information for the component state. Required IA fields: + +- `component_state` +- `visible_elements` +- `action_target` +- `user_event` +- `precondition` +- `immediate_feedback` +- `state_change` +- `affected_surface` +- `disabled_or_error_rule` +- `evidence_ref` +- `coverage_status` + +Use stable anchors such as `id`, `data-preview-id`, or `data-interaction-id` for every visual-state cell and IA matrix row that a coverage record references. + The preview panel may display: +- pages and page regions - component sets and component instances - variant props - states +- page IA rows and component IA rows +- event, precondition, feedback, transition, exception, and return-path evidence - size, density, and theme dimensions - content samples, including long copy, empty, overflow, and error-like visual states when source-backed - viewport-specific snapshots or links @@ -50,6 +108,8 @@ The preview panel must not define product semantics, downstream component names, The file is the machine-readable component coverage evidence. +`source_ref` fields must point to original design sources, provider metadata, or structured asset records. Preview HTML, screenshots, visual diffs, and evidence packets may be referenced only by preview-specific or screenshot-specific fields. + Top-level fields: - ready_gate: PASS|BLOCKED @@ -69,6 +129,7 @@ Each covered record must include: - visual_spec_ref - preview_ref +- interaction_ref - optional source_ref - optional screenshot_refs - dimension values matching the component's required dimensions when applicable @@ -83,6 +144,8 @@ Each missing record must include: The file is the machine-readable viewport coverage evidence. +`source_refs` must point to original design sources, provider metadata, or structured asset records. `page_refs`, `screenshot_refs`, and diff outputs are supporting preview evidence and must not replace source refs. + Each viewport record must include: - id @@ -107,6 +170,9 @@ Preview coverage is ready only when: - `viewport-coverage.yaml` validates against `viewport-coverage.schema.json` - every covered component record has a `visual_spec_ref` - every covered component record has a `preview_ref` that resolves inside `component-matrix-preview.html` +- every covered component record has an `interaction_ref` that resolves inside `component-matrix-preview.html` +- `component-matrix-preview.html` contains the required IA matrix sections and required IA field markers +- page refs in `viewport-coverage.yaml` resolve inside `component-matrix-preview.html` - no missing record remains for required component states, variants, resources, assets, tokens, screenshots, visual diffs, source evidence, visual spec refs, or preview refs - every viewport record is covered and has existing screenshot refs - at least one viewport has page refs @@ -119,6 +185,7 @@ Preview coverage is ready only when: - `VISUAL_PREVIEW_SCHEMA_INVALID` - `VISUAL_PREVIEW_FIGMA_NODE_COVERAGE_INCOMPLETE` - `VISUAL_PREVIEW_COMPONENT_STATE_COVERAGE_INCOMPLETE` +- `VISUAL_PREVIEW_IA_MATRIX_INCOMPLETE` - `VISUAL_PREVIEW_PAGE_COVERAGE_INCOMPLETE` - `VISUAL_PREVIEW_ASSET_TRACEABILITY_INCOMPLETE` - `VISUAL_PREVIEW_VIEWPORT_CAPTURE_INCOMPLETE` diff --git a/templates/intake-visual-spec-package-contract.md b/templates/intake-visual-spec-package-contract.md index c329723..dc6ed8e 100644 --- a/templates/intake-visual-spec-package-contract.md +++ b/templates/intake-visual-spec-package-contract.md @@ -4,6 +4,8 @@ Required visual requirements/spec structured asset package artifacts and readine Visual Spec Package does not generate requirements, tasks, code component names, implementation-owned selectors, or product semantics. It preserves source-backed DOM, ARIA, token, state, relation, locator-strategy, and assertion facts that can be consumed by low-cost CI checks. +`visual-spec.yaml` and `visual-spec-assertions.yaml` are the structured asset records for this package. `visual-spec-evidence-packet.md`, preview panels, screenshots, and visual diffs are supporting artifacts only; they must not create, override, replace, or serve as source-of-truth records for spec items or assertions. + ## Artifact Family Default directory: @@ -65,6 +67,8 @@ Each item must include: Locator strategies must not be implementation-owned CSS selectors, XPath, generated class names, or downstream test IDs. Candidate test IDs may be recorded only as `test-id-candidate` and must remain intake-owned guidance, not implementation ownership. +`source_refs` must point to original design sources, provider metadata, upstream `visual-requirements.yaml` records, or source-backed structured asset records. Preview refs and evidence packet refs belong in helper fields such as `preview_refs`; they must not be used as `source_refs`. + ## `visual-spec-assertions.yaml` The file must describe low-cost assertions over visual spec items. @@ -91,6 +95,8 @@ Each assertion must include: Ready assertions should use `ci_suitability: ci_low_cost`. Manual review and blocked assertions are allowed as explicit evidence, but they cannot satisfy CI readiness. +`evidence_refs` for ready assertions must point to source-backed design refs or structured visual spec records. They must not point to preview HTML, screenshots, visual diffs, or evidence packets as the only fact source. + ## Readiness Visual Spec Package intake is ready only when: diff --git a/templates/schemas/component-coverage.schema.json b/templates/schemas/component-coverage.schema.json index 236fe44..2f4d130 100644 --- a/templates/schemas/component-coverage.schema.json +++ b/templates/schemas/component-coverage.schema.json @@ -18,7 +18,7 @@ "required": ["id", "source_ref", "name", "required_dimensions", "covered", "missing"], "properties": { "id": { "type": "string", "minLength": 1 }, - "source_ref": { "type": "string", "minLength": 1 }, + "source_ref": { "$ref": "#/$defs/source_ref" }, "name": { "type": "string", "minLength": 1 }, "required_dimensions": { "type": "object", @@ -33,11 +33,12 @@ "type": "array", "items": { "type": "object", - "required": ["visual_spec_ref", "preview_ref"], + "required": ["visual_spec_ref", "preview_ref", "interaction_ref"], "properties": { "visual_spec_ref": { "type": "string", "minLength": 1 }, "preview_ref": { "type": "string", "minLength": 1 }, - "source_ref": { "type": "string", "minLength": 1 }, + "interaction_ref": { "type": "string", "minLength": 1 }, + "source_ref": { "$ref": "#/$defs/source_ref" }, "screenshot_refs": { "type": "array", "items": { "type": "string", "minLength": 1 } @@ -64,7 +65,8 @@ "visual_diff", "source_evidence", "visual_spec_ref", - "preview_ref" + "preview_ref", + "interaction_ref" ] }, "reason": { "type": "string", "minLength": 1 }, @@ -85,6 +87,7 @@ "VISUAL_PREVIEW_REQUIRED_ARTIFACT_MISSING", "VISUAL_PREVIEW_FIGMA_NODE_COVERAGE_INCOMPLETE", "VISUAL_PREVIEW_COMPONENT_STATE_COVERAGE_INCOMPLETE", + "VISUAL_PREVIEW_IA_MATRIX_INCOMPLETE", "VISUAL_PREVIEW_PAGE_COVERAGE_INCOMPLETE", "VISUAL_PREVIEW_ASSET_TRACEABILITY_INCOMPLETE", "VISUAL_PREVIEW_VIEWPORT_CAPTURE_INCOMPLETE", @@ -94,6 +97,13 @@ "VISUAL_SPEC_PROVIDER_EVIDENCE_MISSING", "VISUAL_SPEC_ASSERTION_COVERAGE_INCOMPLETE" ] + }, + "source_ref": { + "type": "string", + "minLength": 1, + "not": { + "pattern": "(visual-evidence-packet\\.md|figma-evidence-packet\\.md|visual-spec-evidence-packet\\.md|component-matrix-preview\\.html|previews/|screenshots/|visual-diff|diff-output)" + } } }, "additionalProperties": true diff --git a/templates/schemas/viewport-coverage.schema.json b/templates/schemas/viewport-coverage.schema.json index b366c74..126bb7e 100644 --- a/templates/schemas/viewport-coverage.schema.json +++ b/templates/schemas/viewport-coverage.schema.json @@ -34,7 +34,7 @@ "source_refs": { "type": "array", "minItems": 1, - "items": { "type": "string", "minLength": 1 } + "items": { "$ref": "#/$defs/source_ref" } }, "visual_spec_refs": { "type": "array", @@ -85,6 +85,13 @@ "VISUAL_SPEC_PROVIDER_EVIDENCE_MISSING", "VISUAL_SPEC_ASSERTION_COVERAGE_INCOMPLETE" ] + }, + "source_ref": { + "type": "string", + "minLength": 1, + "not": { + "pattern": "(visual-evidence-packet\\.md|figma-evidence-packet\\.md|visual-spec-evidence-packet\\.md|component-matrix-preview\\.html|previews/|screenshots/|visual-diff|diff-output)" + } } }, "additionalProperties": true diff --git a/templates/schemas/visual-requirements.schema.json b/templates/schemas/visual-requirements.schema.json index 014e235..5422cba 100644 --- a/templates/schemas/visual-requirements.schema.json +++ b/templates/schemas/visual-requirements.schema.json @@ -91,7 +91,7 @@ "source_refs": { "type": "array", "minItems": 1, - "items": { "type": "string", "minLength": 1 } + "items": { "$ref": "#/$defs/source_ref" } }, "evidence_type": { "enum": ["observed", "inferred", "candidate", "unsupported", "missing", "out_of_scope"] @@ -111,7 +111,7 @@ "weight": { "type": "number" }, "source_refs": { "type": "array", - "items": { "type": "string", "minLength": 1 } + "items": { "$ref": "#/$defs/source_ref" } } }, "additionalProperties": true @@ -242,6 +242,13 @@ "FIGMA_METADATA_PARITY_FAILED", "FIGMA_READY_WITHOUT_COMPLETENESS_PROOF" ] + }, + "source_ref": { + "type": "string", + "minLength": 1, + "not": { + "pattern": "(visual-evidence-packet\\.md|figma-evidence-packet\\.md|visual-spec-evidence-packet\\.md|component-matrix-preview\\.html|previews/|screenshots/|visual-diff|diff-output)" + } } }, "additionalProperties": true diff --git a/templates/schemas/visual-spec-assertions.schema.json b/templates/schemas/visual-spec-assertions.schema.json index 9fc962b..2450933 100644 --- a/templates/schemas/visual-spec-assertions.schema.json +++ b/templates/schemas/visual-spec-assertions.schema.json @@ -63,7 +63,7 @@ "evidence_refs": { "type": "array", "minItems": 1, - "items": { "type": "string", "minLength": 1 } + "items": { "$ref": "#/$defs/source_ref" } }, "ci_suitability": { "enum": ["ci_low_cost", "manual_review", "blocked"] }, "status": { "enum": ["ready", "blocked", "reference_only", "out_of_scope"] }, @@ -90,6 +90,13 @@ "VISUAL_SPEC_DOWNSTREAM_OWNERSHIP_LEAK", "VISUAL_SPEC_READY_WITHOUT_EVIDENCE" ] + }, + "source_ref": { + "type": "string", + "minLength": 1, + "not": { + "pattern": "(visual-evidence-packet\\.md|figma-evidence-packet\\.md|visual-spec-evidence-packet\\.md|component-matrix-preview\\.html|previews/|screenshots/|visual-diff|diff-output)" + } } }, "additionalProperties": true diff --git a/templates/schemas/visual-spec-package.schema.json b/templates/schemas/visual-spec-package.schema.json index 71238be..4e72435 100644 --- a/templates/schemas/visual-spec-package.schema.json +++ b/templates/schemas/visual-spec-package.schema.json @@ -52,7 +52,7 @@ "source_refs": { "type": "array", "minItems": 1, - "items": { "type": "string", "minLength": 1 } + "items": { "$ref": "#/$defs/source_ref" } }, "des_refs": { "type": "array", @@ -142,6 +142,13 @@ "VISUAL_SPEC_DOWNSTREAM_OWNERSHIP_LEAK", "VISUAL_SPEC_READY_WITHOUT_EVIDENCE" ] + }, + "source_ref": { + "type": "string", + "minLength": 1, + "not": { + "pattern": "(visual-evidence-packet\\.md|figma-evidence-packet\\.md|visual-spec-evidence-packet\\.md|component-matrix-preview\\.html|previews/|screenshots/|visual-diff|diff-output)" + } } }, "additionalProperties": true diff --git a/tests/test_extension_contract.py b/tests/test_extension_contract.py index 1749b49..e119ddd 100644 --- a/tests/test_extension_contract.py +++ b/tests/test_extension_contract.py @@ -15,6 +15,7 @@ TEST_CASE_VALIDATOR = ROOT / "scripts" / "python" / "validate_test_cases_intake.py" VISUAL_PREVIEWS_VALIDATOR = ROOT / "scripts" / "python" / "validate_visual_previews.py" VISUAL_SPEC_PACKAGE_VALIDATOR = ROOT / "scripts" / "python" / "validate_visual_spec_package.py" +FIGMA_METADATA_CAPTURE = ROOT / "scripts" / "python" / "capture_figma_metadata_shards.py" def write_visual_intake_fixture(intake: Path, source_type: str, fidelity: str, file_name: str): @@ -139,6 +140,62 @@ def write_visual_intake_fixture(intake: Path, source_type: str, fidelity: str, f ) +def write_figma_metadata_fixture(intake: Path): + import hashlib + + metadata = intake / "figma-metadata.part-001.xml" + metadata.write_text( + '\n', + encoding="utf-8", + ) + digest = hashlib.sha256(metadata.read_bytes()).hexdigest() + (intake / "figma-metadata.index.yaml").write_text( + "\n".join( + [ + "file_url: https://www.figma.com/file/example", + "file_key: example", + "page_id: page-1", + "selected_node_ids: ['1']", + "captured_at: '2026-06-23T00:00:00Z'", + "mcp_tool: get_metadata", + "design_version_or_timestamp: '2026-06-23T00:00:00Z'", + "selected_subtree_complete: true", + "raw_metadata_complete: true", + "expected_root_node_ids: ['1']", + "captured_root_node_ids: ['1']", + "missing_root_node_ids: []", + "gap_count: 0", + "gaps: []", + "shards:", + " - path: figma-metadata.part-001.xml", + f" byte_size: {metadata.stat().st_size}", + f" sha256: {digest}", + " root_node_ids: ['1']", + " node_count: 2", + " truncated: false", + "", + ] + ), + encoding="utf-8", + ) + (intake / "figma-node-inventory.yaml").write_text( + "\n".join( + [ + "raw_node_count: 2", + "inventory_node_count: 2", + "excluded_node_count: 0", + "missing_node_count: 0", + "duplicate_node_count: 0", + "truncated_raw_evidence: false", + "node_inventory_coverage: 100%", + "parity_passed: true", + "", + ] + ), + encoding="utf-8", + ) + + def write_prd_intake_fixture(intake: Path): intake.mkdir(parents=True, exist_ok=True) source_dir = intake / "source-files" @@ -311,15 +368,53 @@ def write_image_visual_intake_fixture(intake: Path): def write_visual_previews_fixture(html_dir: Path): visual_intake = html_dir.parent write_visual_intake_fixture(visual_intake, "figma", "high", "figma-source.txt") + write_figma_metadata_fixture(visual_intake) html_dir.mkdir(parents=True, exist_ok=True) screenshots = html_dir / "screenshots" screenshots.mkdir(exist_ok=True) (screenshots / "home-desktop.png").write_bytes(b"fake-png") (html_dir / "component-matrix-preview.html").write_text( - '
' + '
' + '
' + '
Home default state
' + "
" + '' + '' + '' + '' + '' + '' + '' + '' + '' + '' + '' + '' + "" + "
headerdefaultclick Savebutton enabledsubmit action is requesteddefault to submittednoneblocked state remains missing when unsupportedfigma://node/2covered
" + '
' '' + "
" + '' + '' + '' + '' + '' + '' + '' + '' + '' + '' + '' + '' + '' + "" + "
defaultSave labelbuttonclickenabledpressed visual feedbackdefault to submittedhome pageunsupported states remain missingfigma://node/2covered
" + '
Coverage evidence conclusion
' "
", encoding="utf-8", ) @@ -345,6 +440,7 @@ def write_visual_previews_fixture(html_dir: Path): " source_ref: figma://node/2", " visual_spec_ref: ../visual-spec-package/visual-spec.yaml#VS-home-save-default", " preview_ref: component-matrix-preview.html#button-md-primary-default", + " interaction_ref: component-matrix-preview.html#button-md-primary-default-click", " screenshot_refs:", " - screenshots/home-desktop.png", " missing: []", @@ -378,6 +474,7 @@ def write_visual_previews_fixture(html_dir: Path): def write_visual_spec_package_fixture(package_dir: Path): visual_intake = package_dir.parent write_visual_intake_fixture(visual_intake, "figma", "high", "figma-source.txt") + write_figma_metadata_fixture(visual_intake) package_dir.mkdir(parents=True, exist_ok=True) (package_dir / "visual-spec.yaml").write_text( @@ -1567,6 +1664,150 @@ def test_visual_validator_blocks_unsupported_claim_even_when_packet_says_pass(): shutil.rmtree(work_dir) +def test_visual_validator_blocks_helper_artifacts_as_source_refs(): + work_dir = ROOT / ".tmp" / "test-visual-helper-artifact-source-refs" + if work_dir.exists(): + shutil.rmtree(work_dir) + intake = work_dir / "visual-design" + write_visual_intake_fixture(intake, "image", "medium", "wireframe.png") + + requirements = yaml.safe_load((intake / "visual-requirements.yaml").read_text(encoding="utf-8")) + requirements["requirements"][0]["source_refs"] = [ + "previews/component-matrix-preview.html#home-page", + "source-files/wireframe.png#full", + ] + (intake / "visual-requirements.yaml").write_text(yaml.safe_dump(requirements), encoding="utf-8") + + result = subprocess.run( + [sys.executable, str(VALIDATOR), "--json", str(intake)], + cwd=ROOT, + text=True, + capture_output=True, + ) + + payload = json.loads(result.stdout) + assert result.returncode == 1 + assert "VISUAL_SCHEMA_INVALID" in payload["blockers"] + assert "VISUAL_REQUIREMENTS_UNTRACEABLE" in payload["blockers"] + helper_refs = payload["details"]["visual_requirements"]["supporting_artifact_source_refs"] + assert helper_refs[0]["refs"] == ["previews/component-matrix-preview.html#home-page"] + + shutil.rmtree(work_dir) + + +def test_figma_metadata_capture_stages_shards_and_passes_validator(): + work_dir = ROOT / ".tmp" / "test-figma-metadata-capture-pass" + if work_dir.exists(): + shutil.rmtree(work_dir) + intake = work_dir / "visual-design" + write_visual_intake_fixture(intake, "figma", "high", "figma-source.txt") + raw_dir = work_dir / "raw" + raw_dir.mkdir(parents=True) + (raw_dir / "root-1.xml").write_text( + '\n', + encoding="utf-8", + ) + (raw_dir / "root-2.xml").write_text( + '\n', + encoding="utf-8", + ) + + capture = subprocess.run( + [ + sys.executable, + str(FIGMA_METADATA_CAPTURE), + str(intake), + "--metadata-source", + str(raw_dir / "root-1.xml"), + "--metadata-source", + str(raw_dir / "root-2.xml"), + "--file-url", + "https://www.figma.com/design/example/Foo", + "--file-key", + "example", + "--page-id", + "page-1", + "--node-id", + "1", + "--node-id", + "2", + "--captured-at", + "2026-07-02T00:00:00Z", + ], + cwd=ROOT, + text=True, + capture_output=True, + ) + + assert capture.returncode == 0, capture.stdout + capture.stderr + assert (intake / "figma-metadata.part-001.xml").exists() + assert (intake / "figma-metadata.part-002.xml").exists() + index = yaml.safe_load((intake / "figma-metadata.index.yaml").read_text(encoding="utf-8")) + inventory = yaml.safe_load((intake / "figma-node-inventory.yaml").read_text(encoding="utf-8")) + assert index["raw_metadata_complete"] is True + assert index["selected_subtree_complete"] is True + assert index["captured_root_node_ids"] == ["1", "2"] + assert index["shards"][0]["sha256"] + assert inventory["raw_node_count"] == 3 + assert inventory["parity_passed"] is True + + result = subprocess.run( + [sys.executable, str(VALIDATOR), str(intake)], + cwd=ROOT, + text=True, + capture_output=True, + ) + + assert result.returncode == 0, result.stdout + result.stderr + assert "Visual design intake readiness: PASS" in result.stdout + + shutil.rmtree(work_dir) + + +def test_figma_metadata_capture_blocks_truncated_shard(): + work_dir = ROOT / ".tmp" / "test-figma-metadata-capture-truncated" + if work_dir.exists(): + shutil.rmtree(work_dir) + intake = work_dir / "visual-design" + intake.mkdir(parents=True) + raw = work_dir / "truncated.xml" + raw.write_text('\n', encoding="utf-8") + + capture = subprocess.run( + [ + sys.executable, + str(FIGMA_METADATA_CAPTURE), + str(intake), + "--metadata-source", + str(raw), + "--file-url", + "https://www.figma.com/design/example/Foo", + "--file-key", + "example", + "--page-id", + "page-1", + "--node-id", + "1", + "--captured-at", + "2026-07-02T00:00:00Z", + "--json", + ], + cwd=ROOT, + text=True, + capture_output=True, + ) + + payload = json.loads(capture.stdout) + index = yaml.safe_load((intake / "figma-metadata.index.yaml").read_text(encoding="utf-8")) + assert capture.returncode == 1 + assert payload["status"] == "BLOCKED" + assert "FIGMA_RAW_METADATA_TRUNCATED" in payload["blockers"] + assert index["raw_metadata_complete"] is False + assert index["shards"][0]["truncated"] is True + + shutil.rmtree(work_dir) + + def test_validator_passes_complete_minimal_figma_intake(): work_dir = ROOT / ".tmp" / "test-validator-pass" if work_dir.exists(): @@ -1817,10 +2058,48 @@ def test_visual_previews_validator_reports_schema_errors_in_json(): shutil.rmtree(work_dir) +def test_visual_previews_validator_blocks_helper_artifacts_as_source_refs(): + work_dir = ROOT / ".tmp" / "test-visual-previews-helper-source-refs" + if work_dir.exists(): + shutil.rmtree(work_dir) + html_dir = work_dir / "visual-design" / "previews" + write_visual_previews_fixture(html_dir) + + component_coverage = yaml.safe_load((html_dir / "component-coverage.yaml").read_text(encoding="utf-8")) + component_coverage["components"][0]["source_ref"] = "component-matrix-preview.html#button-md-primary-default" + component_coverage["components"][0]["covered"][0]["source_ref"] = "screenshots/home-desktop.png" + (html_dir / "component-coverage.yaml").write_text(yaml.safe_dump(component_coverage), encoding="utf-8") + + viewport_coverage = yaml.safe_load((html_dir / "viewport-coverage.yaml").read_text(encoding="utf-8")) + viewport_coverage["viewports"][0]["source_refs"] = [ + "previews/component-matrix-preview.html#home-page", + "figma://node/1", + ] + (html_dir / "viewport-coverage.yaml").write_text(yaml.safe_dump(viewport_coverage), encoding="utf-8") + + result = subprocess.run( + [sys.executable, str(VISUAL_PREVIEWS_VALIDATOR), "--json", str(html_dir)], + cwd=ROOT, + text=True, + capture_output=True, + ) + + payload = json.loads(result.stdout) + assert result.returncode == 1 + assert "VISUAL_PREVIEW_SCHEMA_INVALID" in payload["blockers"] + assert "VISUAL_PREVIEW_ASSET_TRACEABILITY_INCOMPLETE" in payload["blockers"] + assert payload["details"]["component_coverage"]["supporting_artifact_source_refs"] + assert payload["details"]["viewport_coverage"]["supporting_artifact_source_refs"] + + shutil.rmtree(work_dir) + + @pytest.mark.parametrize( ("edit_kind", "expected_blocker"), [ ("missing_selector", "VISUAL_PREVIEW_FIGMA_NODE_COVERAGE_INCOMPLETE"), + ("ia_matrix", "VISUAL_PREVIEW_IA_MATRIX_INCOMPLETE"), + ("interaction_ref", "VISUAL_PREVIEW_IA_MATRIX_INCOMPLETE"), ("component_state", "VISUAL_PREVIEW_COMPONENT_STATE_COVERAGE_INCOMPLETE"), ("page", "VISUAL_PREVIEW_PAGE_COVERAGE_INCOMPLETE"), ("asset", "VISUAL_PREVIEW_ASSET_TRACEABILITY_INCOMPLETE"), @@ -1844,6 +2123,18 @@ def test_visual_previews_validator_blocks_incomplete_coverage(edit_kind, expecte ), encoding="utf-8", ) + elif edit_kind == "ia_matrix": + html = (html_dir / "component-matrix-preview.html").read_text(encoding="utf-8") + (html_dir / "component-matrix-preview.html").write_text( + html.replace('data-preview-section="component-ia-matrix"', ""), + encoding="utf-8", + ) + elif edit_kind == "interaction_ref": + html = (html_dir / "component-matrix-preview.html").read_text(encoding="utf-8") + (html_dir / "component-matrix-preview.html").write_text( + html.replace('data-interaction-id="button-md-primary-default-click"', ""), + encoding="utf-8", + ) elif edit_kind == "component_state": component_coverage = yaml.safe_load((html_dir / "component-coverage.yaml").read_text(encoding="utf-8")) component_coverage["components"][0]["missing"].append( @@ -1914,6 +2205,36 @@ def test_visual_spec_package_validator_passes_complete_minimal_bundle(): shutil.rmtree(work_dir) +def test_visual_spec_package_validator_ignores_preview_gate_status(): + work_dir = ROOT / ".tmp" / "test-visual-spec-package-ignores-preview-gate" + if work_dir.exists(): + shutil.rmtree(work_dir) + package_dir = work_dir / "visual-design" / "visual-spec-package" + write_visual_spec_package_fixture(package_dir) + previews_dir = package_dir.parent / "previews" + previews_dir.mkdir(parents=True) + (previews_dir / "component-coverage.yaml").write_text( + "ready_gate: BLOCKED\n" + "blockers: [VISUAL_PREVIEW_KNOWN_GAP_UNRESOLVED]\n" + "components: []\n", + encoding="utf-8", + ) + + result = subprocess.run( + [sys.executable, str(VISUAL_SPEC_PACKAGE_VALIDATOR), "--json", str(package_dir)], + cwd=ROOT, + text=True, + capture_output=True, + ) + + payload = json.loads(result.stdout) + assert result.returncode == 0, result.stdout + result.stderr + assert payload["status"] == "PASS" + assert not any(code.startswith("VISUAL_PREVIEW_") for code in payload["blockers"]) + + shutil.rmtree(work_dir) + + def test_visual_spec_package_validator_blocks_source_intake_blocked(): work_dir = ROOT / ".tmp" / "test-visual-spec-package-source-blocked" if work_dir.exists(): @@ -1947,6 +2268,44 @@ def test_visual_spec_package_validator_blocks_source_intake_blocked(): shutil.rmtree(work_dir) +def test_visual_spec_package_validator_blocks_helper_artifacts_as_fact_refs(): + work_dir = ROOT / ".tmp" / "test-visual-spec-package-helper-fact-refs" + if work_dir.exists(): + shutil.rmtree(work_dir) + package_dir = work_dir / "visual-design" / "visual-spec-package" + write_visual_spec_package_fixture(package_dir) + + package_doc = yaml.safe_load((package_dir / "visual-spec.yaml").read_text(encoding="utf-8")) + package_doc["items"][0]["source_refs"] = [ + "../previews/component-matrix-preview.html#button-md-primary-default", + "figma://node/2", + ] + (package_dir / "visual-spec.yaml").write_text(yaml.safe_dump(package_doc), encoding="utf-8") + + assertions_doc = yaml.safe_load((package_dir / "visual-spec-assertions.yaml").read_text(encoding="utf-8")) + assertions_doc["assertions"][0]["evidence_refs"] = [ + "visual-spec-evidence-packet.md#summary", + "figma://node/2", + ] + (package_dir / "visual-spec-assertions.yaml").write_text(yaml.safe_dump(assertions_doc), encoding="utf-8") + + result = subprocess.run( + [sys.executable, str(VISUAL_SPEC_PACKAGE_VALIDATOR), "--json", str(package_dir)], + cwd=ROOT, + text=True, + capture_output=True, + ) + + payload = json.loads(result.stdout) + assert result.returncode == 1 + assert "VISUAL_SPEC_SCHEMA_INVALID" in payload["blockers"] + assert "VISUAL_SPEC_PROVIDER_EVIDENCE_MISSING" in payload["blockers"] + assert "VS-home-save-default" in payload["details"]["visual_spec_package"]["provider_evidence_gaps"] + assert "VSA-home-save-visible" in payload["details"]["visual_spec_assertions"]["provider_evidence_gaps"] + + shutil.rmtree(work_dir) + + def test_visual_spec_package_validator_reports_schema_errors_in_json(): work_dir = ROOT / ".tmp" / "test-visual-spec-package-schema-error" if work_dir.exists():