Complete Analyze artifact output contract

[Paul Lizer (paullizer)]

Summary

Completes the Analyze artifact output-contract roadmap tracked by #1233. Analyze now remains Markdown-first while requested generated outputs are produced as contract-faithful siblings. The cumulative feature preserves Search/Analyze structured-output equivalence for supported durable tabular requests, separates public schema from internal lineage, executes deterministic rules server-side, verifies and repairs semantic fields before checkpoint publication, and commits generated artifact sets only after every required member is valid.

Fixes #1233

Phase PRs

• Phase 1: Add analyze deliverable contract baseline #1225 Phase 1: executable deliverable contract baseline and 200-row oracle.
• Phase 2: unify Analyze deliverable intent #1226 Phase 2: unified intent and Analyze Markdown admission.
• Phase 3: separate public schema from tabular lineage #1227 Phase 3: public schema, internal lineage, and passthrough safety.
• Phase 4: Add tabular transformation contract #1229 Phase 4: deterministic tabular transformation contract.
• Phase 5: Add durable artifact-set lifecycle #1230 Phase 5: durable artifact-set lifecycle.
• Phase 6: Render plural artifact sets #1231 Phase 6: plural artifact UI.
• Phase 7: Add rollout rollback controls #1232 Phase 7 rollback controls.
• Phase 7A: stabilize Analyze artifact closure baseline #1234 Phase 7A stabilization and compatibility baseline.
• Phase 7B: implement tabular output correctness contract #1235 Phase 7B production correctness planning, verification, and repair.
• Phase 7C: enforce artifact-set publication lifecycle #1236 Phase 7C artifact-set publication lifecycle.
• Phase 7D: record Analyze artifact integration evidence #1237 Phase 7D final deterministic integration evidence and release notes.

Shipped Behavior

• Successful Analyze output remains Markdown-primary.
• Explicit CSV/JSON/XML durable structured outputs are sibling artifacts.
• Public outputs use requested schema/order and hide internal lineage fields unless requested as user data.
• Production Search and Analyze shared facades can persist the same reviewed transformation contract without injected output hints.
• Deterministic fields execute server-side through the allowlisted expression graph.
• Semantic fields use bounded independent verification and targeted repair before canonical checkpoints.
• Required invalid output fails closed after bounded repair exhaustion.
• New generated artifact-set members are staged until a completed, validated run manifest commits the publication generation.
• Direct artifact download/promotion reauthorize new artifact-set members against the committed run manifest.
• Legacy generated artifacts and old durable run readers remain compatible.

Rollout Defaults

• Planner, semantic validation, rollout, and legacy fallback controls remain backend-owned.
• Semantic validation defaults to off for new deployments until activated through rollout settings.
• Rollback and paused assignment controls stop new shared durable assignment without stranding accepted runs.
• Legacy readers, executors, status, resume, cancel, view, download, promotion, and fallback code remain available.

Validation

• Latest origin/Development is included in this integration branch.
• App version: 0.250.180.
• Syntax and VS Code diagnostics: clean for cumulative touched backend and test paths.
• Correctness matrix passed:
  • generated-file compatibility: 35/35
  • Analyze contract: 6/6
  • document-analysis artifacts: 6/6
  • JSON/XML exports: 7/7
  • public schema projection: 3/3
  • transformations: 5/5
  • semantic validation: 8/8
  • exact production Search/Analyze 200-row correctness: 2/2
• Lifecycle and rollout matrix passed:
  • artifact lifecycle authorization: 6/6
  • artifact-set lifecycle: 4/4
  • background exports: 8/8
  • rollout rollback: 3/3
  • rollout telemetry: 4/4
  • legacy retirement controls: 3/3
• Adapter and source-scope matrix passed:
  • shared planner: 8/8
  • parity contract: 4/4
  • Search preflight adapter: 8/8
  • Analyze preflight adapter: 6/6
  • all-scope source context: 4/4
• Full deterministic scale matrix passed through:
  • 30,000-row bounded finalization
  • 100,000-row deterministic planning and hardening contracts
  • authorization revalidation, cancellation, restart, lease fencing, idempotent publication, and route suppression checks
• Route policy suites passed: 12/12.
• Available Playwright generated-artifact UI tests passed: 3/3, with 4 authenticated environment-gated skips.
• Broken access control full-file guard passed on cumulative backend paths.
• Full-file XSS scan still reports pre-existing chat-messages.js sinks; this integration adds no JS lines in Phase 7D, and changed-line XSS scan had no new JS lines to scan.
• Markdown and Windows-safe diff hygiene passed.

Rollback

• Set semantic validation mode to off to stop new semantic verification/repair assignment.
• Set tabular Analyze parity rollout state to paused or rollback to stop new shared durable assignment.
• Existing accepted runs keep their persisted contract and remain readable/resumable/cancelable through compatibility paths.
• Reverting the final integration branch restores the previous staged roadmap state without data migration.

Deferred Follow-Ups

• Paid live semantic validation above 3,000 rows requires separate approval.
• Destructive legacy code deletion is intentionally deferred.
• Long-running cleanup sweepers for abandoned staged artifact-set members remain a follow-up.
• Pre-existing full-file chat-messages.js XSS sink warnings are outside this closure and should be handled separately.

[Copilot]

Pull request overview

Completes #1233 by making Analyze output Markdown-first while producing contract-faithful structured sibling artifacts.

Changes:

• Adds deterministic planning, semantic verification, and repair.
• Enforces atomic artifact-set publication and authorization.
• Adds plural artifact UI, rollout controls, tests, and documentation.

Reviewed changes

Copilot reviewed 38 out of 39 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
application/single_app/config.py	Bumps application version.
application/single_app/functions_analysis_deliverables.py	Defines deliverable contracts and validation.
application/single_app/functions_generated_file_exports.py	Detects formats and guards passthrough.
application/single_app/functions_settings.py	Adds backend rollout settings.
application/single_app/functions_simplechat_operations.py	Enforces artifact lifecycle authorization.
application/single_app/functions_tabular_generated_exports.py	Implements durable generation and publication.
application/single_app/functions_tabular_orchestration.py	Normalizes planning and rollout.
application/single_app/functions_tabular_semantic_validation.py	Verifies and repairs semantic fields.
application/single_app/functions_tabular_transformations.py	Evaluates deterministic transformations.
application/single_app/functions_workflow_runner.py	Produces Markdown-first Analyze artifacts.
application/single_app/route_backend_chats.py	Integrates guarded export routing.
application/single_app/route_enhanced_citations.py	Gates artifact download and promotion.
application/single_app/static/js/chat/chat-messages.js	Renders plural artifact sets.
functional_tests/test_analyze_artifact_phase7_rollout_rollback.py	Tests rollback assignment controls.
functional_tests/test_analyze_deliverable_contract.py	Tests the Analyze contract.
functional_tests/test_assistant_table_csv_artifact.py	Tests guarded file serialization.
functional_tests/test_document_analysis_lossless_artifacts.py	Tests Markdown-first document artifacts.
functional_tests/test_generated_artifact_lifecycle_authorization.py	Tests publication authorization.
functional_tests/test_generated_json_xml_exports.py	Tests structured format integration.
functional_tests/test_support/analyze_deliverable_contract_fixture.py	Supplies the 200-row oracle.
functional_tests/test_tabular_background_generated_exports.py	Tests background artifact projection.
functional_tests/test_tabular_phase3_public_schema_projection.py	Tests lineage-free public schemas.
functional_tests/test_tabular_phase5_artifact_set_lifecycle.py	Tests atomic artifact sets.
functional_tests/test_tabular_phase7_lifecycle_coverage.py	Tests lifecycle compatibility.
functional_tests/test_tabular_phase7b_production_correctness.py	Tests Search/Analyze equivalence.
functional_tests/test_tabular_phase8_ui_telemetry_rollout.py	Tests safe rollout metadata.
functional_tests/test_tabular_phase9_legacy_retirement.py	Tests legacy fallback controls.
functional_tests/test_tabular_shared_request_planner.py	Tests shared request classification.
ui_tests/test_chat_background_generated_export_status.py	Tests plural artifact rendering.
docs/explanation/features/ANALYZE_DELIVERABLE_CONTRACT.md	Documents the cumulative contract.
docs/explanation/features/TABULAR_TRANSFORMATION_CONTRACT.md	Documents transformation behavior.
docs/explanation/fixes/ANALYZE_ARTIFACT_PHASE_7A_STABILIZATION_FIX.md	Documents stabilization fixes.
docs/explanation/fixes/ANALYZE_ARTIFACT_PHASE_7B_CORRECTNESS_FIX.md	Documents correctness fixes.
docs/explanation/fixes/ANALYZE_ARTIFACT_PHASE_7C_PUBLICATION_FIX.md	Documents publication fixes.
docs/explanation/release_notes.md	Records the release closure.
docs/reference/logging-tags.md	Registers the new logging tag.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.