[spdd] Daily spec work plan - 2026-07-15

[github-actions]

Summary

This daily SPDD plan covers 5 specification files reviewed in rotation index 10–14:

• specs/safe-output-outcome-evaluation.md (Working Draft v1.0.0)
• specs/security-architecture-spec-summary.md (v1.0.0, Candidate Recommendation)
• specs/security-architecture-spec-validation.md (Validation report, July 6 2026)
• specs/security-architecture-spec.md (v1.0.0, Candidate Recommendation)
• scratchpad/github-mcp-access-control-specification.md (v1.1.0, Draft)

Key findings: the validation report identifies a Sandbox Isolation evidence gap (T-SI-001 to T-SI-007, fully unverified), several partial-evidence gaps in Output/Network/Threat Detection compliance tests, and the safe-output outcome evaluation spec lists 5 not-started evaluators plus 10 partial evaluators needing JS-side dedicated logic. The MCP access control spec has no compliance test section filled in.

---

Priority Work Queue

P0 – Critical gaps blocking conformance claims

• Add sandbox isolation evidence to specs/security-architecture-spec-validation.md (T-SI-001 to T-SI-007 are fully unverified)
• Add formal test TestFormalPM11_PreActivationContainsMembershipStep to pkg/workflow/security_architecture_sg_formal_test.go

P1 – Incomplete evaluators affecting outcome accuracy

• Implement dedicated JS evaluator for close_discussion in actions/setup/js/evaluate_outcomes.cjs
• Implement dedicated JS evaluator for create_discussion in actions/setup/js/evaluate_outcomes.cjs
• Implement dedicated JS evaluator for push_to_pull_request_branch in actions/setup/js/evaluate_outcomes.cjs
• Document reply_to_pull_request_review_comment outcome logic in specs/safe-output-outcome-evaluation.md (currently not-started)

P2 – Spec quality and norms improvements

• Tighten Safeguards section in scratchpad/github-mcp-access-control-specification.md (Section 9 lacks normative MUST/MUST NOT language for rate-limit and 5xx error handling)
• Add Sync Follow-up notes to specs/security-architecture-spec-summary.md for the formal test suite added in v1.0.0

---

SPDD Checklist

• [Generate / T-SI] Add sandbox isolation evidence section to specs/security-architecture-spec-validation.md covering T-SI-001 through T-SI-007 (AWF chroot, Docker socket visibility, MCP container isolation). Done condition: all 7 test IDs change from GAP to EVIDENCED or PARTIALLY EVIDENCED in the compliance matrix.
• [Generate / SG-PM-11] Add test TestFormalPM11_PreActivationContainsMembershipStep to pkg/workflow/security_architecture_sg_formal_test.go. Compile a workflow with RBAC enabled, assert the pre_activation job YAML contains a check_membership step. Done condition: test passes with go test -run TestFormalPM11.
• [Generate / outcome-eval] Implement dedicated JS evaluator for close_discussion in actions/setup/js/evaluate_outcomes.cjs replacing the generic fallback. Done condition: status changes from partial to implemented in specs/safe-output-outcome-evaluation.md table.
• [Generate / outcome-eval] Implement dedicated JS evaluator for create_discussion in actions/setup/js/evaluate_outcomes.cjs. Done condition: status changes from partial to implemented.
• [Generate / outcome-eval] Implement dedicated JS evaluator for push_to_pull_request_branch in actions/setup/js/evaluate_outcomes.cjs. Done condition: status changes from partial to implemented.
• [Analysis / outcome-eval] Document not-started evaluators (reply_to_pull_request_review_comment, autofix_code_scanning_alert, create_code_scanning_alert, link_sub_issue, update_project, update_release) with a concrete acceptance/rejection definition in specs/safe-output-outcome-evaluation.md. Done condition: each row has a non-empty Accepted/Rejected column in the default acceptance map table.
• [REASONS / Safeguards] Add normative error-handling norms (RFC 2119 MUST) to Section 9 (Security Model) of scratchpad/github-mcp-access-control-specification.md for rate-limit (403/429) and 5xx responses from GitHub MCP server. Done condition: at least 2 new MUST-level requirements appear in that section.
• [Sync / summary] Update specs/security-architecture-spec-summary.md Spec Maintenance Tasks table to record the formal test suite (15 tests, pkg/workflow/security_architecture_sg_formal_test.go) under a new Sync row referencing the behavioral coverage map. Done condition: table contains a row with status ✅ for the formal test sync.
• [Analysis / T-OI] Add evidence entries for T-OI-003 through T-OI-007 (token precedence, secret expression validation, write isolation) to specs/security-architecture-spec-validation.md. Done condition: Output Isolation row in compliance matrix moves from PARTIALLY EVIDENCED to EVIDENCED.
• [Analysis / T-NI] Add evidence entries for T-NI-001 to T-NI-009 (network mode, domain matching, protocol filtering, MCP isolation) to specs/security-architecture-spec-validation.md. Done condition: Network Isolation row moves to EVIDENCED.

---

Per-Spec Findings

specs/safe-output-outcome-evaluation.md — Working Draft v1.0.0

REASONS Canvas:

• Requirements: Well-formed; RFC 2119 norms present. Missing requirement on retry count caps (how many retries before giving up on pending).
• Entities: Outcome categories well-defined. lifecycle_close semantics could be clearer (vs lifecycle).
• Approach: Implementation map is thorough; 5 not-started entries and 10 partial entries create risk that emitted OTel spans have wrong gh-aw.outcome.result for uncommon output types.
• Structure: Table layout good; missing a "priority order for promotion to implemented" guidance.
• Operations: No SLA or maximum retry count defined for pending→terminal transitions.
• Norms: Present via RFC 2119. No norm for what happens when gh-aw.outcome.source_trace_id is absent.
• Safeguards: No mention of what to emit if the evaluator crashes mid-run.

Key gaps: not-started evaluators (reply_to_pull_request_review_comment, autofix_code_scanning_alert, create_code_scanning_alert, link_sub_issue, update_project, update_release) fall back to evalGenericSticky which may produce incorrect outcomes for non-sticky actions.

specs/security-architecture-spec-summary.md — v1.0.0 Candidate Recommendation

REASONS Canvas:

• Requirements: All SG-01 to SG-07 listed. Formal model section newly added (July 9 2026).
• Entities: 7-layer architecture clear.
• Approach: Formal TLA+/F*/Z3 model is strong addition but not yet linked to a CI check.
• Structure: Good; "Next Steps" section has stale sub-tasks all marked done.
• Operations: Spec maintenance task table is up-to-date.
• Norms: SG notes updated. SG-01 clarification note is valuable.
• Safeguards: No mention of what happens when formal model diverges from implementation.

Key gap: Behavioral coverage map (15 predicates) is not referenced from the compliance test matrix in the main spec.

specs/security-architecture-spec-validation.md — Validation July 6 2026

REASONS Canvas:

• Requirements: References all SG and T-* identifiers.
• Entities: Implementation files identified accurately.
• Approach: Mostly static diff review, not automated.
• Structure: Good section breakdown; gap analysis section is the most actionable.
• Operations: No re-validation cadence defined.
• Norms: No MUST-level re-validation schedule (e.g., after every major version bump).
• Safeguards: ❌ GAP — Sandbox Isolation (T-SI-001 to T-SI-007) fully absent from evidence.

Key gaps: Sandbox Isolation (highest risk per document itself), Network Isolation T-NI-001 to T-NI-009, Output Isolation T-OI-003 to T-OI-007.

specs/security-architecture-spec.md — v1.0.0 Candidate Recommendation

REASONS Canvas:

• Requirements: 110+ formal requirements across 8 layers. Well-structured.
• Entities: All 7 layers have entity definitions.
• Approach: Defense-in-depth with explicit fail-secure.
• Structure: W3C-style with appendices A–H. Appendix A now includes job dependency graph.
• Operations: §12 compliance testing matrix defined but gaps noted above.
• Norms: RFC 2119 throughout. Strong normative language.
• Safeguards: SG-07 fail-secure is normative. Test TestFormalSG07 covers it.

Key gap: §12 compliance matrix has GAP status for Sandbox Isolation; formal model (TLA+/F*/Z3) not yet CI-enforced.

scratchpad/github-mcp-access-control-specification.md — v1.1.0 Draft

REASONS Canvas:

• Requirements: Good frontmatter syntax requirements; allowed-repos patterns well-specified.
• Entities: Integrity levels defined (none/unapproved/approved/merged).
• Approach: Type hierarchy clear; MCP Gateway integration flow documented.
• Structure: 11 sections; compliance testing section (§11) is referenced but not previewed here.
• Operations: Deprecated alias repos → allowed-repos migration documented.
• Norms: Design goals use "MUST" but body text is inconsistent (some "should", some "must").
• Safeguards: ❌ Section 9 Security Model lacks normative error handling for rate-limit / 5xx responses from GitHub API through MCP.

Key gap: Section 9 safeguards missing rate-limit/5xx norms; status still "Draft" — promote criteria not defined.

---

Sync Follow-ups

• After adding T-SI evidence to specs/security-architecture-spec-validation.md, update the Executive Summary to reflect new Sandbox Isolation status.
• After implementing JS evaluators for close_discussion and create_discussion, update the implementation status table in specs/safe-output-outcome-evaluation.md.
• After adding TestFormalPM11_PreActivationContainsMembershipStep, add it to the behavioral coverage map in specs/security-architecture-spec-summary.md.
• After tightening Section 9 of scratchpad/github-mcp-access-control-specification.md, re-evaluate whether the spec can be promoted from Draft to Candidate Recommendation.
• Track scratchpad/safe-outputs-specification.md deletion deadline (2026-09-21): ensure all internal links are redirected to canonical docs/src/content/docs/specs/safe-outputs-specification.md before that date.

---

Context

Field	Value
Files reviewed	specs/safe-output-outcome-evaluation.md, specs/security-architecture-spec-summary.md, specs/security-architecture-spec-validation.md, specs/security-architecture-spec.md, scratchpad/github-mcp-access-control-specification.md
Rotation index	10–14 (of 17 total spec files)
Previous run index	9 (2026-07-12)
Run URL	§29431752914

References:

• §29431752914

Generated by 📋 Daily SPDD Spec Planner · 47.4 AIC · ⌖ 5.69 AIC · ⊞ 4.8K · ◷

• expires on Jul 18, 2026, 8:25 AM UTC-08:00