Skip to content

Add 11 loops: model-upgrade-migration, PR-gate guardrails, stateful guardrails, long-horizon migrations#14

Merged
DCCA merged 4 commits into
mainfrom
claude/openspec-skills-import-4xj0qo
Jun 23, 2026
Merged

Add 11 loops: model-upgrade-migration, PR-gate guardrails, stateful guardrails, long-horizon migrations#14
DCCA merged 4 commits into
mainfrom
claude/openspec-skills-import-4xj0qo

Conversation

@DCCA

@DCCA DCCA commented Jun 23, 2026

Copy link
Copy Markdown
Owner

Summary

Implements all remaining roadmap loops — the complex-loops ranking and the stateful / long-horizon backlog are now fully built out. 11 new loops across 4 commits, each following loopy's trigger → detect → act → output → guardrails contract with injected boundaries (fully unit-tested with fakes / a memory StateStore).

Model-upgrade migration (415f3e9)

  • model-upgrade-migration — golden-set diff → human-gated model switch on runPlan (baseline → candidate → diff → approve → apply). Reuses prompt-eval-gate's scorecard/regression logic.

Wave 1 — PR-gate & freshness guardrails (5593467)

Deterministic, single-shot, report-PR loops:

  • i18n-drift — flag missing/orphaned translation keys across locales
  • perf-budget — flag performance regressions vs a stored baseline
  • a11y-baseline — fail only on new accessibility violations vs a baseline
  • runbook-freshness — flag runbooks past their review interval

Wave 2 — stateful guardrails (619e69b)

On the durable StateStore (+ a human Gate where approval is warranted):

  • test-impact-budget — flag tests grown past a rolling EWMA runtime baseline; rolls the baseline forward
  • data-contract-guard — additive schema changes auto-record; breaking changes block behind a human gate before a PR records the new baseline
  • cost-guardrail — track per-resource idle streaks (grace period); gated remediation report once idle past minStreaknever auto-deletes
  • eval-set-drift — surface production categories missing from the eval set; record surfaced categories so each reports once

Includes a real bug fix: cost-guardrail's idle streak only persisted in act, but act only runs once the streak crosses the threshold — a chicken-and-egg that meant it could never fire. The counter now advances+persists in detect (every run, as the grace period requires); act reads the already-advanced value to avoid double-counting.

Wave 3 — long-horizon migrations (954a0bb)

Export-only programmatic advance* loops on the resumable runPlan primitive (the model-upgrade-migration pattern; resumable via plan:<planId> in the StateStore):

  • api-deprecation-rollout — announce → grace-period (long-horizon wait) → caller-drain → human-gated removal. Won't remove an API while callers remain.
  • dep-major-migration — migrate one major bump (the kind dep-updates skips): verify the consumer build → human-gate a green result → emit the manifest bump. A red build is never gated or proposed; the failing log is surfaced.

Each wave-2 loop is wired into the catalog, loopy run (env-file boundaries + file StateStore), and package exports; the wave-3 programmatic loops add package exports. Every loop has a loop.yaml, README, playbook, and tests.

OpenSpec

Specs (source of truth) + archives added for all 11 loops under openspec/specs/ and openspec/archive/; openspec/roadmap.md and README.md updated.

Validation

  • npm run typecheck / lint — clean
  • npm test249 tests passing
  • npm run build — clean
  • CLI smoke: new runnable loops listed + wired (clear guidance without a data source)

Inventory: 26 loop directories, 21 runnable via the CLI catalog, 5 export-only programmatic long-horizon loops.

🤖 Generated with Claude Code

https://claude.ai/code/session_012CrMioorha3QtPUgWbtDbR


Generated by Claude Code

claude added 4 commits June 23, 2026 22:14
Rank #7 from research deep-dive #3. A long-horizon loop on runPlan + Gate +
StateStore that migrates to a new model version safely:

- loops/model-upgrade-migration/: advanceModelUpgrade() runs a resumable plan
  baseline -> candidate -> diff -> approve (gate) -> apply. It evaluates a
  golden set on the current and candidate models (injected evaluate(modelId)),
  computes regressions + score delta (reusing prompt-eval's Scorecard/
  regressions), blocks at a human approval gate, and on approval emits the
  model-id bump change set (result.bump); on rejection it holds.

Programmatic like the experiment orchestrator (a PR adapter/CLI verb is a
shared follow-up). Shipped via the OpenSpec cycle
(openspec/specs/model-upgrade-migration.md). Validated: typecheck, lint, 174
tests (+2), clean build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CrMioorha3QtPUgWbtDbR
Wave 1 of the remaining roadmap loops (research deep-dive #3), built in
parallel via subagents — deterministic, single-shot, report-PR output, with
injected file-based boundaries; wired into the catalog + loopy run + package
exports.

- i18n-drift: missing/orphaned translation keys across locales
- perf-budget: metric regressions vs a stored baseline
- a11y-baseline: new accessibility violations vs a baseline
- runbook-freshness: runbooks past their review interval

Shipped via the OpenSpec cycle (4 specs in openspec/specs/). Validated:
typecheck, lint, 200 tests (+26), clean build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CrMioorha3QtPUgWbtDbR
…ost-guardrail, eval-set-drift

Four loops on the durable StateStore (+ Gate where human approval is warranted):

- test-impact-budget: per-test runtime vs an EWMA baseline; flags regressions
  past a growth threshold, writes a report PR, rolls the baseline forward.
- data-contract-guard: diffs the current schema vs an approved baseline; additive
  changes auto-record, breaking changes block behind a human Gate before a PR
  records the new baseline.
- cost-guardrail: tracks per-resource idle streaks (advanced every run as a grace
  period) and proposes gated remediation once idle past minStreak; never deletes.
- eval-set-drift: surfaces production categories missing from the eval set as
  proposed new eval cases, recording surfaced categories so each reports once.

Each wires into the catalog, `loopy run` (env-file boundaries + file StateStore),
and package exports, with unit tests over fakes and a memory StateStore. Specs +
archives added; README/roadmap updated.

Fixes a cost-guardrail accumulator bug: the idle streak is now advanced and
persisted in detect (every run) instead of only in act, so it can actually reach
the threshold; act reads the already-advanced value to avoid double-counting.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ration

Two export-only, programmatic loops on the resumable `runPlan` primitive (the
`model-upgrade-migration` pattern: an `advance*` function over a step plan,
persisted under `plan:<planId>` in the StateStore, resumable across runs):

- api-deprecation-rollout: drives an API deprecation across long-horizon stages
  — announce → grace-period (long-horizon wait) → verify-callers (drain) →
  approve-removal (human gate) → remove. It will not remove an API while callers
  remain, and removal is emitted only after a human approves the gate.
- dep-major-migration: migrates one major dependency bump (the kind dep-updates
  deliberately skips) — verify the consumer build/tests against the candidate,
  human-gate a green result, then emit the manifest bump. A red build is never
  gated and never proposed; the failing log is surfaced instead.

Each emits its change set as a FileChange[] for a caller/adapter to open as a PR.
Package exports + specs + archives added; README/roadmap updated. With these, the
complex-loops ranking and the stateful/long-horizon backlog are fully implemented.

Validation: typecheck + lint + 249 tests + build all passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@DCCA DCCA left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review

Reviewed the full diff (114 files, +5804 / −5, all four commits). Findings:

Consistency — all 11 loops follow the established contract and conventions: ESM NodeNext with .js relative imports, import type for type-only imports, *FromManifest factories with typeof config guards, deterministic renderers, and fail-safe detect/act. The wave-3 loops mirror model-upgrade-migration exactly (runPlan steps + an advance* export, resumable via plan:<planId>).

Boundaries — every external dependency is injected (timing/usage/schema/drift sources, verify, remainingCallers, readers) and faked in tests; no network in unit tests. CLI wiring reads env-file boundaries + a file StateStore, matching the wave-1 pattern.

Bug caught during integrationcost-guardrail persisted its idle streak only in act, but act only runs once the streak crosses the threshold, so the counter could never accumulate to fire. Fixed: the streak now advances+persists in detect (every run, which is what the grace period requires) and act reads the already-advanced value to avoid double-counting. Covered by the "builds a streak… then produces a PR once approved" and "never reaches the gate for a busy resource" tests.

Gates / safety — human-gated paths (data-contract-guard, cost-guardrail, api-deprecation-rollout, dep-major-migration) all hold until explicit approval and emit nothing destructive; cost-guardrail never auto-deletes, dep-major-migration never gates a red build, api-deprecation-rollout won't remove while callers remain.

Validation — typecheck, lint, 249 tests, and build all clean locally; CLI list/guidance smoke-tested. OpenSpec specs + archives present for all 11 loops.

LGTM — merging once the build check goes green.


Generated by Claude Code

@DCCA
DCCA merged commit d816d80 into main Jun 23, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants