Motivation
Both 2026 design reviews named measurement as the biggest gap: nothing records whether the workflow's gates actually catch anything, so decisions like risk-triage the 4b gate, merge corvus/corvus-auto, or keep Phase 3.5 mandatory in auto mode rest on anecdote.
Session evidence that motivates it (hand-recorded, n=3 features): plan-reviewer 3 real catches in 3 runs; 5b 16 real catches; 4b implementation gates 0 catches in 12 phases (everything already caught by per-task validation). Telemetry formalizes exactly this signal.
Settled design (from the design discussion)
Location: .corvus/tasks/[feature]/telemetry.jsonl — per-feature, next to MASTER_PLAN.md and CONTEXT.md (the feature's third artifact). Rationale: already inside task-planner's pinned .corvus/tasks/** write glob (zero permission change); verifier allowance extends exactly like the CONTEXT-delta one; no cross-feature write contention; aggregation stays trivial (cat .corvus/tasks/*/telemetry.jsonl | jq — JSONL concatenates).
Format: JSONL (JSON over Markdown for state files, per Anthropic harness findings). Three event types:
- Gate events (per gate run):
{ts, feature, gate: "plan-review|4b|5a|5b", phase, verdict, iteration, criteria, caught, classes[], fix_tasks[]} — caught = findings missed by all predecessors (the pays-rent field)
- Run-shape event (per feature, at 6a):
{feature, tasks, phases, workstreams, dispatches{implementer, quality, planner, reviewer}, fix_iterations, crashes, full_suite_runs, est_hours}
- Anomaly events: subagent crashes/empty returns, refused phantom instructions, unauthorized test runs, stale-count comments
Writers: task-planner at 4c PROGRESS_UPDATE (gate events) + 6a SUCCESS_EXTRACTION (run-shape); orchestrator supplies gate results it already holds. Three-surface rule applies: DISPATCH field + RECEIVER allowlist item + return-diff VERIFIER allowance (same pattern as CONTEXT DELTA — see three-surface-artifact in .corvus/tasks/learnings.md).
Durability: 6a appends a one-line run roll-up into the shared learnings.md entry, so the summary survives even if feature dirs are pruned.
Non-goals: no token/cost tracking (not reliably observable by agents), no external services, no new agents/skills (38/16/18 count pins stay).
Effort: ~5-7 tasks + per-phase tests, one plan-review cycle — same shape as the learnings-loop phase.
Open decision (blocker for the plan)
Persist .corvus/ in git? Currently gitignored → all plans/CONTEXT.md/learnings/telemetry are per-machine.
- Option A: un-ignore
.corvus/tasks/ — durable, cross-machine, reviewable; cost: diff noise
- Option B: keep gitignored, rely on the learnings roll-up as the durable summary
Owner preference leans toward persistence ('we'd want the corvus files to be persisted') — decide before planning.
Sequencing
Build after the modernization release ships and a few real features have run on the new prompts — telemetry has nothing meaningful to measure until then, and pre-release runs would pollute the baselines. Depends on: PR #5 merged + version release.
Motivation
Both 2026 design reviews named measurement as the biggest gap: nothing records whether the workflow's gates actually catch anything, so decisions like risk-triage the 4b gate, merge corvus/corvus-auto, or keep Phase 3.5 mandatory in auto mode rest on anecdote.
Session evidence that motivates it (hand-recorded, n=3 features): plan-reviewer 3 real catches in 3 runs; 5b 16 real catches; 4b implementation gates 0 catches in 12 phases (everything already caught by per-task validation). Telemetry formalizes exactly this signal.
Settled design (from the design discussion)
Location:
.corvus/tasks/[feature]/telemetry.jsonl— per-feature, next to MASTER_PLAN.md and CONTEXT.md (the feature's third artifact). Rationale: already inside task-planner's pinned.corvus/tasks/**write glob (zero permission change); verifier allowance extends exactly like the CONTEXT-delta one; no cross-feature write contention; aggregation stays trivial (cat .corvus/tasks/*/telemetry.jsonl | jq— JSONL concatenates).Format: JSONL (JSON over Markdown for state files, per Anthropic harness findings). Three event types:
{ts, feature, gate: "plan-review|4b|5a|5b", phase, verdict, iteration, criteria, caught, classes[], fix_tasks[]}—caught= findings missed by all predecessors (the pays-rent field){feature, tasks, phases, workstreams, dispatches{implementer, quality, planner, reviewer}, fix_iterations, crashes, full_suite_runs, est_hours}Writers: task-planner at 4c PROGRESS_UPDATE (gate events) + 6a SUCCESS_EXTRACTION (run-shape); orchestrator supplies gate results it already holds. Three-surface rule applies: DISPATCH field + RECEIVER allowlist item + return-diff VERIFIER allowance (same pattern as CONTEXT DELTA — see
three-surface-artifactin .corvus/tasks/learnings.md).Durability: 6a appends a one-line run roll-up into the shared learnings.md entry, so the summary survives even if feature dirs are pruned.
Non-goals: no token/cost tracking (not reliably observable by agents), no external services, no new agents/skills (38/16/18 count pins stay).
Effort: ~5-7 tasks + per-phase tests, one plan-review cycle — same shape as the learnings-loop phase.
Open decision (blocker for the plan)
Persist
.corvus/in git? Currently gitignored → all plans/CONTEXT.md/learnings/telemetry are per-machine..corvus/tasks/— durable, cross-machine, reviewable; cost: diff noiseOwner preference leans toward persistence ('we'd want the corvus files to be persisted') — decide before planning.
Sequencing
Build after the modernization release ships and a few real features have run on the new prompts — telemetry has nothing meaningful to measure until then, and pre-release runs would pollute the baselines. Depends on: PR #5 merged + version release.