feat: Martian benchmark harness — 5-phase evaluation pipeline#51

Draft

Nelson Spence (Fieldnote-Echo) wants to merge 8 commits into

mainfrom

feat/martian-benchmark-harness

Commits on Mar 13, 2026

feat(bench): add Martian benchmark harness — full 5-phase pipeline

Nelson Spence (Fieldnote-Echo)
and
Claude (claude)
committed
test(bench): add 33 tests for Martian benchmark harness

Nelson Spence (Fieldnote-Echo)
and
Claude (claude)
committed
fix(bench): strip markdown fences from Claude judge/extract responses

Nelson Spence (Fieldnote-Echo)
and
Claude (claude)
committed
fix: resolve tool + structured-output grammar conflict on local endpoints

Nelson Spence (Fieldnote-Echo)
and
Claude (claude)
committed
fix: suppress output_schema for non-structured providers (Anthropic et al)

Nelson Spence (Fieldnote-Echo)
and
Claude (claude)
committed
fix: preserve diff context in retry messages

Nelson Spence (Fieldnote-Echo)
and
Claude (claude)
committed
feat(bench): add indexed benchmark runner with lite toolkit fallback

Nelson Spence (Fieldnote-Echo)
and
Claude (claude)
committed
fix: add pragma allowlist for test fake credentials

Nelson Spence (Fieldnote-Echo)
and
Claude (claude)
committed