Skip to content

benchmarks: record repeated-investigation pre-fix failure - #31

Merged
anusbutt merged 2 commits into
mainfrom
benchmarks/repeated-investigation-pre-fix
Aug 14, 2026
Merged

benchmarks: record repeated-investigation pre-fix failure#31
anusbutt merged 2 commits into
mainfrom
benchmarks/repeated-investigation-pre-fix

Conversation

@anusbutt

Copy link
Copy Markdown
Owner

Outcome

Records the completed repeated-investigation benchmark as a pre-fix baseline. The result is explicitly classified as a failure: GraphKeeper retrieved the stored grounded claims first, but the session still repeated substantial source and Git-history investigation.

The change is evidence-only under benchmarks/results/repeated-investigation-pre-fix-2026-08-14/. It does not modify GraphKeeper implementation, schemas, templates, or application behavior.

Evidence

Included in the result folder:

  • identical control and GraphKeeper task prompts
  • paired session outputs and investigation-step records
  • exact GraphKeeper entity, claims, producing run, and captured evidence
  • a short factual README stating the pre-fix failure

Focused verification performed before commit:

  • all three captured JSON files parsed successfully
  • the README was checked for the explicit FAIL result and no-product-change statement
  • the commit contains only the eight benchmark evidence files

Full build and test suites were not run because this PR adds documentation and captured benchmark data only.

Checklist

  • Documentation reflects the benchmark result accurately.
  • Schema compatibility is unchanged.
  • Append-only/evidence guarantees remain unchanged.
  • npm run build passes — not run; no implementation changes.
  • npm run typecheck passes — not run; no TypeScript changes.
  • npm test passes with no hidden skips — not run; evidence-only change.
  • npm run package:smoke passes — not run; package contents are unchanged.
  • No secrets, credentials, telemetry, generated archives, or unrelated edits are included.

Compatibility and recovery

No runtime, platform, diagnostic, schema, hook, or package behavior changes. Rollback is removal or revert of the single evidence commit; no manual recovery is required.

@anusbutt
anusbutt merged commit 7ed5656 into main Aug 14, 2026
6 checks passed
@anusbutt
anusbutt deleted the benchmarks/repeated-investigation-pre-fix branch August 14, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant