Let the publish job push when the branch has moved, and commit the counter - #77
Open
leggetter wants to merge 1 commit into
Open
Let the publish job push when the branch has moved, and commit the counter#77leggetter wants to merge 1 commit into
leggetter wants to merge 1 commit into
Conversation
…unter Two defects, both found because the 1 September run is red on a page that links to it as evidence. 115 of its 116 jobs succeeded; the one that failed was `publish-results`, after every eval had been measured. The publish job checks out shallow, so when its push is rejected — a matrix takes six hours and `main` moves — the retry loop's `git rebase origin/main` has no merge base to work from and all three attempts fail. Full history fixes it. This cannot repair the run that found it: re-running a job checks out the original commit and the original workflow file, so the fix only protects the runs after it. `results/totals.json` was never staged. `publish-snapshot` writes it and the page reads it for the "runs recorded" counter, so every CI publish left it at whatever the last hand-publish had written. It moved from 222 to 348 today, by hand, and nothing in CI would ever have moved it. Refs #74 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MQzUoMAwEBJWpEGVvVzSjK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two defects, both surfaced by the blog agent noticing that the run
/evalslinks to renders as failed.115 of that run's 116 jobs succeeded. The one that failed is
publish-results, and it failed after every eval had been measured — so the numbers are sound and the red mark is about how the file got committed.1. The publish job cannot recover from a moved branch
Its checkout is shallow. A matrix takes six hours,
mainmoves while it runs, the push is rejected as a non-fast-forward — and the retry loop'sgit rebase origin/mainhas no merge base in a shallow clone, so all three attempts fail.fetch-depth: 0fixes it.This cannot repair the run that found it. Re-running a job checks out the original commit and the original workflow file, so the fix protects the runs after it and nothing else. Making that specific run green would mean re-running the matrix (~$100) or publishing a snapshot that carries rows forward to manufacture a green tick, which is the thing #66 exists to prevent.
2. The "runs recorded" counter was never committed by CI
publish-snapshotwritesresults/totals.json, and the page reads it for the counter under the changelog heading. The commit step stageslatest.json,index.jsonandruns/— nottotals.json. So every CI publish left the counter at whatever a hand-publish had last written. It moved 222 → 348 today by hand; nothing in CI would ever have moved it.Refs #74.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MQzUoMAwEBJWpEGVvVzSjK