Skip to content

Let the publish job push when the branch has moved, and commit the counter #77

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
leggetter wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Let the publish job push when the branch has moved, and commit the counter #77

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .github/workflows/eval-refresh.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -476,6 +476,16 @@ jobs:
# skills/ symlinks into the agent-skills submodule; without it a
# skills experiment installs nothing and silently scores as baseline.
submodules: recursive
# Full history, because the commit step below may have to rebase onto
# a branch that moved while the matrix ran — and did, on 1 September.
# A shallow clone has no merge base with `origin/main`, so
# `git rebase` fails and the retry loop exhausts its attempts with
# nothing it can do. That run published nothing and, worse, is red on
# a page that links to it as evidence: 115 of its 116 jobs succeeded
# and the failure was this step. Re-running it cannot fix that,
# because a re-run checks out the original commit and the original
# workflow file, so the fix only protects the runs after it.
fetch-depth: 0

- name: Install pnpm
uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v5.0.0
Expand Down Expand Up @@ -585,7 +595,12 @@ jobs:
# The published contract and its history. Staged explicitly rather
# than with `git add -A`, so a stray file a run happens to leave in
# the working tree cannot ride along into a published commit.
git add results/latest.json results/index.json results/runs
# `totals.json` too: `publish-snapshot` writes it — it is the "runs
# recorded" counter the page shows — and it was never staged here, so
# every CI publish left it at whatever the last hand-publish had
# written. Found on 2 September, when a hand-publish moved it from 222
# to 348 and nothing in CI ever would have.
git add results/latest.json results/index.json results/totals.json results/runs

if git diff --cached --quiet; then
echo "No eval result changes to commit"
Expand Down