Skip to content

chore(scorecards): rescore all 96 tools with brewed anc v0.3.0#70

Merged
brettdavies merged 1 commit into
devfrom
chore/rescore-tools-anc-v0.3.0
May 1, 2026
Merged

chore(scorecards): rescore all 96 tools with brewed anc v0.3.0#70
brettdavies merged 1 commit into
devfrom
chore/rescore-tools-anc-v0.3.0

Conversation

@brettdavies
Copy link
Copy Markdown
Owner

@brettdavies brettdavies commented May 1, 2026 •
edited
Loading

Summary

First scoring run since both halves of the SHA-pin cleanup landed — this PR captures fresh scorecards for the entire
ANC 100 registry, scored by the just-released agentnative-cli v0.3.0 (no more ANC_COMMIT capture, no more
anc.commit field in the JSON envelope) installed via brew inside the docker scoring image (no operator-local
working-tree state coupling).

96/96 scored, 0 install-missing, 0 score-failed, 0 skipped.

Changelog

Changed

  • All 96 per-tool scorecards on anc.dev refreshed against anc v0.3.0. The JSON envelope no longer carries
    anc.commit; the per-tool page's "Anc build" row reads 0.3.0 (was 0.2.0).

Updated tool versions

Upstream releases since the last scoring run, picked up in this rescore:

  • act v0.2.87 → v0.2.88
  • anc v0.2.0 → v0.3.0 (self-scorecard rolls forward to match the now-installed binary)
  • claude-code v2.1.123 → v2.1.126
  • flyctl v0.4.44 → v0.4.45
  • ollama v0.22.0 → v0.22.1
  • opencode v1.14.30 → v1.14.31
  • starship v1.25.0 → v1.25.1

Type of Change

  • chore: Maintenance tasks (regenerated artifacts).

Related Issues/Stories

Testing

  • Unit tests added/updated
  • All tests passing

Test Summary:

  • 200 / 0 fail unit + regression tests pass.
  • bun run build clean: 111 pages, 97 scorecard pages, 96 badges, 0 orphans.
  • rg 'anc.commit\\|\"commit\":' dist/ returns zero — confirms no rendered surface still leaks the dropped field.
  • Spot-check on a handful of fresh scorecards: jq '.anc' <file> returns {\"version\":\"0.3.0\"} (no commit key).

Files Modified

Modified:

  • scorecards/*.json — 89 same-version files refreshed (anc.commit dropped, anc.version bumped to 0.3.0, run.started_at refreshed).

Renamed:

  • scorecards/act-v0.2.87.json → scorecards/act-v0.2.88.json
  • scorecards/anc-v0.2.0.json → scorecards/anc-v0.3.0.json
  • scorecards/claude-code-v2.1.123.json → scorecards/claude-code-v2.1.126.json
  • scorecards/flyctl-v0.4.44.json → scorecards/flyctl-v0.4.45.json
  • scorecards/ollama-v0.22.0.json → scorecards/ollama-v0.22.1.json
  • scorecards/opencode-v1.14.30.json → scorecards/opencode-v1.14.31.json
  • scorecards/starship-v1.25.0.json → scorecards/starship-v1.25.1.json

(The 7 superseded old-version files were trashed rather than left for the auto-discovery silent-supersede path —
keeps the disk in sync with what /score/<tool> will actually render.)

Created:

  • None.

Deleted:

  • None.

Breaking Changes

  • No breaking changes.

Deployment Notes

  • No special deployment steps required.

After merge, staging at agentnative-site-staging.brettdavies.workers.dev will serve the refreshed scorecards.
The /score/<tool> pages will render <dt>Anc build</dt><dd>0.3.0</dd> for every tool.

Known follow-up

  • scorecards/cf-v0.0.5.json's tool.version JSON field captured a row of Unicode ▄ block characters from
    cf's ASCII-art logo banner. The rendered /score/cf page is fine (filename version 0.0.5 is canonical), but
    the JSON's courtesy field is decorative junk. Captured in
    .context/compound-engineering/todos/020-pending-p3-cf-tool-version-decorative-first-line.md (local-only) with
    three fix-path options. Not blocking this rescore.

Checklist

  • Code follows project conventions and style guidelines
  • Commit messages follow Conventional Commits
  • Self-review of code completed
  • Tests added/updated and passing
  • No new warnings or errors introduced
  • Changes are backward compatible

Additional Context

The docs changes from PR #69 (content/scorecard-schema.md + tests/build.test.ts + RELEASES.md) are
already on dev; this rescore PR rides on top of them and only adds the regenerated scorecards/*.json
content.

After this lands on dev, the same scorecard regeneration can be cherry-picked into the open
release/2026-05-01-spec-vendoring-and-cleanup (PR #68) so that the production deploy to anc.dev ships fresh
scorecards in the same release as the supporting code/schema changes.

@brettdavies
chore(scorecards): rescore all tools with brewed anc v0.3.0
b6c062e 
First scoring run since both halves of the SHA-pin cleanup landed:
agentnative-cli v0.3.0 ships without ANC_COMMIT (no more anc.commit
field in the JSON envelope), and docker/score/ now installs anc via
brew (no operator-local cargo build, no working-tree state coupling).

Net change: 96 files changed across scorecards/.
- 89 same-version refreshes — anc.commit removed, anc.version bumped
  to 0.3.0, run.started_at refreshed, schema 0.5 badge block intact.
- 7 version bumps detected upstream during this run, with the old
  files trashed (no auto-discovery silent-supersede artifacts):
    act         v0.2.87  -> v0.2.88
    anc         v0.2.0   -> v0.3.0     <- self-scorecard rolls forward
    claude-code v2.1.123 -> v2.1.126
    flyctl      v0.4.44  -> v0.4.45
    ollama      v0.22.0  -> v0.22.1
    opencode    v1.14.30 -> v1.14.31
    starship    v1.25.0  -> v1.25.1

Run summary: 96/96 scored, 0 install-missing, 0 score-failed, 0 skipped.
Build clean (111 pages, 97 scorecard pages, 96 badges, 0 orphans).
Tests 200/200. rg 'anc.commit\|"commit":' dist/ returns zero.

Known follow-up: cf-v0.0.5.json's tool.version JSON field is decorative
junk (a row of U+2584 LOWER HALF BLOCK chars from the cf ASCII-art logo
banner). The rendered /score/cf page is fine (filename version is
canonical), but the JSON's courtesy field is ugly. Captured in
.context/compound-engineering/todos/020-pending-p3-cf-tool-version-
decorative-first-line.md with three fix paths to choose from in a
follow-up. Not blocking this rescore.
@brettdavies brettdavies enabled auto-merge (squash) May 1, 2026 10:27
@brettdavies brettdavies merged commit 47331f7 into dev May 1, 2026
2 checks passed
@brettdavies brettdavies deleted the chore/rescore-tools-anc-v0.3.0 branch May 1, 2026 10:27
@brettdavies brettdavies restored the chore/rescore-tools-anc-v0.3.0 branch May 1, 2026 10:28
@brettdavies brettdavies deleted the chore/rescore-tools-anc-v0.3.0 branch May 1, 2026 10:36
brettdavies added a commit that referenced this pull request May 3, 2026
@brettdavies
chore(scorecards): rescore all 96 tools with brewed anc v0.3.0 (#70)
b63a8d2 
## Summary

First scoring run since both halves of the SHA-pin cleanup landed — this
PR captures fresh scorecards for the entire
ANC 100 registry, scored by the just-released \`agentnative-cli v0.3.0\`
(no more \`ANC_COMMIT\` capture, no more
\`anc.commit\` field in the JSON envelope) installed via brew inside the
docker scoring image (no operator-local
working-tree state coupling).

96/96 scored, 0 install-missing, 0 score-failed, 0 skipped.

## Changelog

### Changed

- All 96 per-tool scorecards on anc.dev refreshed against \`anc
v0.3.0\`. The JSON envelope no longer carries
\`anc.commit\`; the per-tool page's "Anc build" row reads \`0.3.0\` (was
\`0.2.0\`).

### Updated tool versions

Upstream releases since the last scoring run, picked up in this rescore:

- \`act\` v0.2.87 → v0.2.88
- \`anc\` v0.2.0 → v0.3.0 *(self-scorecard rolls forward to match the
now-installed binary)*
- \`claude-code\` v2.1.123 → v2.1.126
- \`flyctl\` v0.4.44 → v0.4.45
- \`ollama\` v0.22.0 → v0.22.1
- \`opencode\` v1.14.30 → v1.14.31
- \`starship\` v1.25.0 → v1.25.1

## Type of Change

- [x] \`chore\`: Maintenance tasks (regenerated artifacts).

## Related Issues/Stories

- Story: Closes the loop after agentnative-cli v0.3.0 (#44 over there)
and site PRs #66, #67, #69 — first rescore against the new
no-\`anc.commit\` JSON shape.
- Issue: n/a
- Architecture: n/a (regenerated artifacts; no schema or code changes
here).
- Related PRs: #66, #67, #69 (already on dev); #68 (release branch —
open against main).

## Testing

- [x] Unit tests added/updated
- [x] All tests passing

**Test Summary:**

- 200 / 0 fail unit + regression tests pass.
- \`bun run build\` clean: 111 pages, 97 scorecard pages, 96 badges, 0
orphans.
- \`rg 'anc.commit\\|\"commit\":' dist/\` returns zero — confirms no
rendered surface still leaks the dropped field.
- Spot-check on a handful of fresh scorecards: \`jq '.anc' <file>\`
returns \`{\"version\":\"0.3.0\"}\` (no \`commit\` key).

## Files Modified

**Modified:**

- \`scorecards/*.json\` — 89 same-version files refreshed (anc.commit
dropped, anc.version bumped to 0.3.0, run.started_at refreshed).

**Renamed:**

- \`scorecards/act-v0.2.87.json\` → \`scorecards/act-v0.2.88.json\`
- \`scorecards/anc-v0.2.0.json\` → \`scorecards/anc-v0.3.0.json\`
- \`scorecards/claude-code-v2.1.123.json\` →
\`scorecards/claude-code-v2.1.126.json\`
- \`scorecards/flyctl-v0.4.44.json\` →
\`scorecards/flyctl-v0.4.45.json\`
- \`scorecards/ollama-v0.22.0.json\` →
\`scorecards/ollama-v0.22.1.json\`
- \`scorecards/opencode-v1.14.30.json\` →
\`scorecards/opencode-v1.14.31.json\`
- \`scorecards/starship-v1.25.0.json\` →
\`scorecards/starship-v1.25.1.json\`

(The 7 superseded old-version files were trashed rather than left for
the auto-discovery silent-supersede path —
keeps the disk in sync with what \`/score/<tool>\` will actually
render.)

**Created / Deleted:** None as net-new in the working tree (the 7
renames net to 0; the 96 modifications stay
in-place).

## Breaking Changes

- [x] No breaking changes.

## Deployment Notes

- [x] No special deployment steps required.

After merge, staging at
\`agentnative-site-staging.brettdavies.workers.dev\` will serve the
refreshed scorecards.
The \`/score/<tool>\` pages will render \`<dt>Anc
build</dt><dd>0.3.0</dd>\` for every tool.

## Known follow-up

- \`scorecards/cf-v0.0.5.json\`'s \`tool.version\` JSON field captured a
row of Unicode \`▄\` block characters from
cf's ASCII-art logo banner. The rendered \`/score/cf\` page is fine
(filename version \`0.0.5\` is canonical), but
  the JSON's courtesy field is decorative junk. Captured in

\`.context/compound-engineering/todos/020-pending-p3-cf-tool-version-decorative-first-line.md\`
(local-only) with
  three fix-path options. Not blocking this rescore.

## Checklist

- [x] Code follows project conventions and style guidelines
- [x] Commit messages follow [Conventional
Commits](https://www.conventionalcommits.org/)
- [x] Self-review of code completed
- [x] Tests added/updated and passing
- [x] No new warnings or errors introduced
- [x] Changes are backward compatible

## Additional Context

The docs changes from PR #69 (\`content/scorecard-schema.md\` +
\`tests/build.test.ts\` + \`RELEASES.md\`) are
already on \`dev\`; this rescore PR rides on top of them and only adds
the regenerated \`scorecards/*.json\`
content.

After this lands on \`dev\`, the same scorecard regeneration can be
cherry-picked into the open
\`release/2026-05-01-spec-vendoring-and-cleanup\` (PR #68) so that the
production deploy to anc.dev ships fresh
scorecards in the same release as the supporting code/schema changes.
This was referenced May 3, 2026
Closed
Merged
brettdavies added a commit that referenced this pull request May 3, 2026
@brettdavies
release: spec vendoring + SHA-pin cleanup arc + 96-tool rescore (#73)
31af939 
## Summary

First production release since the v0.1 launch (#60, 2026-04-30).
Promotes **8 PRs** from `dev` to `main`/anc.dev. Two
PRs from the post-launch batch (#61, #63) were verified as no-ops for
`main` and skipped: both were pure docs
reorganizations into `docs/research/` and `docs/design/`, neither of
which lives on `main` (or should).

The headline is **#64**: this site is now a downstream consumer of
`brettdavies/agentnative` (the spec repo) via a
vendored snapshot at `src/data/spec/`. Footer, OG card, and per-tool
badges now read from three independent spec-version
sources by design (vendored, scoring, manual reconciliation), preventing
the conflation that would otherwise lie about
at least one currency dimension.

Trailing the headline, this release also clears the SHA-pin cleanup arc
end-to-end: render side (#66), schema side
(#69), and a fresh 96-tool rescore against brew-installed `anc v0.3.0`
(#70). This is the first scoring pass where every
tool's `anc.commit` is `null` by construction (no `.git/` in the brew
cellar).

## Changelog

### Added

- Vendored `agentnative` spec snapshot at `src/data/spec/` (`VERSION`,
`CHANGELOG.md`, all 7 principles, `README.md`).
  Site is now a true downstream consumer of the canonical spec repo.
- `scripts/sync-spec.sh`: manual remote-first vendoring with
local-fallback.
- `scripts/SYNCS.md`: single-page cross-repo sync map (CLI / spec /
skill in, scorecards / Cloudflare out).

### Changed

- Footer, per-tool badges, and OG card pull from three distinct
spec-version sources (`SITE_SPEC_VERSION`,
`scorecard.spec_version`, `anc`'s self-scorecard `spec_version`). Each
tracks its own currency dimension.
- `anc` is now brew-installed inside the scoring docker image (replaces
the local-cargo-build-from-checkout path that
  produced wrong-SHA scorecards).
- All 96 tool scorecards regenerated against brew-installed `anc v0.3.0`
(#70). Supersedes the v0.2.0 baseline shipped
at v0.1 launch. First scoring pass where every `anc.commit` is `null` by
construction.

### Removed

- Skill SHA-pin enforcement surface (`source.commit`, `verify` block)
from `/skill.json` and the build validator. Update
detection now uses the skill bundle's `bin/check-update` against `main`
on GitHub.
- `anc.commit` field from the scorecard schema documentation and
invariant tests (#69). The field is no longer emitted
by `agentnative-cli` or surfaced in any rendered scorecard. Pairs with
the upstream cleanup in `agentnative-cli`.

### Fixed

- Per-tool scorecard pages no longer link to an incorrect commit SHA on
the `agentnative-cli` repo.
- Per-tool reproduce-fence at `/score/<tool>.md` now tags as `bash` (was
untagged).
- Correct stale `src/data/skill.json.version` (`0.1.0` to `0.2.0`) to
match the published `agentnative-skill` release
(#72). The manifest was never bumped alongside #67's deprecation of the
SHA-pin surface, so `/skill.json` consumers
  had been reading a stale value.

### Documentation

- Release runbook now uses a triple-diff verification (main → release /
release → dev / dev → main) plus a patch-id
cherry sweep (#69). Catches both directions of drift: guarded paths
leaking IN and real feature commits being missed
  OUT.
- "Status-check context pitfall" subsection added to RELEASES.md (#71).
Captures the exact-match rule for
`required_status_checks[].context` and the `gh api .../check-runs`
diagnostic for confirming real contexts after a
  first CI run.

## Type of Change

- [x] `feat`: New feature (non-breaking change which adds functionality)

The release is multi-typed (feat + fix + chore + docs) but `feat`
headlines because spec vendoring is the largest
user-facing surface.

## Related Issues/Stories

- Story: First post-v0.1 release; clears the post-launch backlog of
dev-side work plus the deep-check failure on the
  SHA-pin probe.
- Issue: n/a
- Architecture:
`docs/solutions/best-practices/agentnative-version-model-2026-05-01.md`
(cross-repo version model,
  dev-only)
- Related PRs: #62, #64, #65, #66, #67, #69, #70, #71, #72 (all merged
to dev between 2026-04-30 and 2026-05-03)

## Testing

- [x] Unit tests added/updated
- [x] All tests passing

**Test Summary:**

- 200 / 0 fail unit + regression tests pass on the release branch
(pre-push hook verified)
- `bun run build` clean: 111 pages, 97 scorecards, 96 badges
- Staging at `agentnative-site-staging.brettdavies.workers.dev` already
verified post-each-PR-merge: `/skill.json` has
no `source.commit`/`verify`; per-tool scorecard pages render the `Anc
build` row as version-only (no commit link)
- Skill-distribution e2e probe (the test that motivated this release;
see deep-check failure 2026-05-02 06:49 UTC) will
go green on next deep-check after merge: the assertion that fails on
`main` is removed by #67.

## Files Modified

**Modified:**

- `docs/DESIGN.md`: companion-artifacts section reframed shipped vs
research split (#62)
- `scripts/og/og.css`, `tests/e2e/flows.e2e.ts`: stale-ref cleanup (#62)
- `src/build/util.mjs`: `SPEC_VERSION` + `SITE_SPEC_VERSION` exports
(#64)
- `src/build/shell.mjs`: footer reads `SITE_SPEC_VERSION` (#64)
- `src/build/build.mjs`: badge generation reads scorecard `spec_version`
(#64)
- `src/build/scorecards-render.mjs`: `renderAncBuildHtml` /
`renderAncBuildMarkdown` collapsed to version-only; SHA
  allowlist regex + repo URL constant removed (#66)
- `src/build/skill.mjs`: `COMMIT_RE`, `verify` validation, "pinned at
commit" prose, and `## Verify` markdown section
  all removed (#67)
- `src/data/skill.json`: `source.commit` and `verify` object dropped
(#67)
- `docker/score/Dockerfile`: `brew install brettdavies/tap/agentnative`
replaces `COPY docker/score/anc` (#66)
- `docker/score/build.sh`, `docker/score/README.md`: drop cargo-build
preamble + `ANC_CLI_ROOT` plumbing (#66)
- `content/scorecard-schema.md`: `anc.commit` field documented as
captured-but-unsurfaced (#66); subsequently removed
  from schema doc + invariants (#69)
- `scripts/og/generate.ts`: reads `anc-v*.json`'s `spec_version` (#64)
- `tests/build.test.ts`: three commit-render tests consolidated (#66);
SHA-pin tests dropped (#67); `anc.commit`
  assertions removed (#69); spec-version coverage added (#64)
- `tests/e2e/skill.e2e.ts`: pin-freshness checks dropped (#67)
- `tests/regression.test.ts`: `source.commit` / `verify.expected`
invariants dropped (#67)
- `public/og-image.png`: OG asset regenerated (#64)
- `RELEASES.md`: skill-release procedure no longer requires a manifest
re-pin (#67); triple-diff verification added
  (#69); status-check context pitfall added (#71)
- `scripts/SYNCS.md`: re-pin language scrubbed from the skill flow (#67)
- `package.json`: version metadata updates from #64
- `scorecards/*.json`: 96 tool scorecards regenerated against `anc
v0.3.0` (#70); 7 of them renamed to track upstream
version bumps (act, anc, claude-code, flyctl, ollama, opencode,
starship)

**Created:**

- `src/data/spec/VERSION`: pinned spec semver (#64)
- `src/data/spec/CHANGELOG.md`: vendored copy of the spec repo's
changelog (#64)
- `src/data/spec/README.md`: vendoring workflow doc (#64)
- `src/data/spec/principles/p1-...` through `p7-...`: 7 principle files
vendored from the spec repo (#64)
- `scripts/sync-spec.sh`: vendoring script (#64)
- `scripts/SYNCS.md`: cross-repo sync map (#64)

**Renamed:**

- 7 scorecard files renamed in #70 to track upstream version bumps:
`act-v0.2.87.json` → `act-v0.2.88.json`;
`anc-v0.2.0.json` → `anc-v0.3.0.json`; `claude-code-v2.1.123.json` →
`claude-code-v2.1.126.json`;
`flyctl-v0.4.44.json` → `flyctl-v0.4.45.json`; `ollama-v0.22.0.json` →
`ollama-v0.22.1.json`; `opencode-v1.14.30.json`
→ `opencode-v1.14.31.json`; `starship-v1.25.0.json` →
`starship-v1.25.1.json`.

**Deleted:**

- None.

## Key Features

- **Three-source spec versioning model.** Footer / per-tool badges / OG
card each pull from a different spec-version
source by design. Conflating them would lie about at least one currency
dimension. Documented in
`docs/solutions/best-practices/agentnative-version-model-2026-05-01.md`
(dev-only).
- **Brew-installed `anc` in the scoring image.** Install parity with
every other registry tool, no more operator-state
coupling, scorecards from this image will always emit `anc.commit: null`
(no `.git/` in the brew cellar) by
  construction.
- **End-to-end SHA-pin cleanup.** Render (#66), schema (#69), and
rescore (#70) land together; the deep-check failure on
the skill-distribution pin probe goes away by removing the assertion in
#67.

## Benefits

- **Honest version surfaces.** Each spec-version display tracks its own
currency dimension; no more conflation.
- **Schema-render coherence.** `/skill.json` shape now matches the
documented update model (`bin/check-update` against
  `main`); per-tool scorecard pages no longer link incorrect SHAs.
- **Dead-ceremony reduction.** Skill releases no longer think about a
SHA bump that doesn't carry a contract.
- **Release-runbook hardening.** Triple-diff and status-check pitfall
both came from real-world misses; both are now
  permanent runbook entries.

## Breaking Changes

- [x] No breaking changes for end users.

`/skill.json` loses `source.commit` and the `verify` object. Agents that
read these advisory fields will see them
missing rather than wrong; consumers that don't read them are
unaffected. `agentnative-cli`'s
`src/skill_install/skill.json` fixture pulls from this repo's
source-of-truth, so its `skill-fixture-drift` workflow
will surface the shape change on its next PR. Coordinated CLI-side
update should land in lockstep.

## Deployment Notes

- [x] No special deployment steps required.

After deploy:

- `https://anc.dev/skill.json` will lose `source.commit` and the
`verify` object.
- Per-tool scorecard pages on `anc.dev` will no longer render commit
links.
- `anc.dev/score/anc` will display `anc v0.3.0` (was `v0.2.0`).
- Footer / OG card / badges continue to display version labels (now from
the vendored spec source).

## Skipped from this release (verified no-op for main)

- **#61** `refactor(layout): split docs/design into shipped src/styles +
research`. The "shipped" half
(`src/styles/foundation.css`, `scripts/og/og.{css,html}`, etc.) was
already on `main` via the v0.1 squash; the
remaining changes were a rename into `docs/research/design/`, which is
excluded from `main`.
- **#63** `chore(docs): move docs/VOICE.md to docs/research/VOICE.md`.
`docs/VOICE.md` never existed on `main`; the move
target (`docs/research/VOICE.md`) is excluded from `main` (now blocked
by `guard-main-docs.yml`'s extension to cover
  `docs/research/` and `docs/ideation/`).

## Checklist

- [x] Code follows project conventions and style guidelines
- [x] Commit messages follow [Conventional
Commits](https://www.conventionalcommits.org/)
- [x] Self-review of code completed
- [x] Tests added/updated and passing
- [x] No new warnings or errors introduced
- [x] Changes are backward compatible

## Additional Context

`docs/research/` and `docs/design/` are intentionally excluded from
`main`. They hold research material that lives on
`dev` only. The central reusable workflow at
`brettdavies/.github/.github/workflows/guard-main-docs.yml` already
enforces this for `docs/research/` and `docs/ideation/` alongside
`docs/plans|brainstorms|solutions|reviews/`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@brettdavies