Skip to content

alternate port: anthropic/claude-sonnet-5 — 37/37 gated effects, $17.2118, retrieval instead of context - #16

Closed
staccDOTsol wants to merge 1 commit into
omacom:masterfrom
staccDOTsol:zoo/sonnet
Closed

alternate port: anthropic/claude-sonnet-5 — 37/37 gated effects, $17.2118, retrieval instead of context#16
staccDOTsol wants to merge 1 commit into
omacom:masterfrom
staccDOTsol:zoo/sonnet

Conversation

@staccDOTsol

@staccDOTsol staccDOTsol commented Aug 16, 2026

Copy link
Copy Markdown

alternate port: anthropic/claude-sonnet-5 — $17.2118 all-in (DHH did not run this model)

One of five frontier models given DHH's exact challenge (same plan.md, same
Python reference) with one change: the 396k-token reference was retrieved
per-ask
through a holographic-memory layer instead of re-sent in context.

  • 37/37 effects, each gated on compiling AND rendering styled ANSI animation
  • $17.2118 total, provider-billed, every ask on the attached LEDGER.jsonl
  • 4.3x measured vs carrying the corpus (real cold/warm sends, not assumed cache rates)
  • watch the effects run from this binary: https://ttfx.awesomemcp.fun/show/sonnet

On the pure-math subset, we ran YOUR goldens against sonnet's code on glibc
(the platform class that generated them): 15,014/15,015 samples bit-exact
(99.993%)
— all 12 implemented named easings at 1001/1001, and all three
cubic-bezier goldens attempted via a transcribed make_easing Newton solver
(the sibling ports skipped beziers): two of three bit-exact, the third at
1000/1001 with the single miss 1 ulp off (1.4e-17) — an LLVM lowering artifact
we reproduced but couldn't close, reported rather than trimmed. The 354-case
parity suite remains this repo's bar and is not claimed here.

This adds alternates/anthropic_claude-sonnet-5/: the same challenge, run on the cost
axis, with a receipt for every ask.

honesty section (condensed from the run's live during-mortem)

The harness was improved WHILE this ran, and every dollar of that is in the
numbers above, not edited out: a render gate added mid-run dropped and re-bought
already-paid effects; three reasoning-budget truncation bugs bought ~$7 of
unusable responses before diagnosis; early asks shipped 3x-fat retrieval chunks;
restarts re-bought cores. Projections use the blended all-waste rate (marginal
rates run ~5x lower). The "without retrieval" comparison is MEASURED, not
modeled — real cold+warm full-corpus sends at provider-billed prices, which
found NO default cache discount for Anthropic or DeepSeek-via-OpenRouter. DHH's
side counts only his published successes; ours counts every failure. The
finished Rust port in this repo was quarantined from the models throughout —
they saw only plan.md and the Python source. Full ledger of every ask attached
(LEDGER.jsonl: tokens, provider-billed USD, timestamps).

Close it if exhibits don't belong here — the branch and the live page stand on
their own either way.

…118 via leCore retrieval

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@staccDOTsol staccDOTsol changed the title alternate port: anthropic/claude-sonnet-5 — 28/37 gated effects, $17.1706, retrieval instead of context alternate port: anthropic/claude-sonnet-5 — 37/37 gated effects, $17.2118, retrieval instead of context Aug 16, 2026
@dhh dhh closed this Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants