alternate port: claude-fable-5 — 37/37 gated effects, $54.82 vs $550 (beats by 90%), retrieval instead of context - #14
Closed
staccDOTsol wants to merge 1 commit into
Closed
Conversation
…69 via leCore retrieval Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
alternate port: anthropic/claude-fable-5 — $54.9769 all-in vs $550 DHH paid for the same model (beats by 90.0%)
One of five frontier models given DHH's exact challenge (same plan.md, same
Python reference) with one change: the 396k-token reference was retrieved
per-ask through a holographic-memory layer instead of re-sent in context.
Full parity wasn't targeted — the 354 cases in this repo remain the bar. But on
the pure-math subset, we ran YOUR goldens against fable's code on your pinned
parity platform (Linux/glibc 2.35): 12/12 implemented easing functions,
12,012/12,012 samples — 100.00% bit-exact. (One $0.157 post-run ask, on the
attached ledger, closed a 1-ulp powi/powf expression-order gap; fable wrote its
own fix. Unimplemented easings are absent, not failing.)
This adds
alternates/anthropic_claude-fable-5/: the same challenge, run on thecost axis, with a receipt for every ask.
honesty section (condensed from the run's live during-mortem)
The harness was improved WHILE this ran, and every dollar of that is in the
numbers above, not edited out: a render gate added mid-run dropped and re-bought
already-paid effects; three reasoning-budget truncation bugs bought ~$7 of
unusable responses before diagnosis; early asks shipped 3x-fat retrieval chunks;
restarts re-bought cores. Projections use the blended all-waste rate (marginal
rates run ~5x lower). The "without retrieval" comparison is MEASURED, not
modeled — real cold+warm full-corpus sends at provider-billed prices, which
found NO default cache discount for Anthropic or DeepSeek-via-OpenRouter. DHH's
side counts only his published successes; ours counts every failure. The
finished Rust port in this repo was quarantined from the models throughout —
they saw only plan.md and the Python source. Full ledger of every ask attached
(LEDGER.jsonl: tokens, provider-billed USD, timestamps).
Close it if exhibits don't belong here — the branch and the live page stand on
their own either way.