Record: Packed Causal N-gram + Dirichlet Backoff — val_bpb 0.0109 (3-seed mean, NEW SOTA)#1076

Closed

sofiabod wants to merge 67 commits intoopenai:mainfrom

sofiabod:submission/dirichlet-backoff-0109

Commits on Mar 18, 2026

Commits on Mar 23, 2026

Commits on Mar 26, 2026

exp52: n-gram cache 4M buckets, single eval pass, fix zero-prob mixing
sofiabod
committed
exp54: 5-gram fixed alpha=0.2 cache (PR #769 recipe)
sofiabod
committed
exp55: truly sequential n-gram (fix chunking stale-count bug)
sofiabod
committed
exp56: dict-based n-gram cache (zero collisions), fixed alpha=0.05
sofiabod
committed
exp57: multi-order backoff 2-5 gram dict cache, alpha=0.2
sofiabod
committed
exp58: rewrite n-gram to match PR #753/#769/#779 (dual hash tables, per-window score-first, entropy-adaptive alpha, tc>0 check)
sofiabod
committed
exp59: 9-gram + per-order entropy thresholds + distributed prefill
sofiabod
committed
exp60: adopt PR #825 full stack (MHA 8/8, MLP 3.5x, XSA-all, BigramHash 6144, int5, stride=32) + 9-gram prefill
sofiabod
committed
exp61: submission-ready (BigramHash 4096, skip diag evals, int5 QAT)
sofiabod
committed
Record: Order-Adaptive 9-gram Backoff + Distributed Prefill (mean val_bpb=0.4405, 3 seeds)
sofiabod
committed
exp62: two-pass full rescore + 16M buckets + 9-gram (PR #870 approach)
sofiabod
committed
exp63: alpha max 0.95 + per-order multipliers (suppress bigrams, boost 7-9 gram)
sofiabod
committed
exp64: extend to 15-gram backoff + entropy centers for orders 10-15
sofiabod
committed
exp65: 11-gram (from 15) to fit eval budget
sofiabod
committed
exp66: add LongPhraseCache (variable-length suffix matching) on top of two-pass 9-gram
sofiabod
committed
exp67: trim phrase cache to 3 probes [48,36,28] to fit eval budget
sofiabod
committed
exp68: single-pass phrase cache (score-first, causality-clean) + n-gram prefill
sofiabod
committed
exp69: single-pass 9-gram + prefill + alpha 0.95 + order mults (no phrase, no two-pass)
sofiabod
committed
exp70: stride 32->48 to fit eval budget
sofiabod
committed

Commits on Mar 27, 2026

Commits on Mar 28, 2026