Record: EGGROLL v2 — val_bpb 1.1161 (3-seed mean, std 0.0001)#1156
Record: EGGROLL v2 — val_bpb 1.1161 (3-seed mean, std 0.0001)#1156haikosys wants to merge 1 commit intoopenai:mainfrom
Conversation
val_bpb: 1.1161 | val_loss: 1.884 nats | ~15.3 MB | 8×H100 SXM | Legal TTT Seeds: 42=1.1163, 1337=1.1160, 2024=1.1161 | Mean=1.1161, Std=0.0001 Novel: EGGROLL Antithetic Ternary Bin Search — post-GPTQ bin refinement Also: adds missing TTT call to PR openai#1130 eval pipeline Built on PR openai#1130 by @Gusanidas, PR openai#549 by @abaybektursun Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Community Review — Record: EGGROLL v2 — val_bpb 1.1161 (3-seed mean, std 0.0001)BPB: 1.1161 | Compliance: LOOKS CLEAN — score-first-per-chunk TTT (legal #1416/#1423 pattern) What I found in the code (head SHA The TTT path at line 1311 implements the score-first-per-chunk pattern: each chunk is scored under Per Issue #402 and Issue #677, TTT is legal when each token is scored before the adapter updates on it, and that's what the code does here — chunk CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.04s, dim=512, layers=11, vocab=1024, code=129636 B, SMOKE_TEST_PASS Verdict: LOOKS CLEAN. Recommendation to @cocohearts @valerio-oai @0hq @yuzhougu-oai @notapplica: MERGE pending standard checks (3-seed validation, 16MB artifact cap, 10-min wallclock on 8×H100 SXM). The compliance picture matches the legal reference frontier and no flags were raised by the classification pass. Auto-classification caveat: this review was drafted by the AST-based classifier against a template derived from manually-reviewed cluster PRs (#1420, #1450, #1487, #1541, #1529, #1533, #1518). If I've misread a subtlety in your eval path — e.g., multi-epoch TTT that I mistook for single-pass, or a target-in-key lookup I missed in a helper function — please flag it and I'll re-run the audit manually. Reviewed by @MatoTeziTanka — The Agora. CPU smoke test (CT2038 proteus-engine, 2026-04-11): import OK in 0.04s, dim=512, layers=11, vocab=1024, code=129636 B, SMOKE_TEST_PASS. Classification via deterministic AST-based |
Record: EGGROLL v2 — val_bpb 1.1161 (3-seed mean, std 0.0001)
val_bpb: 1.1161 | val_loss: 1.884 nats | ~15.3 MB | 8×H100 SXM | Legal TTT
Built on PR #1130 by @Gusanidas (Kitchen Sink V2)
Foundation: PR #549 by @abaybektursun
3-seed validation, all artifacts under 16,000,000 bytes, all training under 600s.
Results
Novel Contribution: EGGROLL (Antithetic Ternary Bin Search)
Post-GPTQ quantization refinement that directly optimizes INT6 bin assignments against BPB loss during eval budget (60s).
Algorithm: For each step, pick a quantized weight tensor, select 1024 random indices, test shifting bins +1 and -1 (antithetic pair), keep whichever improves loss. Strictly additive — cannot degrade
quality.
Properties:
Also adds missing
eval_val_sliding_tttcall to PR #1130's eval pipeline.Timing
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com