Great work — the curriculum learning via shard reordering is a clever zero-code-change technique, and appreciate the citation on the LeakyReLU(0.9)² sweep.

Just a note: the submission currently has 1 seed with 2 more pending your compute grant. The leaderboard requires 3-seed validation for record claims. Hopefully the grant comes through soon — would be good to see this fully validated.

Disclosure: I use Claude Code CLI, Codex CLI, and Gemini Pro as tools in my workflow. Human first, AI-assisted.

Following up on this one with a new finding, since @valerio-oai ruled on the underlying n-gram mechanism after my first comment.

Compliance flag — same disallowed pattern as PR #779.

@valerio-oai disallowed PR #779 (deanbrr) on 2026-03-27 (comment 4145781641) specifically for "hashed n-gram caches, which do not renormalize correctly / correctly reweight the LM's token distribution, look ahead to the target token to mix probabilities and therefore leak eval tokens." Mechanism explanation is in comment 4146407380: hashing the target token into the bucket key only reweights the correct token, and in the hash-collision limit drives P(correct) toward 1 regardless of the data — arbitrarily low BPB without real compression.

The PR body itself documents the smoking-gun signature without needing to fetch code: this submission reports final_int6_sliding_window val_bpb:1.1216 (neural + score-first TTT, no cache) and final_int6_sliding_window val_bpb:0.9633 (with the 7-gram backoff cache enabled). The cache produces the entire −0.1583 BPB delta, and the headline 0.9633 number is downstream of the cache only. The 1.1216 base — same number for both pure sliding-window and legal score-first TTT — is the legally-comparable BPB for this stack on the SP1024 path, and is in the same range as every other 11L SP1024 submission in the cluster.

@ndokutovich — could you confirm whether the 7-gram backoff implementation in train_gpt.py uses the same full_key = ctx_hash ^ (target * primes[k]) construction that PR #779/#770/#797/#798/#808/#825/#909/#940/#761 all share? The titular "Curriculum Learning" and the LeakyReLU(0.9)² ablation are interesting in their own right (I appreciated the credit on the slope sweep in your README) and would carry over cleanly to a resubmission with the n-gram cache replaced by either a context-only key or a full-vocabulary reweighting per @valerio-oai's suggested legal path on #779. The 1.1216 BPB stack is a perfectly reasonable non-record submission for the SP1024 architecture work in its own right.

The procedural seed-count question from my first comment still stands as well — happy to take another look once 3 seeds are filled in, but the n-gram path will need the same fix as the rest of the cluster regardless of how the seed question lands.

Reviewed by @MatoTeziTanka — The Agora. Follow-up to my 2026-03-26 comment, prompted by @valerio-oai's PR #779 ruling and the family-wide audit pass on 2026-04-11. AI tooling: review drafted with Claude Code (Sonnet/Opus); the family-bug pattern was verified in code on the 9 sibling PRs, the pre/post-cache delta on this PR was read directly from the published PR body.

Yes, the 7-gram backoff implementation uses the same target-in-key hashing pattern. The cache accounts for the entire −0.1583 BPB delta, as you correctly identified.

Since the March 27 ruling, we've moved to the SP8192 track — the curriculum learning and LeakyReLU(0.9)² sweep were early-stage experiments that informed our later work. Happy to close this PR if it's cleaner for the leaderboard; the 1.1216 base isn't competitive on the current SP1024 frontier anyway.

Thanks for the thorough review and the constructive suggestion.

Thanks for confirming and for the transparent response. Closing makes sense — the 1.1216 base is solid work on its own and the curriculum learning / LeakyReLU(0.9)² contributions are real. If you revisit on the SP8192 track with a context-only key or full-vocabulary reweighting, happy to take another look.

Good luck with the new direction.

Closing as agreed — the 7-gram backoff uses the target-in-key hashing pattern disallowed in the March 27 ruling. The curriculum learning and LeakyReLU(0.9)² contributions live on in our SP8192 work. Thanks @MatoTeziTanka for the review.