Skip to content

Cautious Muon + SP4096 + Depth Recurrence — val_bpb 1.1604 (non-record)#1381

Open
X-Abhishek-X wants to merge 2 commits intoopenai:mainfrom
X-Abhishek-X:cautious-muon-sp4096-depth-recur
Open

Cautious Muon + SP4096 + Depth Recurrence — val_bpb 1.1604 (non-record)#1381
X-Abhishek-X wants to merge 2 commits intoopenai:mainfrom
X-Abhishek-X:cautious-muon-sp4096-depth-recur

Conversation

@X-Abhishek-X
Copy link
Copy Markdown

Summary

Per-Seed Results

Seed val_bpb val_loss Artifact Size
42 1.1568 2.6619 15,179,504 B
314 1.1611 2.6717 15,173,470 B
999 1.1634 2.6770 15,159,223 B
Mean 1.1604 2.6702 15,170,732 B

Cautious Muon (Key Modification)

After Newton-Schulz orthogonalization and MuonEq-R row normalization, the update is masked:

caution_mask = (g * raw_grad > 0).to(g.dtype)
g = g * caution_mask / caution_mask.mean().clamp_min(1e-3)

This filters stale momentum directions, providing ~1.47x effective convergence per step with zero parameter overhead.

Architecture Stack

Compliance

  • Track A fixed predictor
  • Causal, normalized via softmax (F.cross_entropy)
  • Artifact under 16MB (max 15,179,504 bytes)
  • Training under 600s wallclock on 8xH100 SXM

Reproduction

SEED=42 torchrun --standalone --nproc_per_node=8 train_gpt.py

Credits

Applies Cautious Muon (arXiv:2411.16085) to mask Muon optimizer updates
where Newton-Schulz direction disagrees with raw gradient sign.
Built on PR openai#1334 base with SP4096, depth recurrence, parallel residuals,
MuonEq-R, QK-Gain 5.0, GPTQ INT6 + Brotli.

3-seed mean: 1.1604 bpb (seeds 42, 314, 999)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 5, 2026 13:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Track A (10min_16mb) records folder documenting an experiment that applies “Cautious Muon” masking to Muon optimizer updates on top of the existing SP4096 + depth recurrence + parallel residuals stack, and includes 3-seed training logs plus submission metadata.

Changes:

  • Adds a new record folder with README.md describing the technique and results.
  • Adds submission.json capturing aggregated metrics and per-seed artifact sizes.
  • Adds per-seed training logs and a self-contained train_gpt.py entrypoint.

Reviewed changes

Copilot reviewed 3 out of 6 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
records/track_10min_16mb/2026-04-05_CautiousMuon_SP4096_DepthRecur/README.md Documents the method (Cautious Muon) and experiment results + reproduction instructions.
records/track_10min_16mb/2026-04-05_CautiousMuon_SP4096_DepthRecur/submission.json Provides submission metadata and aggregated/per-seed metrics.
records/track_10min_16mb/2026-04-05_CautiousMuon_SP4096_DepthRecur/train_gpt.py Self-contained training script payload for reproducing the run.
records/track_10min_16mb/2026-04-05_CautiousMuon_SP4096_DepthRecur/train_seed42.log Seed 42 training + eval log for the submission.
records/track_10min_16mb/2026-04-05_CautiousMuon_SP4096_DepthRecur/train_seed314.log Seed 314 training + eval log for the submission.
records/track_10min_16mb/2026-04-05_CautiousMuon_SP4096_DepthRecur/train_seed999.log Seed 999 training + eval log for the submission.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +7 to +11
| Seed | val_bpb | val_loss | Artifact Size |
|------|---------|----------|---------------|
| 42 | 1.1568 | 2.6619 | 15,179,504 B |
| 314 | 1.1611 | 2.6717 | 15,173,470 B |
| 999 | 1.1634 | 2.6770 | 15,159,223 B |
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The results table rows start with double pipes (|| ...), which GitHub Markdown renders as an extra empty first column. Use single leading | for each row so the table displays correctly.

Copilot uses AI. Check for mistakes.
Comment on lines +16 to +21
The primary modification is applying the Cautious optimizer principle to the Muon optimizer. After Newton-Schulz orthogonalization and MuonEq-R row normalization, the update is masked to only apply where the orthogonalized direction agrees with the raw gradient sign:

```python
caution_mask = (g * raw_grad > 0).to(g.dtype)
g = g * caution_mask / caution_mask.mean().clamp_min(1e-3)
```
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description of Cautious Muon says the mask is applied “after Newton–Schulz orthogonalization and MuonEq-R row normalization”, but later the stack list claims MuonEq-R happens before Newton–Schulz. Please make the ordering consistent with the actual implementation to avoid confusion when reproducing.

Copilot uses AI. Check for mistakes.
- **SP4096 BPE tokenizer** (from PR #1218, @clarkkev)
- **Depth recurrence** layers 4,5 (13 virtual layers from 11 physical, activated at step 3000)
- **Parallel residuals** from layer 7 (separate attn/MLP lanes with learnable merge)
- **MuonEq-R** row normalization before Newton-Schulz (arXiv:2603.28254)
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bullet claims “MuonEq-R row normalization before Newton-Schulz”, which contradicts the earlier description of the masking point. Please align the ordering here with the actual optimizer pipeline (and with the earlier section) so readers don’t implement the wrong sequence.

Suggested change
- **MuonEq-R** row normalization before Newton-Schulz (arXiv:2603.28254)
- **MuonEq-R** row normalization after Newton-Schulz orthogonalization (arXiv:2603.28254)

Copilot uses AI. Check for mistakes.
Comment on lines +8 to +12
"val_loss": 2.67020395,
"val_bpb": 1.16043988,
"val_loss_std": 0.00764948,
"val_bpb_std": 0.00332438,
"seeds": [42, 314, 999],
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most existing 10min_16mb submission.json files include a bytes_total field (e.g., records/track_10min_16mb/2026-03-22_11L_EMA_GPTQ-lite_warmdown3500_QAT015_1.1233/submission.json:9). This submission uses artifact_bytes_* instead; consider adding bytes_total (and optionally bytes_code) for consistency and easier leaderboard/tooling ingestion.

Copilot uses AI. Check for mistakes.
"github_id": "X-Abhishek-X",
"name": "Cautious Muon + SP4096 + Depth Recurrence + Parallel Residuals",
"blurb": "Applies Cautious Muon (arXiv:2411.16085) to the Muon optimizer — masks Newton-Schulz updates where the orthogonalized direction disagrees with the raw gradient sign, providing ~1.47x effective convergence per step with zero parameter overhead. Built on PR #1334 (aryanbhosale) base with SP4096 vocabulary, depth recurrence (layers 4,5), parallel residuals (from layer 7), MuonEq-R, QK-Gain 5.0, and full GPTQ INT6 + Brotli compression. Mean val_bpb = 1.1604 (3 seeds, std = 0.0033).",
"date": "2026-04-05",
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The date field is just YYYY-MM-DD, while other submissions typically use an ISO-8601 timestamp (e.g. 2026-03-22T00:00:00Z). Using a consistent timestamp format helps automated consumers parse and sort submissions reliably.

Suggested change
"date": "2026-04-05",
"date": "2026-04-05T00:00:00Z",

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +2
import lzma as L,base64 as B
exec(L.decompress(B.b85decode("{Wp48S^xk9=GL@E0stWa8~^|S5YJf5;Y&G8oLvAkn@VT6Qap3bu0*kgCR~YUqB0W9R)iarr*QtEZpesGY3>~CZRiK|6Dwut$nH#N!RYqQnA}G^`ZsFO;ar92)Xt#3E3Ki5S1}OfSx<=$c<4=h|J{kt$27^CQ01M+lVgZ0tGgX0&I*V@{U&JgYc0U!(4F-btCy*+qzv6Dp~UW!y~6{U*}y$E@2-R}vd?t*s#fnDO{!j>OImt34A(d+9n>hnnvzmd((<?7SwGA+W$Or}R^?z{Ree`d98K#2b?5_cfZ%Lt17{dO-GN2&5;3ECS=cE#O*hTiZtOgX4XIkBuR{PAJ5$6e;Eel8-VVQFtsWBkSb@w{YAVOHKclDdD}`(Z5P0z`*Zpr!j8psZi3dj0Awg*|SM}VUSWz|5#6=0zD3_4d4q0&x{n`YrJE1HXz*`$RPocfD=uX!IiV{O9O7|!NUnrgGHr-RntBL^weTkpGrkcD-11{fi$WX(*bn-nbgD6YkvCezcUG_I`KWJk#hl}8>_D1Cghg~(bQ$Yj)>!Y%{*o9ex8FWa#U)!OI!!5Prl^?bnBX2V(=(Bvc+CvGo!S{LhLn7pSsR!}@U=OBW0)h6IYneQ1{|$<&k9TS^qGQpb-;#vEPAl%11UF)?6mtC8c04XzR$+h2=j84E2|i`pOEt$uyM`lGs*ejIF-}^SvSRZK$ePh1`gt+?%1r#=OV<HVjrIqeJYGxh%pSpPlDdQ66RHQe_t;x^R`DvG*6PkO>y3pW`{ofc)6PhQQRP|_h56zl+sQ(le1eJ^&&qZxdGb15aOb^-R1ouqi-H1_w|H;g(()bKz_!0+L{#HFmtQSw%~n|MX3ij_2{lW(_*6gdIz`XT%tzkhK-k5tAu}`=>u|z+<A5OWlS!1&|7BshLa`v33WEyuh5cV6d%c)Zp^vZ`+%XW7Ga$QNikdrw8}teN{lfKFYW8jLKpUE<fS0HI?!#S}z5(PnD*$%*&e)`J?Z?l)B@7}E*EtBQ<ajK2J2UVwA>|uP4UnMNw^{d&KAm;P6`40&zphh*D=e*8?KGZuo~y*`y#Wg}r(PV<PiL09`OF@Nh{a<w{zds*!NOI%lISf!q#(&1k8wXKJfU5I>}?J$O<io<ujGSX9NWFVj^&nmc<AiUyr-`}m?om^MsCAZ)88TX4@F=Gr{ba+%8mF)D2K;PS)r}K`0OW1F4MRM)*s7t_jmGr27SW%DBvhmeslZ(N-kfEgbZN}VIhl%?M7XjWH8wf?lo_X|5_oRRVxUolXyMc*~(GxY5pkyJ-C9J))sNjfO=O;m+FUfndz~U9#}!)KfJsF`v`a^5o$1$QiNNu!W#ZYR)zDNo?=6w(J-Mr%~Ccd@JvU7j6*UqI1G_U5)xIOVfFefW#-2fZay)3C_@H2WgGU<<j3Y(=kG=f*7C;k&v(6y7C}?)ki&L~IiRiQ(NiyD1>ae&vv%2(eb|cSn}oTUc1&B%M^B@xlvn10$Ol5{Rc(||g*`0AO8zk14;L<B89tNMDJH<nBP9kHyG#H@%Y{;9I%*-8=cLIDmul}ETySn#xzQAAJ)H3oDC_09q<t7H;T?vgXJnFe2W2eEM#4@zx^{Q`RI4rmXYqJ`j>7H$%dzpg>)#q=gM_#uqc-I?vb;GXT^_H=R{GSK>=VuvqSe%#z|2c2YlT&*kjZ2;Kl!jcm$Q-fVo|ict&Ja4ywb`i8dvCk+h?3OCDUXX&zMsrr>Pa>PMm^gxGuU~bBkga*GhQ$y8PeK(vV5>_veh8WQRi`fT<y2?%qa)RDFxTOGTC_Yl#}yg8C;;?WCn_iwWXDzN03qd_<{4@#oI{v^Qp5Ta*t1IOY_9!%73?T>)Py)OBaHdkXS7+2I5uh#w2)7T@22C(Qzp-K-tVkOU|oncF?OgMCg#5sW{yA4RZO!84eUpjR$IRG8(QIv>WqY`&obF`+L^a;Ik$ZPqJ0w+oVi8+fefP_6}>2vq29M(7pWSxwX%h;ULwd}`?78ERR(NNJ_4KVp|BN9>arZL(Qm544oVRUtWw?7Jk|KZ1C{%$Jv!T6;U}wiQVVE)4<71$jhkVz<Qtn`)a82WW$q0Hr{4+UPsbm$mWoflR~5=rA+eFXgkwC=?_lCr!4xuwO>^+^AaNS!~_W(vR4{WJ^svyxk^G3sz?R<rC;XSxpV71HMoESHR0<s%rK-;z}#rRPgX{3Q(A0Wp$2KXLI=1lOC(?@TlnLPSgNk$`a3`*6ksGvvQDcbXT5`hv<KTw5Y2&yEeu8k!Vh|_EpHFWU(bfUMSirttllpOsLyWqmB4^^^xE{p{jc~83a5_Lv6FW;@4UlDMPfv?~X~7+CW&3H+kc&D_4ToujfL`8*93hUr+a~GGo0xNE~Rg0>Yo_N1#V&d43t{_A|@#vLRTXJ)mh_LsXCAJVuNHeI8p77$-9H%-!CvcwCI8NI<;c*6{I5uPqIoTdj3Xcr{R+d)6lXf`Tkt(h3PL4|paL6IHOjh`kZhHjzRo&SJapcs5d^H130}9Do<&`*(lN1e>3UmSuApd3~p1>+iw(L!(EeP^xfD*_e|;Pwu$5EVQsNPN$X%#{QH-cBp=0SJ2S&fh>5{xov2Q&Dt8e2OD|)&mt5}pMa#y6QbXmJX!S1_u;C#vl1c(#HwFUIq)H-{6h49qdpK8DkSGDi&|0Gm*`a@fNS(bc?SbL#Nw1qWH_Sm8H<BSYbB+7>5d?bot%W2!a%_xL$l$wZMM<P2S6~e+Ooc>0%5SQBB#7p$O|$EKTZ^s?k{J%APQfIG>nefD{BM8Wv~Raz5FG};~ZrIG9BYxbU)6FnJ8gyh&Uq4Ujyc@m~D|>5=W&d7!2;#A`)#%c)a{*uY@1mteo&`!#O*c-WiWMu#FM+$tI&t-Vk$6T+lan2}MAv`?+7wse?lKLKyL?o?r>aEE}`ezoaZ6fF6dUE2tY2^trIgC4f5fCd!?N^%$XSL$_1XK0UJ!A@|%cDxnG}8w&B3$6`#1#KhW&Ql2C)5ib>m5Z=M-qv1|o$Q0|W(_^uCKX09E!%ojzZI?^Mz~TizB4o%X)QO_ql@26ymjSb3)uBO=dFg!x-)Qn;kD+UVD&>jk<8!}Ec-D__4+R~K1U5lNK|w2cixht+FS$0V>V>SzHs(Zk;OrtRtI3kcE`ureK?716@eYVye|+&bYUx-7g@AdF@_<77`U8(GXS0!&D^P-u;Xz}0WKVcbvGrEFSfxcV>TP{Zc-rn<riUyaMV`Pg_`~y%0<DnQZefADy^d3T|Fb4)EJH7Da9D*%i)O4~&hH^su=$@?2oQ_!TUNFuhg}a5zjTC?yRG$bdHLuKhKnNF`ksq?@{fV6I+4mIUygtt_wI%hETo_9<%eR}B^m>}A78K>*`a+rhV?lldBS^}%@}u<;Dvw<?Gc}<Wygn$lO0Dj)J4<~x>Qjfuz0#=9ImWl*WoNvxZqR-cYjGYAGA;O^VeTt+L*r29&lp&rMOX=u*JS$iZmC|mYP!1l_3fkt4g*|<MTxOq(euNH(JFt+d(QKMI?zE`8|xAPT8z>qc@{!A}d!ZdZdq-LX&)z*#yYN+$?Bqe|01W`-A|R{90OA;BuKH7-D8K#4(MtnL@`>u%l)7Wo*3m++MZ63TBcQJ>;K$x;rQt+>)>cEy7f1sq-E4<n`1+Up0H0*+B5qruY?(p~nhcRcp0Y<>-P=o=ctbBsP$HEE{Q{GTSMN5dk@gm=-fy<)o05o|SSOL_PN>QA5sw1_nYpza?AifEtB`gGmk_a_KAt7OI9}fdzZ_1<re)5zJwY>%B7De4O;fOO3G|`*hODWE5!{={Lx~<E%MJ<r_R!VkF85otI5gE?oa_d}oX!6a&Ix3mK`G;I9q*OO>J%N4q)qhTlK<c(3(FqTL8DLxXNLMoD5p?`8T1?m0G+SCBSl+4tkK0gruwl-=7j`1t9K;nxuA_f}P#KtZ?SRv}H(zN2^}SwNP2+$W`<Su}6k;sB#M<Xb@*Vu}TA-FpGD#+~J)ZA?kucvS*G`F(*2HiFm(e2q#KMtIA`-I)g-IGfjTE{-eU)P>;^Vw9bHSA_Rjjp~NFEtw76&M^}{dD!n3d^>P(=?OwyW56Vh-Y=t$TUe%*lb*hV-ny<{h?{d1B!^>a_Q*8%Kdqz9d2T04mMyR?`RR|!?x65zhQKdmwv;0x;sm38pzc6?(-obP9jShOmmBDa3go7BP9vbs9e&i9G9It6$5xhA7WL-OStpny<`(&kVy4?EY97TS{CRW^|JV3P3MOU;M}3+-DhEnhy3!eIqtF+S-{R4s7R6+d;<+>uM>@JZqKZpVUC=+lYb!!T&vJQ_3;B(#xqMak-T8X!C7Y3r#O#;_VW-k*e#2c@cGI4hYrF*;PJq<v2)#o*K!M+`;Y=}R_BBZrSx1Hyg@>%|7492kH1(f@a^mAw^}9vO?<RWoU0{NxgAM%io;0Tsa_a2V#~aS*!I;#<f<Q08Q67gIzL7B||Nepa@^*;7pdD25wTM^~zz-Q0iSjlR@a%yFv(!7tJDoU;ALlNJzvm={V0i*T`$u{%|2XX@L-%9vj(W`s2c#Y7wTx5z4lFl7L>3(2y=4XxWf9{jAevrBh<|B#@>K#!Hj>cVI#kk^TT{*aP}pUue5rsHq(8L+tdb}!yS8}4mQ*vPU%%T>@i_qMFbLb#8SrKL(y*BcH_j`YW)jm|T1MSK=p0r8lb8HXOC#uTA@^z!q4|?60<sx5#i)CrY;b!Lh!+-CdnXiBmuZAfIHedsX9WV+IizyAlz0{S$Eh0W&k<@)ubL1D>;qd!ah5v9(Nj8;|4c%vp9p0(MOWt_SM3Z@xfoG$eFn%?m)Ewo{?^?%<SwrQ^^8=>@mxbUbKIBry{Rl4fN>E4f<&q?gs;fV9T4u(<QIJ3E4~W2fs-2}V-OI&y!M_6HHmh7cu(r(c2@&Yk@xBGFu2^xekaTWw5a6y;#qFvFdlLA)%w8NDQqN~^;hk7Q4+K;RVu$=@6B>+c2s;SK)0GPc9EQp&loO%TJOhA4v<l`MPKt?$P8uHwl2De<8sBx<y&soWr#X#X%e4QDcs~MP{$oj<QwGXr=^M`{SseWjm6KHY6nBgFKFqiJ~;%6a<9%*jBohCey_XFoEO_iz=;3oM)iA)8Gd;fzcnV~BQ}L$o~_eP8j$oYCP3a)f-rsyGsu*3rqiP8)fcF#=gm3V(2II)Q;0%Q0+y4VPE{DM&NjzF3r6rj^HP$5yrTqibwFgRB0R1@$FRH>1N9qwBbGv`rJ^aCf7kmC-=jfuipSBg6u15TPS61~aBaYw-4Prj*JmPofD{@U68LcjjN~8kBju#C%QXlBhn0MBialPcUz#yBW?Zt~=)J&lw9tsUc^N|mtb^%JYFY2W13!k1KuU?j7XyhzV?#G2;gCs%Zm(iwj2p|VujV<~EGevXYPk5X%8w(_gPL&n0zAHVckP}#);*Xh;mO58ylqV`ZC6wO%L;v|`lL#+2)>teaL-;>%7T}VR@5yN4cl1*?w&|^WMNODQYk6`wk+DhUoiQzwI>y^eSjxVVg6+F6hOjKO~fDDG!y+o_*|8~{-o#Pw>P{qe4;KQ`Rs__H^q`xDV!w#&udFw$=tAjCBbBb*su`G&DDR``?6N}$lV8pjuG?3g8gnwn?q5ax?f513pn?ikJAp6*kZI>*f2U9?6tbKV+);v3=^Pg8o^U7gfQvB+7ix0>uMtUS+6C%1>y`)#GbjVDTP2;2^d2UGZgpB%09^<jQhHd36ni+*;nCJ30>vS#`J`Ezq_UQN}^0W=SHy4EKXmYhp=7HdK1bmv;|9;Q8V<wMd5zludwR;V^ifZFDnK1|BsOseGV<1ZoS^USrD;zk69GPQ<de(rRT?S?fK&1Hjpk}v9)Dc3>W4CDnMI6gAuD@RF1i5&4-V#f#5v#(Co$06y`*OkgGr#-JIW2n9Nf3azCs+jI<^h$>jKVhO*8_6s)dcRHzwS&VnERy_r3KW!w1!5wA^R+GRy?bQR1RTM7ce#4@k{4qt^hoam%JU)|NXu91Gz>+Tiyo1p&pQ;C)O*w??j>_q}RpBFq0rX>7FujB@8=e9f}5yJ<ib39$#8aRTX7{~8a{GqsbsGzDVbj&pp_A=Sr>|!L&*+cx0?M(g(%c0g1R%n2?$@m$~oJ;xH#bzP(pzwm@-WzsL-btFU##Pqj?H$yQP@j;Dt=dnyc7!OYUVL=aD!I3HbP;3oT43ai-YjpXM06`E`_IzF-F~}3_&~=M&c&26Z~vL;DtfZ{Vyb<v<`v=2Ee(mkIQb!9#&fwUOgV$yr4Y#j1w9)O@?nZZQIAwP{D9kE1fGCpZ;EMk0{N|rQJIXpBkd>L>U>(1^f{=xv7Qcga!kF2$<qEs$ceg`(=Upg(?oro2zNYX#X-&_R2^DKi(T2Q?1$jEi}qg&6>8=KhH~AiaAbPKX$;G7&OpN%z(X--EjXO9%h#-%`8dP&P<8-Y&;I2`f%XnQg!$NKLlK!Uk&1NSS?X$iFhfNf^Y!B82w~(EGFYZ&t;R^5Vnh7Lui5tdS*!kJaRLDXmj_^r<JxvJ(H?PZQ%ljiJJicxuFdcIrVQ-h8gXU$;qAUTI5e$EGBqH|MH3@7xXs;}uj(Q6Q!-LALCew~H&fvmjBq%#Z(&bgxRHC~M3W|p4LC^ykvI=NhB^l#vD*ZGBbOTbJGBHmFKc#(|BNpC;~J7;(Nz#`mZe~g91kVvIV3(9DV^fUAc0(kZLTRAH2~4bh9T(<gN8np0cb-7##&}=Is#?9;^`dOp_?=;*u3fan|oksbR6eIqjAeOODU^0SOO|8+%(+yF2%c)Y~?zUth_*-nt2T?-d6Pv`LNjdbU*{W-LBXIAmr2TVHI^a@Ym})qhUoy96tDS+n}}qgM?T_=^eENcB|SH1cT4Pp~UN)vzib8)kpKY9;nheP0zBfQuvSjnS5ftZXJ}OLuTy51(PGM{4Om2t)TR8pdqX3hyz$6XCbXQ4GM4n5v_Qe%K8H1-3?57*vqU`u3)H<qieUWU&Ej0AJc`KE5kR-NCj4;7nt40(mioh8kM0#vVyf&=zzR(9})dBS336j*bE}WE0_b^tEz8ca^lr{!LbMi2vTH!euD1SeJkl7rkijwuODZuT`XJoMO-*p*G+B+?#O#)AURV%j7Ivls1B+(B45$Q|IuT!<R!>}wuZJ=DtZ0-odWL3csa!;yc=<#YVQe2NF|jlU`ZUL12tF_BhQ0%23OZgwIlaJ)#I;uk_$?{8x@Sfu@`xBidLQZoNJI#Q7{0!$dD}BxSgIH%D`Sp40N1;>l2_ht7P9bn?B$CGLB>5S-;!b20&7GKuz~;F}Pcpo$;`wb_DiVo(gZ(E3uc49-fN$$?`2{el5T!DN@CD1_=Yn<UlraHt!7dc*tKX^|3inL~96>-ksOFa}CBbdtv}0NwK7)mx-v!_$Km-<)%V)yh>6OWLuLarb&PkRZoB14*Vn-1YsCu*l<x%bM_^@E72tcV$r3B%8Ud_Qi{e@XMGK7Bh5myhtdDhilTDzv`23=fLATqpB=-w>6}2|YDWxK8iH788e&WiNtZ>w1vRSN)DMe0i>2=D$TF56DX=*VeN;%4Z3bw5+MnnKj_#NST>>6ei@y{e%MZcGv5~3A8tdc(xBfqYbd^-`p9+8{U%}B`QWb)?c3zVHYra&?Q0B%UiXRlv0f|lliRA6}ix&wQg<#wZil3Fb!IG-FbL2yj*dcI-F2w<C?n!LV1&!H|f`2PzT3J0>JFPlPRwe`|ZSSQ`x`-Fti3D9;=w|;^Yi7g0O|+}R0h4YTxc9Z%a*BNW>D<~EY~*tpq)rEv937SIVSiU>ZOAANUcK*Pg&Xr;7i(skvhJFD1*{nPxe3USCE{Q2phxq9xK<frYckg`X;uvNIxrdV^1`mO!+Jwh0V5(bTKc@A^)uO}aQzZRk!;l3q5$ZDV|3DoQQQ}dcHRRv8OEl8jc7nfm|G~~r6ii&Kn<FS!w7k^AzD2lN5lFSy)ElLHgFW7dlyD}{&|&gW^@DQN25KRrYF%gr%C6cZvCde#L~tjOLGFPPiT;dzTt)&MUGRmGR$yc{qm!(3M8uW9Z{Fhq67<c<#*nckFNeFML+9<75;>EO=?PoI+F{3uk8*kA}2DeJ|x^HwO+O04DjHOn*dQrk_pg+0E6g8ONMQh>x$A=9FK#K%$9J?w%kxFK~4L+ojeTk21n8|WtdQ8Iny5P8wsjt(c`Tkp6AtH)(tG9pN`w9AYn&6(SlO5F2=VthJ)2Q&QNtMZr1aUmvd657wnbL;J{t)nnyw4rHW+P3OTk6QTzxO$tSa<p%)psTSiFhkxmz?HDNQb`qKs`dOsOdEv^3*OL3Cq0#kMOfu3f9*=z_Hb1N5|ym8#QF-T2{X##I&pO#|hG2<vDYQKi?ZR7bi3g*$GtO;q4w%?6;o7-Gt%c0eqeQ05`iHbS`#f1Z6T;fS7J$+LE=p9li%PgQSTaGMD0^n>^&Mg0YF_<SuZ2utD3X&8SmiG800)9Edyi7Ateb?cHPgZ{gSvpP?5GxsOc<_{|nE434_6@>D8KDZU@xyI7^h?lOrl89-)ZT$%3xefxMXHH1$*cr0>k_qiT3BuuKP-o`GU{!5_AcVe10hT{ZJoED8Ny<RI*Kd=h;7qFf2mlE7KLC~$`y-Uq7_`+E1KFX%U(iniObqKKdU&C8~X|P(JmEOSETkx4LIi$cI=rOXlqEz9J<{&CN#dkgcWF7EXC%b2ige`?)~UHv)0MOBdJ~G5)f>}?XYn%Pfz!8`qt1_+X+>KqhW$>u<=OWkA>7Z5N}&?;Yr@L-@Xk-iB4`I=C)6Xr%JvH8FGt=i8eY1-!o(j@zK^SwP3I(hiA-%=dvP~*?XP5SAeVXWxROS#`E(}=WqY$s+V*V*0dsrww7##U4GBqeio#f1DhHq=*4k&7W?h%DIz2=E7B@VSJDlH!{)^y>V~;96_kVj^FBpwB7aFUK>@@YM$^cgP(3K0WYt=tX@>uYcYyF%0LC;@=m~P+J~8@w7%1sWhUE<?C^g_G%dKg+j~*?|KU9$x5*FA1A@yYNI-}sT?7*vDLe%#vPe%92*G5t<^`BNKeXSGh(L-iRNqG6y8utDuMTOu%9|LA<$$wka_OU>5nSW@u9CO-Ult9~?)_nI?mek9}&+_x@;ZMsovf-+kS3c;TSzRV5hSLdd6g7vzvrjk^*iB1QXs`OQRPyT7h~Q;r_jgR7)Cri<#!w(=$C}(oW^g?-9R7o-?z&XH!twx(3S5UCA@wKqAW{K>CZ7<1@-A<2sPmr!Q414^Z##x8XkU~ka-s4~D><-kjd9W&-HX3Zi10J<8nrMM_MNnB4Xlop<VBeb@uS?a_x84Oy6^G|BJZsD#T6)PYE9^}U{Gpj$A@>+#w)ZaQD2bXd$$&^8SG;?B}q$FXyxv6DZSDzk^{*9Pcew^k8EQ^a!)d~5eP7cLYd;mS)q(?7z0)-s2A+#SvfZPY$Q`4jd*fCDj{bgc1u8l6T5Z}q~Cx<sb3wocG-ttuSpEx8`NMxfPR)@`%YuA;vb0wg;9WSJgAd&0>?-6B9#T-SC2XBEfbo}ldQ2cQS-DmWb$XYq2oPqEh$Hd*YwIT@OET4&E;4{`YDKFuSLBSh`_pw>KSn=G+Yb{ok)vCilc$dR<f(0gcmUA9%Gs%Wpy(~0XXW(M-08q95xYPZUJU3de;f2eyfo{?Uv61??$JsM9$D8dHP3eCNd9ztSZ>~EBwU?=QE%s68S}9)$w%wt!m??KIdT#D}x4U+(Ku5;(=kvs@npZVLJW`jww46>Ur3JYHqspdEH}jpoYQV^4XJb{F+nXsQQE6Sg@=JJ#iWhz%owt25(#=2LS7V124jA2@@o5pdShumd%$wONbs>ZaIhmd|C^<I!G?dAH+|4y=Hou)517|bj0sR@R9!eH=9eYcoYuHRPODg=S6`6XD^$=Ac%m-8e>(Pe<!ov^*a)BzYP_@50Q#NR-kRRQ-(r;Cf)x}ZOS9AIPy`M%^VxqOLY@hG-qbtVRN2Aw1R)W;yp0mVQ;X_p(b6^(pck|A)Fr*n6K}1UE+KD2z_5t$?ZvK6j7OkA`fmC9EmQX<<x#_-JM*J_fbxh;YE4!oAFp403jN?qizvHtJpE{QU@JV^{x2+x)A)lR30WVy!!`T#J346!2Q>VT?Ji`ak>&d6UmLP_4a+CPup0EgSJ<K#`hA{mK8M$6}2OyWr8Xg==y~?duj%bQd2*o{OS3*Zs8fKUZ6qX#}~SXDrcPj=5Ub+u6e7sekZ{D`BY-N-%m<l(RHijfQU&msbn*yDMiAPxz*Zjge&Bf>j7S)@P5{p*~)sHWBPyWGQ!UKnWa-nGtb~QF|^N)R-BP0i2y$Z#%r4C1zL*ru$b*v13olOCX?P+B5A_5rpB%x4_bGn8&Tg4I+;6gOafGz=n(!J6BJs1+;E0<ohnWv-5M~SFl+4{Q(2@BH}@wBVUg(=Ip=GR1a6dJ_X|5&c4*$*Z@gPX9Gs`Hk(AZbqMs4YF(9@hi2;-0Qc`m&Vo@bmePkT8#*|7y$Z0B$ij}`w#=~qwsBS>)Z+Q>5qZp#w`N={4#nUU+u=Z=57F)&(9UKrh{cyh54$XX1KYN^N8*h>g?p)kM^Y47M+fxlNW5gGv3|NcSv97OK{o_NB@QXS~)=#Qkk%oXB^A|vu(^$y3j7YF`EjQH1Ds|XMUqHC}D?Z-;#x8eGpMreR_=olNY_Y=8>|j6l@7(pJl=)=xm1*}*yK6{+Xqb;|0Hup&#{ixQA(?x|6b59@9hx~sg!%wiEhR<ap*Rr>g~>%_eoFu!6lWBannefTaq!8vMw7g)UPVTnDG2TaVsau50h^X&wHocL<j+a%JPDu-Z%)=sog^l5={$xh3TF30+8=9$#TnWn06GBlnz9_piNT9#CvZ5b)PMn0kugj{CxvlTVFfQP(0@Uk(`sN4`W$n3ei;pYSjD-Yl@sj^m${nK<(p?B(Fh&5_qknB9APPB%%&dCPgX1YIBrLj+}?E%gB@|U-Bkj9=f$pr&1u81b!zeZ6CUN-AS`tefUrwZ1D4BtGvhAe;>0+)5(`bJAl6y#fV9Bol()sY{)(jK8KFAUK>*Z*yak@^q*8dPUPU57H1#Y%K!sqg?JC-)zV3KaWA`>7K$u_kv*8U*dKDq4*j^{9vAS0M4(C+!7jq<kVC617e1_7gNm+dS<ogUPj>64!Or5UU0~^C?GZcx4iXWMu%WUh^+Ubv+x0TosX!^@GMRU@N|7@7ho!SiJVhx<d6!Y2WB&W&j_$I>7R$U-oX~$Z+Fij7&d12^?$7U18*68VR`DyI*ppL{7EH%M2z4OYtHpJktnN<iW7LZwt|I$5Q3smNWF&qg>;u0kP13K?Ur%zkd%_%T|^a$Ev*{@sT66<FEYH#cp{Z*a47;rfnlc3Ex3blwnlN>vTlI~cg6T*eVpQV$%x+&ha5g3CC?M(H*>Ffo=p|~t^{C2wGoPU2u@HX+5Nox6?f!xZkV988U<I#S^2(uE&1^bK$A^eo$G>r*`+)1WFu-La|7RMX~>1~A>E#*99R~kKB#*#ls+;TjnD8C$eXUMM8C80OMo!AR9x2}DDVG1=bCF|}{W#en=N2EVDDT?kL>xx%XnqtIVau=7K+U*>p!5Rz_Itk6*4V7#;q{gz0ghmy!;Ymy*?*%2LfpR~n+}V6<t_bQQLdc0|I9E8$%8ism<{k<sWQE_d_ZM!M!q&(NY5R@;MrzUsk}4TYt;vvfqff_kL_!2;K19=>7Q1sMNK<IMtwOkxIg+XSy~|0~sr^AWUx!SyzEzdaN|RXZ?YxD(Dm-S>{a2eIeVceLa*f^r%o5?8^x!juVk7^uDqygDTKryJx((q0B2Q7xpRR{j*cg*>Le#k+grSNGgC3wC0XEPWXF6wCjWDE6AKYlc&icyb6~XXN5wSS`kU9h~y4Sk8=|qM&zoH!_izs6XOQrHoYT<&>Xf#wfZTk-eWkgPNdd?*)!drp~(noHP9KCCSo<dvW#^j%X)oM|P7b&B-D_}WqTz}bFWXvfsVnoDx?`llCNd<piA4){eR>C0@$W@Dji!MrlOE~w>Kmg`=&^Q$W`HgZ&OF(z|BA<6rO>S8BS4)_IE#(0(h8nv(F{T^;qQJ?XKEwdYn2v*yN`O+3mAuwe%$F~UTWk#n!M?H3^Q|Q*&IcZH@*BY<_Z@9b-)I&8Usjv0>i+bMz%k3IZS<)K*tRKadBTPgZbD>GVUgNrcExfffBL@Rb@<OS!w^-)0a+rrGFz-g_*-dfklF>tG#11lJFggtN%xl7#>qRk5epdZMbSXr6<hf0PN*K+5=vBFBY0_SlwdpJS@GA)y@^*GwlMv)`FBetZ^19&Fm_@v3a#B_bUZxo`oS%LP+8#0gnefo9IoI%R?i0)y&Fc#Z-+7DV)M{L0ZtDsw@G+p7%*>XpbiGZJ8Nv#;RS+`Hwv*x3Y{4VqqA{AJsREQMaGtxaaP@C#gl6QfADnp<HB{s6!rMJHafFi)x}e6BDlS|RQj`ZnDY0~R>Exi5YNjs|9P|FmIF#f`EFeUtCUghxVKUxnj7W)*`WC7WJUY8uJb?s+Q=x^mLh^oKk4kZC=F@^J|XODnP~JuC@|%LfwLji2Ejo2xDxC4w9u6Yyje@-wdftm6aA3?*iehG_0lG}O6E%;DJAPV>`DiduKqNJ-^FLR`stC-9O%qcMHd<g4vRC|1PIR#eoAUv-^38HxrBNLX6ZQY#jl6x3H<NS_ssX?t3;*<xiosmp0bS19;XO+KQf93#yCIC$2a8`0vc(_-eKT-Hv!S6rV%=v;vTL=887{_;5i2>`;b55Wk!jn=&|PFU9<?TG3kR7<7dERE2PK8-VMR16#)zRe4{1lV~gG|9`N~!ZmkcZ-bcijNwTaUhSMp{cUinh*V4syk`M!H_67w{hCE+`3}-gzxzZ-o+GjrMZq8PhAr_08YwWYhRQAR}wauwBf*vCs?%Gv*kl*?P<;;h#XEkwA!z8U-I>g#N!8jgP!+nj%1t9P!ug+Kf>+Jw*&~BfS$2mQ=b8l2q5VEGH4D3Lw-c)7cw0T$lS-#}>mk3lwY;@W~0r{ecE!PEIaY(&gbgw|xtDMEoMV(7^-r3~Of9udbmgZmgo+@)^@}L!g0FAHtlierF9FOYbbpO(^1a(!*H!&#EZ65;sLf(bAi5Kl}@Y5N+rYZ0M0a^1iEoSiUNg67@Xz~x*ht9+`=-{3XDi7PCfEpYjIHLOGq%SfAM}SU&3)`a^)^yM~-H{;sV`!J1mQMG5Wc+8L6#YXa5dr<PwLq&du7L`?4B?AoH1ZtcLkEIQv1bh$t{gll-t`l`A(O4gS5r1&Kd#*@(&jx9r8;%mS#iFyhWR8{&;-dXR2h!dYqUv&MXf|^6?5xX9QNA0s_B+rH?5V|^B}5O7qKu%seTgR{}a||9#%>=YFTdC$Si!Xx<}1AsT0Hq`7;<LWQ{b{w1foa?)^|R<@nrPa+OZ8064K|VrrrV;l0Rz=`9o14HVToQS109RMq83e~*IQJsxbuL&PmJgF?ly6V|H@s;oQ*%u>v4vA+21dgZ{-&Bw=ma+cth|H*}68(jpb&Q8mgVY|Jfu1z3S>Zu6m$dWmR<QOpA<>Zlm{+BDS*62?l<X(ia4Vu&3fXbxHY3&BfBpo(HBJ`2a2a#Q@G=8j!H-PvFYEi&MJP2aOG!XAP(b(ZGw6L5~j*>uBwlX8I9B18?FEi7{VkETFRhhOAFw0L6hs{Zuky&w@d&j;Ta~;J90}3r5E&&Y(9x(C&$Ngnphp$K)>e)c?$2Y0S_>5s@*%e@6Ycy^;uyuiopogo!I}DCi-z$2%eN(odm8K%JlTJpC-z{hT2u{M$z;Fq+ppgi_E>@bJz}PNik%Jnr#``=Ly+itRGh6inv2S**)wED?9gj<qlgsaY{{~_qv38iYYyoyn<yZ@6n8jdF^tiO@kQP-6Omb_e2hp~op_PBKcVWwH_SwTwI6CbQ+s|YY3tM8)$g#@g;C8P_dhg2Ga$sz*i^ne%C`EF!OyP(SFU5)h?hqc0_2S?iO1UVZrzYOg(nOIX!6a_ns2KHMllV1{SeVGAKxk;j0+Af%(;xEzh1gqzgQZCwQv~#dHb@gexinIqA|!=YdQN+N<yeTeW8r(|skh(#s;3FN{{IKUChwYfTsojT9`y8DHX`^N0<zmOPptNrk3xp8naa*jGipW><$N8@f$c3aDhI#W)G!${06_H}xgYo;bYoGeKvlCb96J5=6q;W#1s1_<{I}IPFgAd;5c7&UyCucocFpoEjJbbqS|Vm29k?9T%uC}`aca@&Pa7T1ded`__$jq2qyg9U=X7$B6@DhK&J6H8O+8h~`EG7cB8MB|JKM=1ayC}rht(X#q0YjL^vQ4Iag6&fJ<R0g#hTK=LRWUUd4NQxU&mh_FzZwGPw;y<Xt+@NGJ;AYP0KHs;#bLYMbE3LQT4~Us9p$w7;`)%YK<rduK-~AAREa9Ax3PK+%`ux<0S?ZwnZBa1@KImEZYrI53*PJLL^hyP6DdQ#y^6j-kbu=mWv!gDG*l<Pq~jNkqcIoSpWkuobQD>$Y#o?SmFMyeDt{>`HaX_!RDNqC%=o^Ou}PP8{nrKDer7BL(-{m_1!pR)@^qn&kCH2nox%~4jk&qqcBoeYOP1F6obHX`mHWIKA8yB;m8jM*|)a4ma+yKR$Q%%=uNpx6kk|JVHEV0?WqgeaQQ2P3lVcduO@RjtYH|~=t!nX+5AXZ-6t!5oiq{<R@)|@pTz&r93gp9!cW0hZoXgVa&}g9lO5{h$Jb{YC{0!ioho3`M&N(OrcJ|2&e6##13Ei~!A)f4!Peh-sq(FISiGH3Vs9$HB^;nMcEUM$ZQ?i#Su~xOsyv$ygzFYXFGDj8(E_cshW#9??_uEjI3twnwf`YzuGUZUGMWBB#?zpDWuDFFTMi5YGga=)LYB@`q>|G*Mtw5Lfe(Tzo;aZ|X<{i7q4R!kn9lRP+a;u=WVW384KXHkhDG;4i`$dyv8ufX^_>a;(>h^ZbmXXM7f=q{udeS1pAa=R;|ejV#Q<Jdv?MaH$`Mr$2uc07Kl})va(8I5=AJvdv?_y?sElhxWM`g30Jd%}CNQ>b%2dri{zUQA8ry`edYlcL4M@k_Y53fyi{z0MOmM`MbSw9HM4tU9I%(eh$(Og3GYtK-gRV9b;RvUB87x>QE%YX$p#A`=mMUus5m*J;-Y<G9CVHG#TBr7k@?%rr7RMKni&Mkjw6f<C{1@e5wEgv{&DBaRXZJam3XU93Lr6%0L&<ERX%cC9(M|VD>Rkg$A2U^U&qs(<u;Z6ozL-`&JTMjnq-bFD7)Xd1GH*Z~AUF$CCbo?fP^A~1ipp7pgo+7SkiA1}rrffE4DJC-FNE*3aaycFY9q{dZnL1~mmVH&;L=8q^kof3hA^r*Jx7szm;VEAfE&=1s!;14u_*JlHaZg`>j@yy93y?TgQW1gjEL-Y73l{;2(G7x4B$Vczf$NEG=6_iqc2Z^oyP=;=wGSuFd}*y*}8GD-(^M&`^<8}JcCCG1JG&&MzMTVtrqTa<k;b-iL3<hExi&3&<J8%_HpK3<?4peC0W*Wly>&1Qd^PhoRp$oBSXV5^#7Lf@O%}kcL{0~OZOh?m12TuuhaY}$Xt7Jg9Xg9FBgBFg_;P1$vh--dKKiL%L7F#<~3h(6$!$bogWE2AP#dHHw5{%YG5DTpTn!v6Xi^g;YOy#=6xD-@fqJn9(+#ip>3)i14N^@TM-f|xT?G^U5y*UJUJ?ddGD2pZ(SNC0T!)P+A&_IpU$m)#8MCiI}%bD*V`$}=kj~P!&ZOgaZjnd88$x=Mm!mY^$3edEqLxv&2P@*HZdBuigIgCMn}le8Y+JOo2f&zwdpsU#`d84>S-u6w&5Wy{I6wSw#}GJE$ld&8FKH;h+VXKSfM-{LUaluPT9G@BkCI8<LyLT@eHDJK)RQ&kW1GjXCW#V{>eAyC@1t)pJ?Q6-!6UX8qIZXME?JMW(u^0IycKm{4N7I<Gkig{TgWl=3VuvijH8%l;r8hYA{z#xw?}g=S<HMxmfgz2i|z6sS&c_XqY+~1Re0O>JqN-)x_)`3l2_cEIn?+!GyyPBRexQc-Eg^coxePfGa*ak=s!Inr}R)-Hyme{p{41^H3mOddryaaO{Edu8S?U4Pw)HI}98#CQpxxP_xH2vev>Gj>H+Ldl~&KO_pe2lX+ABY6!(1BoI#ms*SOlCt7aHfX+hEV>%$El$E8i-j4d!&*QP4No?g8b5Sf&zU<Kovgkxaw|M)Cm6QT%6g`jD2Eqbo`N3ta?&WpH14p@X%NUUz^Wv!p1OF?8+{@Qd?=vTl-@S>v+V&zzhVG6k4^YmXU2Z&-=Ei^c-(G;|^<76TdDx8vG|Ei+8^vhMon)G7<92~EaE74KwVg_i%7`Xu@sSG^vI1f-Fm2htuOj0D^Zugf#&#;}0wb7NM8wTf(`tUYNiaL+*Pl)Yy1N@6Pm8V1dSIF71&A*u53@=xwDE2|(eO54632oBNOJzR3BeIpy6D&06Ok>a{ho%QvQI@jpiQ}*_lTGwX0137={JKe!r`ojvQ63de6Lio0&#7lJK0yORuCheen4W9#C6mZAqd6R;y&1-_h&Ib=WX(1be3<N$54H>-;|M(=Thd?joY&Gv>>+coi%U#IImh`$i2Li*?}bci>yL!6cX#4$GCw^5W26G$2F+B8K$iy9mEnv9+?Y{vxeSoTUcuGTRopyXbDdWc~oTN%A9qgDz|~_O%TLt?a{=9?AA||iqCwYPP01DfOp+gR6N`gKMd@#kk9p>y`Pkyd_c>z49QN$`N8$PUon8_`qxs<lC(DSUVZ^qJ#(_|1TB3+5Wj1oyN6DnED@70U`MHDFUV0B4za1Nk5}g23uGCDUpe(x5x|p7H;pZ=!f0M`^|A7r{Qg1Gca>$6R@(%_Gclrq%pYEPb?jk(7oQG#ci)}IEIrv{<(L_3xTHCO780rG{9f<QtQG3F&u+jqp!+0`LPSCufCqAIdMeHT-(Np$(6d_r2yed5g^D-yfSnHNhZ{StLSnTC&QzccdVgRXz^5s~!@;9^sIJp$sjTo8dC4RKZkYiZdHpoG+aMfKLuxSh6tUlSUg9^`ZZ`Rx1_y#?kmH=!eQ@*8$^*+LhqvH+MK!C7@w0W%dNQw;-yI(Kv#r@3A6!|k#gvF%HdqKI??6mA>P~{mdMlYS$H%1~y7l3?*l;~U=uB}Kn6n-<$jff(A}Z_Mr?<Ek`mN+5ZC^-}9^}z$%cA0^ua_YNq~EO1L6`o9v2f#$_3dB!aQubvI;@G=23-6-ov6XG6S7%Xp8XnT)sc5JEc`T0f5<8;$z01@QS)+A-~<-m6y?BqW@Yl`$?O@s2PD{9#-&;4ym!=xv{}Sz(+&yVKj*68mZEukE6Ngti}W(u4%hgOG8~?t1XHlo6|X<x_o=__uI*!gN)CI18KI3#0rZIrq(9d}nOU^ILhyjKQ7az<v3J*iPJxP_VFXo}bH1SIV~D4;V%46oV0U(?&yWzTG(VA@UvSp+zNaG3O-sp=&GmK$nD+gM**~9WC<3m-!FIeykzJ2Gv+LNpskx(ZauZ?kL(&i=BXonYLh$PY(#TbdIx&qFAL~$^qgrK86@nIs2I^){tUu02<)FEAlt-!O#4Y$;C+iMv3j-2K9#*j~^2hbavwO7}By|MLvNav#wB@s?6yrL1xq@2_?V6iJe~|E@ZJl`%y)DV@N-i5LYLC-sJ1NS2R?rF`=J0vP-4!+oV(Maj^{fP*Ly<-9IYzz{Xf@=z!|GdqTZ$ITTB&$ZlZlerrqmzV9&{NUHmpjiS~S3Q=h_;m;nJ<~s#XX{thM|8k_7Gtp<}c*5L`n9{wLA+qH;<nE(mfFV7*Z%$38<?r2Oe~(qF0*pcbIY@3P7=9~n~yxnr7NIH-SDJ3R>70Nib-bFzmF*3UnDRVBctPpW@(e%4^Qm_J`=uo1fde)$|GvUHjQX|Mo@&y1>>ew*=PGW{}ynp+7zFI_P`SoW17O1Z$y9Uf{9GZ$3sb8&^aOz`XFO|r~PK^_;fJaH{zQW~8?mb!$IOZ>qC6Wn9o?!3k&Oy+p*A$F_r5O5(^hGbpr0A?6g(f-Ba-#idbEAdGc>we^nKeQOywn=x*_{;i)Wku@|7nl>pvIAbNm3i1%f;d4iBtb)G@q%DT3wt-3TFaHY4obI`I53oO$T-%(xT#o2J7JjRDU|CEQ+1)}HOjx+QZf`_4%p8H8jD+h?Z34R3jmmW-!2z$>>8S$eAYaC5=MJP^j*>Z;sbHyOG0%L+8z33%-WxbS&7P|v#90}d3HuSf5<{E_{Mn7bEaZsev~}efE*0`TP1X#toJ4u3REOU;rqiNl>KR#dADKsqIbUgn2rJ_CoozFTtNjEWKd4hlww>9yhtuRa!v_`ia$lNr+v;UB6PJMAiIs2LmSibjDFU`4f*R4T$F);qvO#7J4%7e$ACsxIZQ$xkvOv(%_imt?{uu$TCty{sJ6^Mi~ywU%&BU5cE(_jt1^xq@9$I9xe_&->XA&@+s8?Q;(mICpigER9HNjQK_V&Banxw@Py%~8eT9x|y#g20CUlOYOf-xL-Mrr3&j<*&KEBvIz6u{~H(qh>0R1f=t1oQ7=wY|;4U8#sWAE68uQL$YX3*$6pGHIz5CdYZCoa1!mWINAQVke4fglIu3p(+w0`qpMG|*JvQeqZlC#7;z9~g^ce|UK8dkIA|Pv=G0>ZOTMQSkgW%*fkq##lk{^F*3z(O8c{Ts+N{9Dmaa=hw}rAG-nm4xj%!C2e%pD@9?0m-n4ug?Ja5+EB6ShhF{uor{{wuUn%&FBOkr4I46ZstA<rnfi&r0O<^+TMej|)R;nD0k{Qnk}o&h|MGfpDdq-y{^K;nlBvrn@&}QKIu!N)Kt$!U@q;Xpm^wd1Eg7If0;GA{YOnMwNu*lUQ%fUjL=0|bERqOS!kdvOa;(6GT;b!V7MK~-$^_v;{IO|J(VIJXZ%RpvW2UV`fOq|sz+K~28#Na2IWHV<ws|756_f}DH&UU8w|f<NYid^0Zn33pz;)SxWHQRac}x=tQWGtPD;9F>0AitopWGTEXaRC+E-|OW*Nl!EZDX3}r06^h?FBx<IhdBMLBcLni*Tem-D*Uoy+Lw`Db?2y{CQx&<=|11G}^XaT8s^27;O(i=IY|Ue(qd{J_6NyId!ZA2r@3DU*LoK!0TJaMEG4_J*6a#aM!yD5e(peQVv1?{+q0Opwto_*QFP7a!&{0KQD|2uQ26Gh!faG0HPwLMSgavTqBcXYJ;Xm0beZ<FNq!3*1W!rB;kVc&4{4EcjHwHIvn>02j87dt4)T8V|RLuhZ_t7R#PHU2OVuR77}w?QES>bp0Nk+cepWW2wb&$Xq>0-SMw!@@=H<$@Am=>{mfsm8?OrLOD(CK&1Uw~U4-jFVlY9Fv{bzxz+wB;B8-S6v`tiY21iVlLBb#=&X`+x0arm2gEK>z?-jmy|6?enxKM`0!#RClfatP?;1WWa(KHI3bMzo^c#h-h=Ncy-o;lS>jTVjTG#k$;463-Dzh>D7MKxbINRwj#<lfP`*qFPvgx|qs{-Qdqg)*g>et8(xaY^-{WIq&F7@a{Yk18u|lZCiI4?O+QvO47i;+od&xH+(uRZ0cc=bW`y&8jjjB^5#`N1n+uN|<(Q=}K=jxsY&aUDWk#-LF05{e>Xa=|}^`Q>BTCuD(7iQElDBvPp9S{!W^p&U3&2R~Z=n8ISu%;ByeCF#FA5#KcVaev)jy>Ez0A>qe5_wK7PhKSgeepT_b~b;|G6{CjfZo&pJ)bGTc$5T+<!CMy)pkT3$$J?|w@VW|^!L5BJoBgv+J#4&mAslLpiD>x?oUsk%FiWUN(7)#=u1W%E{0xEG9fHw6zH0#5xD~vf^J+iPS@+(jE$QhNk2>V8WLHaBJT4k9N(wld!6wN3KQ~T8TtFJf^x_22WMKBCDZ5Eu=DWi2O!B>A1JzW@lfHLS-zhab{cV|nllGNRDOYz~Gi3_6Bpz$)}lEv3VgU%%FKax1<y?{JMXbT-xn`j=>yVogb3(QmW59ql5{`AJL;!wqYUxH4BP-+<KXin)Ur{Ts$^&T4q=^qS^7?zvMG1X1lfF2TFED170%JkD?R3@4tTbT0Lu;ElO|K;ZmUu(jNXE**)tfA0&u?|}KUUy@5v)AgF60&fRHXhxImB2q2<3_NaL5n0Pz+#J?>g%rZ;Jp_>r&Se?{G!4ps<qZ4_wW@5CwBB~)+0P=vn*ma2(qkE5q$H&#Ls`;$+;SnBHzDwIog6s99@fQ^_gxOgE`Q?N){8!7C9N3i^{E|M~P8<?^f*_4|zkxOQ43W+#?1=j2#c|3&ChI^nM)o=%?nA{D_18AN)-i)yyCWyvN4%U{4t0SjA-e$+GI4i>t$|By@mxXrwU$d*%WE__I!~sQ8NKpKL#h%JahXDh>7&pse?f7w+wiM8{sFTui?*v?%3DtG(~;)jrejOqi;63GIaqnkX4`IAyma$oETQnDKiyUBSxt=azJ_`!R$xeyPj(+4wV+C40lh%BzfgB@`F-pU7ip*3s?(Ko$%y)TSV9ctSTV?kw4*ukN7yW(ft*-;XC({KR+8?aZ`6m32s4V><m1%oJh^s0&Vzn58flOJUiiGle{ME%C^)sm;GJkUbaf)*CZt)ytXL6l^NZ#MmMb|Elz%LfDIW_$oL}u30xtW;~`=!_60ANhwK0ASlcPnmux*BV)_h1WZfGSBL-UxT;Ics!6r~EU#)cnmE!`nWrUMZlp^DpbqZr9MmpD2uWPh%(efbai|J+Z?EzbTX@F?qs`?gQM=AFyuBUBdGWxVPqzAZdGTFRaPef<E3s(X3lv!o;WOnU-7mKGTWy5z-cBA|2!3xrbqJ#NlqhWj>HSo#g55iYVZDsmyg)Li>?5^&a=Hy`as;D9D1OtO;ukj%7AR7M(iq>`UH#(TGfrjpR3svMSkPR!Z|Sm%n!)B-ls)0_ka(Ksm0=jyxOy1%^H^m`MTD(en0KQrc#=+!xk*~Tz;P!OkzA>(u3qGdoAV7yECM_!7??3NJRa*N0^Y~@GDX}C6|y)Q6_RLIgDsFaq6DpEQ7l#9VNZfOW2l8>{3G)m{P#Za8kov^-Bq_!_9OLrTN8;(*8PE+!xc?adkWjaEv=z=aq2}^qlh-dwX9_693Un8;EV0(%QiCGnS)=nYD!TiRQ^71SY+C8Z%p*P3tzR=J#eLoNT*qd*IqV7Z4!1i;2y$=kGy%LZAkWNYz1mrag|cvh2iIZ3y2BN!|14rZzAv#i3bB1X&b4C^L%9qEOKxM`c%hi^bW+a+i2QU5{BCGyg^<NFnX2sShXzRU~WuJ`h-)Mz73$e)6{Z0|1B%2UrMtnR5Ui&{qT&$Yo151F@a$a^b5kvK1Lj#`Bs4pgLewc{4BLw&!kCqY=#&RxQ314#S`ivLG$r++VWje5&=(9F!v#>)C(tY)$;2q&xSRt<3jL{z^8V8E?y$W^7o=ud5P47sTf;5eooeK*$8b5*m|l?QTHixCpKGgJV20`0j1!5vRZI?$)1HG)R_j)!=3csufE9CPu*pnoE-spDJ=Xh_)lsEIQPGOrq;CwlJ(<Yz2;kQ;>cbzj-T6IR@r%PRJqqVB8V&PvMM!RTzV$^>!ar$6*kvic`_n-hk+R=ShIk$;YC%4(cCs)f!~=Ry+G5fpyf?Af<S{Lvj>>e@phiPj9vy{W)~#Y9#?!uGZQn;*byNn(&}J$I6=9I=baSJS3oRzgB552PhavBSQOBQi<LsGRY7@90D=p=N<j~{F$p)+F6yr7+^CwweZ1>euP167%AW8!cp?Dx>NlFy3cF^pwo7;YKySnfx}jh?ZHiC-_N0gB08VIMwFU6zWp_anpH16C6;E;>f}4jAI~avZymKQJCG|w{6lm%?^V1!##WO#S@A~{ufAf71Oa@#7T?x0Ag9l414w7%z8FWt2yllC6ZF9Vx%@j!b@3~L7jNiFPl#N_i*&3EJVTupZ6fc1Ld&cX$M|#QGrzed*_NB(}Cd1NTHJUXu2lJA1hrDb_Q4BzaLk_#yXJ+jS*k4NSXK+n0kg1nm*@ZQ{?oAt|VzRsKnIk<&U}0iWa$nyhRxB_1R_O7Ullje&<x!_pm!pzeSeKW_2hb;u9hd4BSwm-VvO`dA4Jf}M8^M@yzCV;j7{W06Bb4M-+HI(1{?-p8gdX(&fm#8=jpo4o!{_l7MWG?%wblMtt8%$^fCv=tfsT61a~+fdGom628Sji|2)YewFm=9+AnXwCaor^xhf44DMyo}``DNRmU9U>$;^Flmjv8uY+a;dtRMPA|GE)~yvl2(Jwg8AySd>arQ^N3_kUKo#RgtbYU2&idLsXHn6C%`i)w3g*0bP(+c*6_N(Qn4nxY<SZnON@r2{4t;CO9(mdg2>MOxSJ9oAbP%UKHH@4a~p_J%_nR&rkklr5RBvBSs?j7mQw`$&I>nLx!0Jhn#7b#&DVB^}`H)J<ZC+%Tq)*qdW?o0Z~fffTE>#@w9>53jaBoDYe~Kvjr_#90*H<&5_ikAH3VSap*AXAo-wnYgp(zf%VxV9zJNCYE}5M*$IhxL*Kq2=Ippaz0(2?<v0DOQGMyMlO_Kg_9LPxGv5;pNS`A6a(nICa$66j2il&L@!#YBGL63sWy>O1Ixtu6!fbRCEx*29E`LT1;_uq6ybO#GB<Md<3lJDTUV0$9h4?q;SjeK$=2g8pN1isfLK8#<+dfl-Y%o9cH$7atheYt^dF$5(C+{y^^+}jaSZ2|tbYP%F>E+=D^eC|`!CkGU8^GfeF;jF4JVY@6g!KE|EtT9QG>R@7=*v@^P>;CZqr(D~LmJEl>@8O>bnNC5+1%FeyiJ|6TDJEYynVC*DZDq;bh0NB?YJqV>NtjNTz~jl{%sW!sMl<LSC*s43ktQzIwL&^03)J_#0Ue#T@~Fy&&TZ`=h%YdvadkBK!_Rx#n*;C&am>MLmK&z;*TyTc5j$fim7)ep{;4qq3VnYpXD3VaNs!+)`VfkX$jcFfcDx_-dH0NLf`XN{piyYkXf5>hYD?kbYQ;*Cw|%O5A1PXCIo#ho!8%S%y}*xv8^GZep?n#d#1C2E_M4?70ie5_nqJYLYG;q>Xj*}EuT3V)coZoR7iWu2`{{AD;*iPkSOO%n}4agHl%cp+H(l#LYMTORDyMov#?E3ef_quodu`UQ0(TmFjy@`FJW23xE%NJj|KNw&lBEhY_u!j0Ox1<1U%9KnQVDB-8W={l$@UGLw9zObM^6?PNDLmZk7GDa3f!6NY0@ix!oP3Mwk?HQRPFh(PcXy;q;ur{2Jx^Wh4hYi|-J=tcH|}&w3xubM6Zrxnqz{5;)6YUj>a+w}1t9w3k3WEUd}NH5d@G^$MuTov;?ub{4fmbOTg<gLf*}c>4Lp&C>}CDQMLhDo?ZS$X=fR0vp-Uo2dWO9FYlVz#8fs?yh8+Hwdl<a%9Dxs@>s#D7ramK_Ot)t9+AMiMQS)5@rJ)TriWQtsZ+ZQzU)<H2R+e1N!<!*~7rw69(uApY7Ubl4F-}I4yD<%X4Jaf2+O7=4Qo%dl0<8Wg&|$cKr4dSci5)1@%B~jPWwSa?iyhDAmfXX?=u^?^)D>s`zb)zVYOa5Fl~oiT(nNc}2eIbFd}UJn0pJHM=+00}o{7Y-_-zo|)jMgH7G@dJLH+Ctze61X1_2Nhl}Mz*`;AX$_4Q%mpf>0@Frd;E3o`JZD}ElW=f%`tU62*2Pg5*F63A?Jx{DCPMdrK>Sa{8!uH@AFWa-(|qaQE9IlkQETl05xK-_DQcEu1NN5hB}ZV$1{lOiYc-YRI`O_oXwQe2%{R@K2wAH!zKY9H<N<nUE!=xqY7`<9G?V=TcLnVp&-yvKj{$_Sn|PUNqy>jV78KH)dmhOkxn?L5R>`(S#>`#%Bz_WTu`Hq$Npf<I?H(=egDQB7*j#a#4jBzHY;RcC9Bdk--I{t0g;U&@u*Wju758RDfI5G8z#rTh2^rbuttGDDWo9xJ7h**wXB)PKb$cF6=?jO>wFb25&S#8({u#^NXq7)q2%DQrUe1lnA}TDq1{9wPI*P2ap~U$a&<dW4**GcP;IrJdr|P$AN6KH~>StgzLGwW<n99ylss3YB(J&plE;mlSZHc552)LO4;V_MVI4S*ZNct)mY7k#o2OAk!llu~%a%EG+YdRr0v8ocWE#?&P7P0t@z=BdkQaL@#Bi14hu2QEr+AIfwtvRXPf=HGNoOZHvFI({PcF^)TCKs-?f<-^9?qGvJOVOT&3jq_J?r)@4x`E1f1$xJ$HFmw=;J*A17)RLKL8->vg|w}?*@=ORyk69tY6U}fF5k_F>Af0De6%PPhn^3wARcqMky3bX#TpsrD(ML)PmgQtBI+#5c2%IpY37BfWYIe#wzPxU$xJ2mHj`e{IP72PyDfmF<|i<*V%t)8H%*oX!`pk|FasO@4gCPVXYG)Lwb53+*#7++My<qA_m%LxP~hrLHbET$ydA9Vi4Y2WYYy_%Jnol%a(QF0bng-JK})FZ;+-UdNfT}nL`obkeN9@)nyLaC{tOSkd|fl$YEyz!ee(C&UoGdJ4Bx}`#&Y(L&mK{g@-SnX&*kmo3P5ZZFQoqlBN6P|BnmxN(<+zz3ZdUXh_&#fp+rh!6wQmr4F{G=Z!<_KyX+|^+#Tkpy~Ca82Ic<|f-2+Ap&d=u9N4>{f!;P~DK!O^i&LJk3J~%OwH~O_kf${HE7(IDj+}=t?#*gcy^5^&WDG^flAXoc!f)d6oZg##&Za-MfWI)!yIW$5_hUZBRFq|;bq{T^KfG`Oi;lB&zX2sM-U`(0Nf>5g91u!hS2$N;kHqVk1v!}H#!&~q-z363k)p7RK%!KX*ud2zmMbofELgjoyI&QeeJFt04+&yEC5+u0XZw;wQ4oK<;;4r=iuL|{u`u<}Mj$HcYc`EgLcp6M4(TfsA&AOz{CpBRQ63FxmSVdl&KhsD&qs);=f;GkyNbh^YbJ%<+kES^R{UvjNbS7huAxhgd_F<qCcZbH?qQt+t;~tXXByHU4N1we{F_*#?_}cN2ha2U6DGr=ZmZG_QQjcZnVnDQMi(Cdd*UmULyDZv00Fp}0lJn26vf`ovBYQl0ssI200dcD"))) No newline at end of file
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

train_gpt.py is an opaque exec(lzma_decompress(base85(...))) blob. This makes review and reproducibility auditing difficult (and can trip security scanners). If possible, include the actual Python source (even if minified) or at least provide the decompressed source alongside this file so reviewers can inspect the implementation of the claimed optimizer change.

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +4
# Cautious Muon + SP4096 + Depth Recurrence + Parallel Residuals

**val_bpb = 1.1604** (3-seed mean, std = 0.0033)

Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This submission is labeled as a “Record” (folder/PR title), but the reported mean val_bpb = 1.1604 is substantially worse than the current 10min_16mb leaderboard entries (e.g. 1.1228 in the repo README). Consider renaming the PR/folder/README name to avoid implying it’s a new SOTA record if it’s intended as a non-record/ablation submission.

Copilot uses AI. Check for mistakes.
- Fix MuonEq-R ordering description (before NS, not after)
- Add explicit optimizer pipeline steps in README
- Add bytes_total and bytes_code to submission.json
- Fix date to ISO-8601 format
- Clarify this is a non-record submission
- Add decompressed source note for reviewability

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@X-Abhishek-X X-Abhishek-X changed the title Record: Cautious Muon + SP4096 + Depth Recurrence — val_bpb 1.1604 Cautious Muon + SP4096 + Depth Recurrence — val_bpb 1.1604 (non-record) Apr 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants