Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions records/motif_sb1_rs2_g018/README.md.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
Tim Megna

This submission explores a minimal recurrent motif as a structural generator for sequence modeling. The motif was informed by an earlier geometric intuition: that large effective structures can be closed by a small affine walk rather than explicitly constructed.

In particular, the form |3n−2|{-2,5} is treated as an affine generator governing recurrence and reuse. This perspective motivated a compact shared-block architecture capable of producing extended structure through iteration rather than parameter expansion.

Two runs demonstrate that this motif provides stable path-like dynamics and effective closure under simple directional compositions (e.g., U/R/D/L walks), while remaining efficient in parameter usage.

Result

A minimal recurrent motif:

shared_block_size = 1
recurrence_steps = 2
recurrence_gate_init = 0.18

Final (roundtrip):
- val_loss: 4.7062
- val_bpb: 2.7873

Artifact:
- compressed: 1.92 MB
- raw: 8.47 MB
achieves:

2.787 bpb
improved compression relative to larger motif variants
significantly reduced parameter and compute footprint


This suggests that effective structure can be generated by a compact shared operator, rather than requiring explicit depth or width.

Notes on Evaluation

Logging and validation timing on the local system exhibit dependence on evaluation chunking (k) and print cadence. This affects perceived runtime during validation (e.g., heartbeat intervals), but does not impact correctness of reported loss or bpb.
2 changes: 2 additions & 0 deletions records/motif_sb1_rs2_g018/run.ndjson
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"run_id":"motif_sb1_rs2_g018_primary","author":"Tim Megna","model":"motif_recurrent_gpt","shared_block_size":1,"recurrence_steps":2,"recurrence_gate_init":0.18,"iterations":80,"warmup_steps":3,"train_batch_tokens":8192,"val_batch_size":8192,"final_val_loss":4.70623326,"final_val_bpb":2.78729,"roundtrip_val_loss":4.70623326,"roundtrip_val_bpb":2.78729,"train_time_ms":11352,"roundtrip_eval_time_ms":364934,"compressed_artifact_bytes":1922999,"raw_artifact_bytes":8474030,"code_bytes":60155,"peak_mem_mib":382,"step_avg_ms":141.9,"notes":"minimal motif shell; primary run; consistent roundtrip"}
{"run_id":"motif_sb1_rs2_g018_confirm","author":"Tim Megna","model":"motif_recurrent_gpt","shared_block_size":1,"recurrence_steps":2,"recurrence_gate_init":0.18,"iterations":80,"warmup_steps":3,"train_batch_tokens":8192,"val_batch_size":8192,"final_val_loss":4.70,"final_val_bpb":2.78,"roundtrip_val_loss":4.70,"roundtrip_val_bpb":2.79,"train_time_ms":null,"roundtrip_eval_time_ms":null,"compressed_artifact_bytes":1922999,"raw_artifact_bytes":8474030,"code_bytes":60155,"peak_mem_mib":382,"step_avg_ms":142.0,"notes":"confirming run; values within tight range of primary"}
6 changes: 6 additions & 0 deletions records/motif_sb1_rs2_g018/submission.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"name": "Tim Megna",
"github": "megnat05-tmm",
"val_bpb": 2.78729,
"description": "Minimal recurrent motif (sb1 rs2 g0.18) achieving improved compression via shared affine operator."
}
Loading