Agent skill · research science · aradotso
openmythos-recurrent-transformer
openmythos-recurrent-transformer — an agent skill for coding agents.
Why this skill is useful
Adds executable scripts and configurations for building and experimenting with advanced recurrent transformer models that are not widely documented.
What it needs
Requires git, pip installed locally. About 6k tokens when loaded. Last updated 2026-07-09. 70 stars on the source repository.
What this skill does
bash git clone https://github.com/The-Swarm-Corporation/OpenMythos.git cd OpenMythos pip install -r requirements.txt Input → Embedding ↓ [Prelude] — N standard transformer layers, run once ↓ [Recurrent Block] — looped T times per forward pass ↑↓ h{t+1} = A·ht + B·e + Transformer(ht, e) ↓ [Coda] — N standard transformer layers, run once ↓ Output Logits python import torch from openmythos.main import OpenMythos, MythosConfig cfg = MythosConfig( vocabsize=32000, dim=512, nheads=8, nkvheads=2, # GQA: 4x fewer KV heads maxseqlen=2048, maxloopiters=8, preludelayers=2, codalayers=2, nexperts=16, nsharedexperts=2, nexpertspertok=2, expertdim=256, lorarank=16, attntype="gqa", ) model = OpenMythos(cfg) print(f"Parameters: {sum(p.numel() for p in model.parameters()):,}") Forward pass with 4 reasoning loops ids = torch.randint(0, cfg.vocabsize, (2, 64)) logits = model(ids, nloops=4) print(f"Logits shape: {logits.shape}") # (2, 64, 32000) python import torch from openmythos.main import OpenMythos, MythosConfig cfg = MythosConfig( vocabsize=32000, dim=512, nheads=8, nkvheads=8, # MLA uses full heads maxseqlen=2048, maxloopiters=16, preludelayers=2, codalayers=2, nexperts=16, nsharedexperts=2, nexpertspertok=2, expertdim=256, lorarank=16, attntype="mla", # MLA-specific compression dims kvlorarank=64, qlorarank=128, qkropeheaddim=32, qknopeheaddim=32, vheaddim=32, ) model = OpenMythos(cfg) ids = torch.randint(0, cfg.vocabsize, (1, 128)) More loops = deeper implicit reasoning logitsshallow = model(ids, nloops=2) logitsdeep = model(ids, nloops=16) print(f"Shallow logits: {logitsshallow.shape}") print(f"Deep logits: {logitsdeep.shape}") python import torch from openmythos.main import OpenMythos, MythosConfig cfg = MythosConfig( vocabsize=50257, dim=768, nheads=12, nkvheads=4, maxseqlen=1024, maxloopiters=12, preludelayers=2, codalayers=2, nexperts=8, nsharedexperts=1, nexpertspertok=2, expertdim=512, lorarank=32, attntype="gqa", ) model = OpenMythos(cfg) model.eval() promptids = torch …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/openmythos-recurrent-transformer