Agent skill · nvidia
kermt-pretrain-scratch
Pretrain a fresh KERMT model from scratch on a user-provided corpus. Builds a new vocabulary from the corpus, instantiates the model architecture from defaults, and launches pretrain_ddp.py inside the kermt container (detached for long runs). Unlike kermt-continue-pretrain, no starting checkpoint is loaded — the model is randomly initialized.
What it needs
About 5k tokens when loaded.
What this skill does
kermt-pretrain-scratch Pretrain a brand-new KERMT model from scratch on a user-provided corpus. Useful when you want to retrain a model on a custom chemistry domain rather than extending one of the released checkpoints. Significantly more expensive than kermt-continue-pretrain — no warm start, so the loss curves need to descend from scratch over many epochs. Skill and runtime paths Set SKILLDIR to the absolute path of this installed skill directory. Export KERMTREPO as the absolute path to the KERMT checkout used for model execution. The bundled container helper mounts that checkout at /workspace and this skill at /skill (read-only). Commands inside the container use /skill/scripts/; defaults are bundled in config/. Hardware requirements Same as kermt-continue-pretrain: GPUs: 1–N CUDA-capable. The runner auto-detects via torch.cuda.devicecount(); --gpus 0,2 overrides. Single-GPU fallback: --batchsize 32 --saveinterval 500. Multi-GPU keeps defaults (--batchsize 256 etc.). Note: --gpus N uses torch.cuda indexing, which can differ from nvidia-smi's display order on multi-GPU hosts (PCI bus vs. CUDA enumeration). To target a specific physical GPU, set CUDAVISIBLEDEVICES before invoking, or run python -c "import torch; print([torch.cuda.getdevicename(i) for i in range(torch.cuda.devicecount())])" to confirm which device you're picking. VRAM: the default --batch-size 256 is sized for A100-class hardware (80 GB VRAM). On smaller GPUs, downscale to avoid OOM: GPU class VRAM Suggested --batch-size -------------------------- ------------ -------------------------- L4, T4, V100 16 GB 16–24 GB 32–64 A100 40 GB, L40, A40 40–48 GB 128 A100 80 GB, H100, H200 80 GB 256 (default) These are rough starting points — pass --batch-size N to override. Disk: tens of GB for shards + vocab + checkpoints, scaled by epochs. Wall time: this is the big difference. Pretraining from scratch on an 11M-mol corpus at 100 epochs typically takes days even on a multi-GPU box. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills nvidia/bionemo-kermt-pretrain-scratch