Agent skill · nvidia
kermt-add-cmim-pretrain
Convert a grover_base checkpoint (encoder-only or encoder + vocab heads) into a hybrid checkpoint by adding a randomly-initialized cMIM decoder + latent_dist, then continue pretraining on the user's corpus as hybrid (vocab + contrast). Effectively kermt-continue-pretrain with a one-time ckpt-conversion step prepended.
What it needs
About 4k tokens when loaded.
What this skill does
kermt-add-cmim-pretrain Convert a groverbase checkpoint (legacy original-GROVER grover.encoders. or modern kermt.encoders., with or without vocab heads) into a fully-formed hybrid (cMIM + vocab) checkpoint, then continue pretraining on the user's corpus as hybrid. This is a thin wrapper: upgradetohybrid.py produces a new ckpt that classifies as modeltype: hybrid via checkcheckpoint.py, and the rest of the workflow is identical to kermt-continue-pretrain. Status: experimental. This workflow is functional end-to-end but has not been benchmarked against the manuscript's from-scratch hybrid training (which produces the released checkpoint). Use as an experimental alternative to kermt-pretrain-scratch when you want to extend an existing groverbase checkpoint rather than restart from random init. Validate downstream performance on your own benchmark before relying on the upgraded ckpt for production work. Skill and runtime paths Set SKILLDIR to the absolute path of this installed skill directory. Export KERMTREPO as the absolute path to the KERMT checkout used for model execution. The bundled container helper mounts that checkout at /workspace and this skill at /skill (read-only). Commands inside the container use /skill/scripts/; defaults are bundled in config/. Hardware requirements Same as kermt-continue-pretrain (the cMIM decoder adds parameters but not substantially; VRAM headroom should be fine). The upgrade step itself is fast (~5 s) and CPU-only — only the subsequent continue-pretrain consumes GPU. When to invoke User has a groverbase checkpoint (encoder-only or with vocab heads) and wants to extend it into a hybrid (vocab + cMIM contrastive) pretrain. Useful for adding the SMILES-reconstruction contrastive objective to a pretrained encoder without restarting pretraining from scratch (which kermt-pretrain-scratch would do at days-scale). For continuing an existing hybrid or cmim ckpt: use kermt-continue-pretrain directly. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills nvidia/bionemo-kermt-add-cmim-pretrain