Agent skill · NVIDIA
tao-run-on-slurm
Remote SLURM GPU cluster execution over SSH with sbatch/srun, Pyxis/Enroot containers, and Lustre-backed
What it needs
About 6k tokens when loaded.
What this skill does
SLURM Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery). Remote GPU compute platform for clusters managed by SLURM. Jobs are submitted from the TAO service or SDK host to a login node over SSH, staged on a shared filesystem, submitted with sbatch, and executed with srun container support. When to use Use SLURM when the user has access to a managed GPU cluster, shared Lustre storage, and scheduler-owned GPU allocation. Do not use SLURM for local files that exist only on the agent machine; data and outputs must be reachable from the cluster. Preflight + SSH Run all five steps in order before generating any launcher or submitting any job: 1. SSH connectivity — confirm SLURMUSER and SLURMHOSTNAME are set and passwordless SSH to the login host works (ssh -o BatchMode=yes). 2. SDK install — optionally install the TAO SDK wrapper for Job handles + S3 wrapping (nvidia-tao-sdk[slurm], on public PyPI). 3. SLURM account — always resolve SLURMACCOUNT via sacctmgr show associations before generating any scripts. If unset and only one account exists for the user, auto-select it. If multiple exist, list them and require the user to export one. Never submit a job without a verified account — an invalid account causes "Invalid account or account/partition combination" after SQSH conversion has already run. 4. GPU partition — verify at least one partition in SLURMPARTITION exists via sinfo. The packaged default polar,polar3,polar4,grizzly is valid on CS-OCI-ORD but may not exist on other clusters. 5. Enroot credentials — for private nvcr.io images, install ~/.config/enroot/.credentials on the cluster once per (cluster, user). Pyxis/Enroot does not read NGCKEY from the job env; without persistent credentials, auth-gated pulls fail with "Could not process JSON input". …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills NVIDIA/tao-run-on-slurm