Agent skill · research science · wshobson
dataset-curation
Prepare, format, and validate datasets for supervised fine-tuning and preference training. Use when converting raw data into training format, applying chat templates, configuring sequence packing, generating synthetic training data, or writing a dataset card before a run.
Why this skill is useful
Provides specific guidelines for formatting, validating, and packing datasets that are crucial for effective fine-tuning, which the AI may not reliably generate on its own.
What it needs
About 4k tokens when loaded. Last updated 2026-08-05. 38,570 stars on the source repository.
What this skill does
Dataset Curation This skill assumes finetuning-method-selection already routed here — the next step is preparing data, not choosing a method. What follows: format selection by target method, the template/packing mechanics behind the most common silent training failures, rules for mixing in synthetic data without collapse, and the dataset card that closes out Phase 2 before a run starts. Input: raw examples (demonstrations, preference judgments, or task prompts) plus a routing decision from finetuning-method-selection. Output format: a formatted, packed, validated JSONL dataset plus a completed dataset card — the Phase 2 artifact /finetune checks before launching training. Format Selection Method Shape Rows --- --- --- SFT, single-turn Instruct (instruction/response or prompt/completion) ~1,000+ floor SFT, multi-turn Conversation / ChatML messages list ~1,000+ floor DPO / ORPO Preference pair (prompt, chosen, rejected) Method-dependent, see preference-optimization KTO Unpaired (prompt, completion, label) Method-dependent, see preference-optimization GRPO / RLVR Prompt-only (prompt + verifier metadata) Method-dependent, see grpo-rlvr-training ~1,000+ rows is the recommended floor for SFT, not a target. Below it, a handful of low-quality or duplicate examples can dominate the gradient; above it, quality over quantity — a smaller verified, deduplicated set beats a larger noisy one. The ChatML shape, for orientation; the other four formats plus a ShareGPT conversion note live in references/formats-and-templates.md: Chat Templates and Loss Masking Apply the target model's chat template before any concatenation or packing, never after — packing raw text and templating the packed blob afterward corrupts turn boundaries, landing role markers in the wrong place relative to each example. Train on assistant responses only. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills wshobson/dataset-curation