Agent skill · software engineering · borghei
batch-api-orchestrator
This skill should be used when the user asks to "batch LLM requests", "should I use the batch API", "estimate batch vs realtime cost", "design a bulk LLM job", or "process thousands of prompts cheaply".
Why this skill is useful
Adds two executable Python scripts for cost estimation and job planning that automate batch processing of LLM requests.
What it needs
Requires python installed locally. About 3k tokens when loaded. Last updated 2026-07-21. 462 stars on the source repository.
What this skill does
Batch API Orchestrator Category: Engineering Domain: AI Engineering Overview Decide when to run LLM work through an asynchronous batch API versus realtime/streaming, then design the job so it is cheap, idempotent, and resilient to partial failure. Batch APIs typically cost roughly half of realtime in exchange for higher latency (results arrive over minutes to hours, not milliseconds), which makes them ideal for evals, backfills, embeddings, and bulk classification/extraction — and wrong for anything a human is waiting on. This skill is model- and vendor-agnostic: it reasons about the batch pattern, not any one provider's API. Clarify First Before recommending or designing a batch job, confirm these inputs. If any is unknown or vague, ASK — do not assume: [ ] Latency tolerance — is a human waiting (interactive), or can results land in minutes/hours? (sets --latency-tolerance and the batch-vs-realtime verdict) [ ] Volume & token shape — how many requests, and the average input/output tokens each? (sets --requests, --avg-input-tokens, --avg-output-tokens for the cost estimate) [ ] Pricing & discount — your realtime per-token prices and the batch discount your vendor offers (sets --realtime-input-price, --realtime-output-price, --batch-discount; defaults are neutral placeholders, not real prices) Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills borghei/batch-api-orchestrator