Agent skill · marketing growth · jeremylongshore
openrouter-load-balancing
Distribute OpenRouter requests across multiple keys and models for high
Why this skill is useful
Adds a complete load balancing implementation for OpenRouter API keys, including error handling and concurrent request patterns that aren't in public documentation.
What it needs
Requires python3 installed locally. Requires openrouter account access. About 4k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
OpenRouter Load Balancing Overview A single OpenRouter API key has rate limits (requests/minute and tokens/minute). To scale beyond those limits, distribute requests across multiple keys. OpenRouter also provides server-side load balancing via provider routing and the :nitro variant for low-latency inference. This skill covers multi-key rotation, health-based routing, circuit breakers, and concurrent request patterns. Prerequisites Two or more OpenRouter API keys exported as OPENROUTERKEY1, OPENROUTERKEY2, OPENROUTERKEY3 so the KeyPool has keys to rotate — see the openrouter-install-auth skill for creating and exporting keys OPENROUTERAPIKEY exported for the single-key concurrent-processing pattern Python 3.8+ with the OpenAI SDK and requests (pip install openai requests) — the concurrent example uses AsyncOpenAI from the same package Adequate credits on every key in the pool; per-key quota is visible via GET /api/v1/auth/key Instructions 1. Export your pool keys and build the KeyPool from Multi-Key Round Robin — it round-robins across keys, trips a circuit breaker after 3 consecutive errors, and auto-recovers a key after a 60s cooldown. 2. Send traffic through balancedcompletion(): on RateLimitError it calls pool.markerror(key) and retries with the next healthy key. 3. For batch workloads, use parallelcompletions() from Concurrent Request Processing — an asyncio.Semaphore (maxconcurrent=3-5) caps in-flight requests against a single key. 4. Layer on server-side distribution per Provider-Level Load Balancing: pass extrabody={"provider": {"order": [...], "allowfallbacks": True}} so OpenRouter spreads the same model across Anthropic, AWS Bedrock, and GCP Vertex. 5. Monitor quota per key with checkratelimits() (GET /api/v1/auth/key) from Rate Limit Awareness, and when 429s hit all keys simultaneously, apply the fixes in Error Handling (more keys, request queuing). …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/openrouter-load-balancing