Agent skill · marketing growth · jeremylongshore
openrouter-load-balancing
Distribute OpenRouter requests across multiple keys and models for high
Why this skill is useful
Adds executable scripts for multi-key rotation and load balancing that enhance request throughput beyond single-key limits.
What it needs
Requires openai, python3, requests installed locally. Requires openrouter account access. About 4k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
OpenRouter Load Balancing Overview A single OpenRouter API key has rate limits (requests/minute and tokens/minute). To scale beyond those limits, distribute requests across multiple keys. OpenRouter also provides server-side load balancing via provider routing and the :nitro variant for low-latency inference. This skill covers multi-key rotation, health-based routing, circuit breakers, and concurrent request patterns. Prerequisites Two or more OpenRouter API keys exported as OPENROUTERKEY1, OPENROUTERKEY2, OPENROUTERKEY3 so the KeyPool has keys to rotate — see the openrouter-install-auth skill for creating and exporting keys OPENROUTERAPIKEY exported for the single-key concurrent-processing pattern Python 3.8+ with the OpenAI SDK and requests (pip install openai requests) — the concurrent example uses AsyncOpenAI from the same package Adequate credits on every key in the pool; per-key quota is visible via GET /api/v1/auth/key Instructions 1. Export your pool keys and build the KeyPool from Multi-Key Round Robin — it round-robins across keys, trips a circuit breaker after 3 consecutive errors, and auto-recovers a key after a 60s cooldown. 2. Send traffic through balancedcompletion(): on RateLimitError it calls pool.markerror(key) and retries with the next healthy key. 3. For batch workloads, use parallelcompletions() from Concurrent Request Processing — an asyncio.Semaphore (maxconcurrent=3-5) caps in-flight requests against a single key. 4. Layer on server-side distribution per Provider-Level Load Balancing: pass extrabody={"provider": {"order": [...], "allowfallbacks": True}} so OpenRouter spreads the same model across Anthropic, AWS Bedrock, and GCP Vertex. 5. Monitor quota per key with checkratelimits() (GET /api/v1/auth/key) from Rate Limit Awareness, and when 429s hit all keys simultaneously, apply the fixes in Error Handling (more keys, request queuing). …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/openrouter-load-balancing--8701e4