Agent skill · software engineering · jeremylongshore
langchain-rate-limits
Rate-limit LangChain 1.0 calls correctly across multi-worker deployments\
Why this skill is useful
Provides specific Redis-backed rate limiting patterns and provider-specific throttle handling that the AI wouldn't reliably generate on its own.
What it needs
Requires python, redis installed locally. About 9k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
LangChain Rate Limits (Python) Overview A team deploys 10 Cloud Run workers. Each worker initializes its ChatAnthropic with InMemoryRateLimiter(requestspersecond=10) — they read the docs, they picked a safe-looking number, they shipped. Thirty seconds later the dashboard lights up with 429s: the cluster is pushing 100 RPS to Anthropic's 50 RPM tier-1 ceiling, not the 10 RPS they configured. The name is the fix — InMemoryRateLimiter is in-process. Each worker has its own counter. Ten workers × 10 RPS = 100 RPS to the provider. This is pain-catalog entry P29 and it lands on every team that scales past one pod. Three more traps wait on the same code path: P07 — .withfallbacks([backup]) defaults exceptionstohandle=(Exception,), which on Python <3.12 swallows KeyboardInterrupt. Ctrl+C during a 429 retry storm silently falls through to the backup chain and keeps billing. P30 — ChatOpenAI and ChatAnthropic default maxretries=6. That is retries, not attempts: 7 total requests per logical call on flaky networks. One .invoke() can bill 7x. P31 — Anthropic's RPM counts cache reads, cache writes, and uncached calls uniformly. Cache-heavy workloads at 50 RPM can 429 on cache writes while the ITPM dashboard shows headroom. This skill covers measuring demand before picking a limit; the InMemoryRateLimiter vs Redis-backed limiter vs asyncio.Semaphore decision tree; the narrow exceptionstohandle whitelist; maxretries=2 math; and the provider-specific limit taxonomy (RPM, ITPM, OTPM, concurrent, cached-vs-uncached). Pin: langchain-core 1.0.x, langchain-anthropic 1.0.x, langchain-openai 1.0.x. Pain-catalog anchors: P07, P08, P29, P30, P31. For .batch(maxconcurrency=...) tuning, see the sibling skill langchain-performance-tuning — this skill is about provider-facing rate caps. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/langchain-rate-limits