Agent skill · software engineering · jeremylongshore
langchain-performance-tuning
Tune LangChain 1.0 / LangGraph 1.0 Python chains and agents for throughput,\n\
Why this skill is useful
Adds specific performance tuning scripts and patterns for LangChain and LangGraph that enhance throughput and latency management.
What it needs
Requires python, redis installed locally. About 7k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
LangChain Performance Tuning Overview An engineer calls chain.batch(inputs1000) expecting 1000 parallel LLM calls. Actual behavior: Runnable.batch and Runnable.abatch in LangChain 1.0 default to maxconcurrency=1, so the 1000 inputs run sequentially with bookkeeping overhead — sometimes slower than a plain for loop. This is pain-catalog entry P08. The fix is one line: Other silent regressions in the same pain catalog: P48 (invoke inside async def blocks the FastAPI event loop), P22 (InMemoryChatMessageHistory loses every user's chat on restart), P62 (RedisSemanticCache at the default scorethreshold=0.95 returns under 5% hit rate), P59 (async retrievers leak connections on cancellation), P60 (BackgroundTasks fires after the response — wrong for per-token SSE), P01 (streaming token counts are only reliable on the onchatmodelend event). This skill wires a production performance baseline: explicit batch concurrency, async-only code paths, Redis-backed caches tuned on a golden set, persistent chat history with TTL, and TTFT instrumentation from astreamevents(version="v2"). Prerequisites Python 3.11+ with langchain>=1.0,<2, langgraph>=1.0,<2, langchain-openai or langchain-anthropic, langchain-community, langchain-redis or redis>=5. A working LangChain 1.0 chain or LangGraph 1.0 graph that already passes functional tests. Redis 7+ reachable from the app for cache and history (local Docker is fine for dev). A FastAPI / Starlette async endpoint, or an equivalent async entrypoint. Observability: a place to emit metrics (Prometheus, OpenTelemetry, or LangSmith) — needed to measure TTFT, p95, and cache hit rate. Instructions 1. Establish a latency budget and baseline. Pick explicit targets before changing code: TTFT under 1s, p95 total under 5s, throughput over 20 req/s per worker, cost under $X per 1k interactions. Run a 5-minute load test with locust or wrk against the current chain and record p50 / p95 / p99 / TTFT / total cost. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/langchain-performance-tuning--26c201