Agent skill · software engineering · jeremylongshore
langchain-cost-tuning
Control LangChain 1.0 AI spend with accurate streaming token accounting,
Why this skill is useful
Adds executable scripts for cost tuning and budget enforcement in LangChain 1.0 that aren't in public docs.
What it needs
Requires langchain-anthropic, langchain-openai, python, redis-py installed locally. Requires anthropic, openai account access. About 10k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
LangChain Cost Tuning (Python) Overview An engineer shipped a new research agent Tuesday. By Friday the Anthropic bill had grown 6x while traffic grew 1.4x. The cost dashboard — wired to onllmend — showed spend up maybe 2x. Reconciling against the provider console on Monday surfaced two compounding bugs: (1) the agent's ChatOpenAI fallback kept the default maxretries=6, so each logical call billed as up to 7 requests (P30); (2) retry middleware was registered below token accounting, so every retry fired onllmend twice — the aggregator summed both emissions while LangSmith deduped them by generation ID, undercounting the dashboard by ~50% against actual billed rate (P25). The fix took an afternoon: cap retries at 2, tag retries with a stable requestid, and migrate token accounting to AIMessage.usagemetadata read from astreamevents(version="v2"). Finding the bug took a week. This skill is that week compressed into a runbook. Cost tuning for a LangChain 1.0 production app has five levers, each with a sharp failure mode: Token accounting — onllmend lags streams by 5-30s (P01); retries double-count (P25); Anthropic cache savings aggregate per-call, never per-session (P04). Retry discipline — maxretries=6 default on ChatOpenAI (P30); Anthropic 50 RPM tier throttles cached and uncached calls against the same budget (P31). Agent loop caps — createreactagent defaults to recursionlimit=25; vague prompts burn a session's budget before GraphRecursionError surfaces (P10). Caching — InMemoryCache ignores bound tools in the cache key and returns wrong answers (P61); RedisSemanticCache ships with a 0.95 threshold that hits <5% of the time (P62). Model tiering — Running claude-opus-4-5 on intent classification is 30-60x more expensive than claude-haiku-4-5 for a task the cheaper model solves at equal quality. Pin: langchain-core 1.0.x, langchain-anthropic 1.0.x, langchain-openai 1.0.x. Pain-catalog anchors: P01, P04, P10, P23, P25, P30, P31, P61, P62. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/langchain-cost-tuning