Agent skill · software engineering · jeremylongshore
groq-reference-architecture
Implement Groq reference architecture with model routing, streaming
Why this skill is useful
Adds complete TypeScript implementations for Groq architecture components, enabling new integrations and patterns that the AI cannot generate from existing knowledge.
What it needs
Requires groq-sdk, lru-cache installed locally. Requires groq account access. About 4k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
Groq Reference Architecture Overview Production architecture for applications built on Groq's LPU inference API. It covers four concerns that every serious Groq integration needs: routing requests to the right model by latency/capability/cost, a middleware band (cache, metrics, retry), a multi-provider fallback chain, and a streaming pipeline. The service layer built here is reusable across a chat UI, an API backend, a batch processor, or an agent. The full layer diagram and how the pieces interact lives in references/architecture.md; the complete, copy-ready TypeScript for every layer is in references/implementation.md. Prerequisites Groq API key — create one at console.groq.com and export it as GROQAPIKEY. The Groq SDK reads it from the environment; the client is constructed as new Groq({ apiKey: process.env.GROQAPIKEY }). Never hardcode the key. Runtime: Node.js 18+ (for performance.now() and native fetch). Packages: groq-sdk and lru-cache (npm install groq-sdk lru-cache). Optional backup provider: an OpenAI-compatible key if you extend the fallback chain beyond Groq's own models. Instructions Build the service layer in five ordered steps. Each step is one file under src/groq/. The router depends on the registry; the middleware and fallback depend on the client; the streaming pipeline stands alone. Full source for every step (verbatim) is in references/implementation.md. 1. Model Registry (models.ts) — declare a ModelSpec for each model with its tier, context window, speed, cost, and capabilities. Skeleton: 2. Model Router (router.ts) — selectModel(req) maps requirements (maxLatencyMs, needsVision, needsTools, costSensitive) to the cheapest model that satisfies them. Callers pass requirements, never hardcoded ids. 3. Middleware (middleware.ts) — completionWithMiddleware() wraps each call with an LRU cache (deterministic requests only, temperature === 0), latency + token metrics, and a pluggable metrics sink. 4. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/groq-reference-architecture