Agent skill · data analytics · jeremylongshore
groq-performance-tuning
Optimize Groq API performance with model selection, caching, streaming,
Why this skill is useful
Provides specific caching and parallelism strategies with code snippets for optimizing Groq API performance that the AI wouldn't reliably generate on its own.
What it needs
Requires node installed locally. Requires groq account access. About 4k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
Groq Performance Tuning Overview Maximize Groq's LPU inference speed advantage. Groq already delivers extreme throughput (280-560 tok/s) and low latency (<200ms TTFT), but client-side optimization -- model selection, prompt size, streaming, caching, and parallelism -- determines whether your application fully exploits that speed. This skill walks through six tuning levers at a high level; the complete, copy-pasteable code for each lives in references/implementation.md, and end-to-end worked scenarios live in references/examples.md. Prerequisites Groq API key — set GROQAPIKEY in the environment. The groq-sdk client (new Groq()) reads it automatically; never hardcode the key. Node.js 18+ with the groq-sdk package installed (npm install groq-sdk). Optional packages for the caching and parallelism steps: lru-cache and p-queue (npm install lru-cache p-queue). A baseline latency measurement of your current integration so you can confirm the tuning actually helps. Groq Speed Benchmarks Model TTFT Throughput Context ------- ------ ----------- --------- llama-3.1-8b-instant ~50ms ~560 tok/s 128K llama-3.3-70b-versatile ~150ms ~280 tok/s 128K llama-3.3-70b-specdec ~100ms ~400 tok/s 128K meta-llama/llama-4-scout-17b-16e-instruct ~80ms ~460 tok/s 128K TTFT = Time to First Token. Actual values depend on prompt size and server load. Instructions Apply these six levers in order. Each is a small, independent change — start with the ones that match your bottleneck (model choice and caching give the biggest wins on most workloads). The full code for every step is in references/implementation.md. 1. Choose the right model for speed. Map each call site to a speed tier: llama-3.1-8b-instant for latency-critical paths, llama-3.3-70b-versatile for quality-sensitive paths, llama-3.3-70b-specdec for 70b quality at higher throughput. Set temperature: 0 so responses are deterministic (and cacheable). 2. Minimize token count. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/groq-performance-tuning