Agent skill · data analytics · jeremylongshore
groq-observability
Set up observability for Groq integrations: latency histograms, token
Why this skill is useful
Adds specific Prometheus metrics and monitoring patterns for Groq API integrations that the AI wouldn't reliably generate on its own.
What it needs
Requires groq-sdk, node, prom-client installed locally. Requires groq account access. About 3k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
Groq Observability Overview Monitor Groq LPU inference for latency, token throughput, rate limit utilization, and cost. Groq's defining advantage is speed (280-560 tok/s), so latency degradation is the highest-priority signal. The API returns rich timing metadata (queuetime, prompttime, completiontime) and rate limit headers on every response. Prerequisites A Groq account with an API key exported as the GROQAPIKEY environment variable — the groq-sdk client reads it automatically (new Groq()). Node.js with groq-sdk and prom-client installed (npm install groq-sdk prom-client). A Prometheus scrape target and (optionally) Grafana for the dashboard panels. Key Metrics to Track Metric Type Source Why -------- ------ -------- ----- TTFT (time to first token) Histogram Client-side timing Groq's main value prop Tokens/second Gauge usage.completiontime Throughput degradation Total latency Histogram Client-side timing End-to-end performance Rate limit remaining Gauge x-ratelimit-remaining- headers Prevent 429s Token usage Counter usage.totaltokens Cost attribution Error rate by code Counter Error handler Availability Estimated cost Counter Tokens model price Budget tracking Instructions Apply these six steps in order. Steps 1-2 are the core instrumentation loop — wrap the client, then feed a Prometheus instrument set from each call. Steps 3-6 add rate-limit tracking, alerting, structured logs, and dashboards on top. The lean client skeleton is below; the full code for every step lives in references/implementation.md. 1. Instrumented client — wrap groq.chat.completions.create so latency, tokens, queue time, and estimated cost are captured on the same path as the request (trackedCompletion). 2. Prometheus metrics — register a histogram (latency), counters (tokens, cost, errors), and gauges (throughput, rate-limit remaining), then feed them from emitMetrics. 3. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/groq-observability