Agent skill · data analytics · jeremylongshore
anth-observability
Set up observability for Claude API integrations with metrics, logging,
Why this skill is useful
Provides specific implementation patterns for tracking Claude API usage with structured logging and Prometheus metrics that the AI wouldn't reliably generate on its own.
What it needs
Requires prometheus_client installed locally. About 3k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
Anthropic Observability Overview Instrument Claude API calls with structured logging, Prometheus metrics, and cost tracking. Every API response includes usage data and rate limit headers — capture these for dashboards and alerting. Structured Logging Prometheus Metrics Key Metrics Dashboard Metric Description Alert Threshold -------- ------------- ----------------- clauderequeststotal{status="error"} Error count > 5% of total claudelatencyseconds p99 Tail latency > 10s claudecostusd daily Daily spend > 80% budget clauderatelimitremaining{dimension="requests"} RPM headroom < 10% remaining claudetokenstotal{direction="output"} rate Output throughput Spike detection Usage API (Server-Side) Error Handling Observability Gap Risk Fix ------------------- ------ ----- No requestid logged Can't debug with support Capture response.requestid Missing cost tracking Budget surprise Track per-request cost No latency histogram Can't spot slow queries Add Prometheus/Datadog histograms Prerequisites Define SLOs, alert owners, budget and rate-limit thresholds, approved metric labels, and retention rules for telemetry. Configure authenticated server-side access through a secret manager and use a sandbox workspace with synthetic requests to verify instrumentation. Establish a redaction/filter policy before enabling logs, traces, dashboards, or usage reconciliation; prompts, responses, secrets, and personal data are never telemetry fields. Instructions 1. Instrument the request boundary with request ID, model, status, stop reason, token aggregates, cache counters, and duration while excluding content and high-cardinality identifiers. 2. Emit success and failure metrics for authentication, 4xx/5xx, 429, timeout, latency, spend, and remaining rate-limit headroom. Validate labels against an allowlist and cap cardinality. 3. Test dashboards and alerts with synthetic success, timeout, rate-limit, permission, and malformed-response fixtures. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/anth-observability