Agent skill · software engineering · jeremylongshore
apify-rate-limits
Handle Apify API rate limits with proper backoff and request queuing.
Why this skill is useful
Adds specific implementation patterns and code snippets for handling Apify API rate limits that the AI wouldn't reliably generate on its own.
What it needs
Requires apify-client, p-queue installed locally. Requires apify account access. About 3k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
Apify Rate Limits Overview The Apify API enforces rate limits per resource. The apify-client library auto-retries 429s (up to 8 times with exponential backoff), so most workloads never notice a limit. You reach for this skill when bulk operations, custom API calls, or large fan-outs push past what the built-in retry can absorb — you then batch, queue, stagger, and monitor to stay under the ceiling. Full runnable code for every step is in implementation.md; combined scenarios are in examples.md. Apify rate limit rules Scope Limit Notes ------- ------- ------- Per resource (default) 60 req/sec Applies to each Actor, dataset, KV store independently Dataset push 60 req/sec per dataset Batch items to reduce call count Actor runs 60 req/sec per Actor Start runs in sequence or with delays Platform-wide Higher limit Aggregate across all resources "Per resource" means: calls to dataset A and dataset B each get 60 req/sec independently. Every response carries X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset (epoch seconds) headers. Prerequisites An Apify account with API access and APIFYTOKEN set in the environment. The apify-client package installed (npm install apify-client). For custom queuing: p-queue (npm install p-queue); crawlee for sleep and crawler-level concurrency. Instructions The workflow is five steps. Each is summarized here with its core lever; the full runnable code for every step is in implementation.md. 1. Understand built-in retries — apify-client already retries 429/500+ with exponential backoff. Tune maxRetries / minDelayBetweenRetriesMillis only when the defaults are wrong for your endpoint: 2. Batch operations (biggest lever) — collapse per-item loops into one batched call (up to 9 MB), chunking only for very large datasets: 3. Queue custom calls — gate raw API calls through p-queue (concurrency + intervalCap) so fan-out reads never exceed 60 req/sec. See implementation.md § Step 3. 4. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/apify-rate-limits