Agent skill · operations · aj-geddes
background-job-processing
Implement background job processing systems with task queues, workers, scheduling, and retry mechanisms. Use when handling long-running tasks, sending emails, generating reports, and processing large datasets asynchronously.
Why this skill is useful
Provides specific implementation patterns and configurations for background job processing using various frameworks that the AI wouldn't reliably generate on its own.
What it needs
Requires celery, kombu, redis installed locally. About 2k tokens when loaded. Last updated 2026-03-04. 307 stars on the source repository.
What this skill does
Background Job Processing Table of Contents Overview When to Use Quick Start Reference Guides Best Practices Overview Build robust background job processing systems with distributed task queues, worker pools, job scheduling, error handling, retry policies, and monitoring for efficient asynchronous task execution. When to Use Handling long-running operations asynchronously Sending emails in background Generating reports or exports Processing large datasets Scheduling recurring tasks Distributing compute-intensive operations Quick Start Minimal working example: Reference Guides Detailed implementations in the references/ directory: Guide Contents --- --- Python with Celery and Redis Python with Celery and Redis Node.js with Bull Queue Node.js with Bull Queue Ruby with Sidekiq Ruby with Sidekiq Job Retry and Error Handling Job Retry and Error Handling Monitoring and Observability Monitoring and Observability Best Practices ✅ DO Use task timeouts to prevent hanging jobs Implement retry logic with exponential backoff Make tasks idempotent Use job priorities for critical tasks Monitor queue depths and job failures Log job execution details Clean up completed jobs Set appropriate batch sizes for memory efficiency Use dead-letter queues for failed jobs Test jobs independently ❌ DON'T Use synchronous operations in async tasks Ignore job failures Make tasks dependent on external state Use unbounded retries Store large objects in job data Forget to handle timeouts Run jobs without monitoring Use blocking operations in queues Forget to track job progress Mix unrelated operations in one job
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aj-geddes/background-job-processing