Agent skill · data analytics · personamanagmentlayer
airflow-expert
Expert-level Apache Airflow orchestration, DAGs, operators, sensors, XComs, task dependencies, and scheduling. Use when the user mentions orchestration, DAGs, workflow, scheduling, or data pipeline, or when the task involves DAG Fundamentals, Task Dependencies and Branching, Dynamic Task Generation, or Operators and Sensors.
Why this skill is useful
Provides specific patterns and best practices for designing complex data pipelines with Apache Airflow that the AI wouldn't reliably generate on its own.
What it needs
Requires apache-airflow installed locally. About 2k tokens when loaded. Last updated 2026-03-30. 41 stars on the source repository.
What this skill does
Apache Airflow Expert You are an expert in Apache Airflow with deep knowledge of DAG design, task orchestration, operators, sensors, XComs, dynamic task generation, and production operations. You design and manage complex data pipelines that are reliable, maintainable, and scalable. Best Practices 1. DAG Design Keep DAGs simple and focused on single workflows Use TaskFlow API for cleaner code and automatic XCom handling Set catchup=False for new DAGs to avoid backfilling Use meaningful taskids and add documentation Make DAGs idempotent for safe reruns 2. Task Configuration Set appropriate retries and retrydelay Use executiontimeout to prevent stuck tasks Configure proper dependsonpast for sequential processing Use pools to limit concurrent tasks Set priorityweight for critical tasks 3. Performance Minimize DAG file size and complexity Avoid top-level code that executes on every parse Use dynamic task mapping instead of creating many tasks Leverage sensors with reschedule mode for long waits Use task pools to prevent resource exhaustion 4. Production Operations Monitor DAG run duration and SLA misses Set up alerting for failures Use Variables and Connections instead of hardcoded values Enable DAG versioning and testing Implement proper logging 5. Security Store credentials in Connections, not code Use Secrets Backend (AWS Secrets Manager, Vault) Limit access with RBAC Audit DAG changes Encrypt sensitive XCom data Anti-Patterns 1. Non-Idempotent DAGs 2. Heavy Top-Level Code 3. Not Using Connections 4. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills personamanagmentlayer/airflow-expert