Agent skill · data analytics · aiskillstore
senior-data-engineer
World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data architectures, building data pipelines, optimizing data workflows, or implementing data governance.
Why this skill is useful
Adds 3 executable scripts for pipeline orchestration, data quality validation, and ETL performance optimization that enhance the AI's capabilities.
What it needs
Requires python installed locally. About 3k tokens when loaded. Last updated 2026-08-07. 405 stars on the source repository.
What this skill does
Senior Data Engineer World-class senior data engineer skill for production-grade AI/ML/Data systems. Quick Start Main Capabilities Core Expertise This skill covers world-class capabilities in: Advanced production patterns and architectures Scalable system design and implementation Performance optimization at scale MLOps and DataOps best practices Real-time processing and inference Distributed computing frameworks Model deployment and monitoring Security and compliance Cost optimization Team leadership and mentoring Tech Stack Languages: Python, SQL, R, Scala, Go ML Frameworks: PyTorch, TensorFlow, Scikit-learn, XGBoost Data Tools: Spark, Airflow, dbt, Kafka, Databricks LLM Frameworks: LangChain, LlamaIndex, DSPy Deployment: Docker, Kubernetes, AWS/GCP/Azure Monitoring: MLflow, Weights & Biases, Prometheus Databases: PostgreSQL, BigQuery, Snowflake, Pinecone Reference Documentation 1. Data Pipeline Architecture Comprehensive guide available in references/datapipelinearchitecture.md covering: Advanced patterns and best practices Production implementation strategies Performance optimization techniques Scalability considerations Security and compliance Real-world case studies 2. Data Modeling Patterns Complete workflow documentation in references/datamodelingpatterns.md including: Step-by-step processes Architecture design patterns Tool integration guides Performance tuning strategies Troubleshooting procedures 3. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aiskillstore/senior-data-engineer