Agent skill · data analytics · daffy0208
data-engineer
Expert in data pipelines, ETL processes, and data infrastructure
Why this skill is useful
Adds executable scripts for ETL processes and data pipeline patterns that enhance the AI's capabilities in data engineering.
What it needs
Requires typescript installed locally. About 8k tokens when loaded. Last updated 2025-12-25. 33 stars on the source repository.
What this skill does
Data Engineer Skill I help you build robust data pipelines, ETL processes, and data infrastructure. What I Do Data Pipelines: Extract, Transform, Load (ETL) processes Data ingestion from multiple sources Batch and real-time processing Data quality validation Data Infrastructure: Database schema design Data warehousing Caching strategies Data replication Analytics: Data aggregation Metrics calculation Report generation Data export ETL Patterns Pattern 1: Simple ETL Pipeline Use case: Daily sync from external API to database Schedule with Vercel Cron: --- Pattern 2: Incremental ETL (Delta Sync) Use case: Only process new/changed records Benefits: Faster (only process changes) Lower API costs Reduced database load --- Pattern 3: Real-Time Data Pipeline Use case: Process events as they happen --- Data Transformation Patterns Transformation 1: Data Cleaning Transformation 2: Data Enrichment Transformation 3: Data Aggregation --- Data Validation Schema Validation with Zod Data Quality Checks --- Data Warehousing Star Schema Design Populate Data Warehouse --- Performance Optimization Batch Processing Database Optimization --- Caching Strategies Redis for Aggregated Data --- Monitoring and Logging Pipeline Monitoring --- Common Patterns Pattern: Upsert with Conflict Resolution Pattern: Dead Letter Queue --- When to Use Me Perfect for: Building ETL pipelines Data migration projects Analytics infrastructure Data quality improvement Real-time data processing I'll help you: Design data pipelines Transform and clean data Build data warehouses Optimize database queries Monitor data quality What I'll Create Let's build robust, scalable data infrastructure!
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills daffy0208/data-engineer