Agent skill · data analytics · aradotso
data-engineering-patterns-fabric-databricks
600+ patterns and concepts for Azure Databricks, Microsoft Fabric, and PySpark data engineering - covering lakehouse architecture, Delta Lake, pipelines, and production best practices.
Why this skill is useful
Provides 600+ specific data engineering patterns and executable code snippets for Azure Databricks and Microsoft Fabric that enhance the AI's ability to generate optimized data workflows.
What it needs
Requires git installed locally. About 7k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Data Engineering Patterns - Fabric & Databricks Skill by ara.so — Data Skills collection. This skill provides access to 600+ field-tested data engineering patterns for Microsoft Fabric, Azure Databricks, and PySpark. These patterns cover everything from pipeline design and Delta Lake optimization to Unity Catalog governance and cost architecture. What This Project Provides A comprehensive collection of patterns organized into 12 books covering: Microsoft Fabric (250 patterns): Pipelines and Data Factory Lakehouse and PySpark Warehouse and SQL Power BI in Fabric Architecture Patterns Azure Databricks (350 patterns): Clusters and Compute Delta Lake Workflows and Orchestration Structured Streaming and Auto Loader Unity Catalog Databricks SQL and Photon Platform and Cost Architecture PySpark: 88 concepts for production Spark across both platforms Installation Clone the repository to access all pattern PDFs: Repository Structure Key Pattern Categories Microsoft Fabric Patterns Pipeline and Data Factory Patterns Common patterns include: Incremental data loading strategies Pipeline retry and error handling Parameter-driven pipeline design Activity dependencies and control flow Copy activity optimization Metadata-driven frameworks Example incremental load pattern in Fabric Pipeline: Lakehouse and PySpark Patterns Key patterns for Fabric Lakehouse: Pattern: Optimize Delta tables in Fabric: Warehouse and SQL Patterns Pattern: Create warehouse tables with proper partitioning: Azure Databricks Patterns Cluster and Compute Patterns Pattern: Configure autoscaling cluster for cost optimization: Delta Lake Patterns Pattern: Time travel and versioning: Pattern: Change Data Feed (CDF) for incremental processing: Structured Streaming Patterns Pattern: Auto Loader with schema evolution: Pattern: Streaming aggregations with watermarking: Unity Catalog Patterns Pattern: Create governed table with row-level security: Pattern: Column masking with Unity Catalog: Workflows and Orchestration …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/data-engineering-patterns-fabric-databricks