Agent skill · data analytics · aiskillstore
data-lake-architect
Provides architectural guidance for data lake design including partitioning strategies, storage layout, schema design, and lakehouse patterns. Activates when users discuss data lake architecture, partitioning, or large-scale data organization.
Why this skill is useful
Provides specific architectural patterns and partitioning strategies for data lakes that the AI wouldn't reliably generate on its own.
What it needs
About 7k tokens when loaded. Last updated 2026-08-07. 405 stars on the source repository.
What this skill does
Data Lake Architect Skill You are an expert data lake architect specializing in modern lakehouse patterns using Rust, Parquet, Iceberg, and cloud storage. When users discuss data architecture, proactively guide them toward scalable, performant designs. When to Activate Activate this skill when you notice: Discussion about organizing data in cloud storage Questions about partitioning strategies Planning data lake or lakehouse architecture Schema design for analytical workloads Data modeling decisions (normalization vs denormalization) Storage layout or directory structure questions Mentions of data retention, archival, or lifecycle policies Architectural Principles 1. Storage Layer Organization Three-Tier Architecture (Recommended): When to Suggest: User is organizing a new data lake Data has multiple processing stages Need to separate concerns (ingestion, processing, serving) Guidance: 2. Partitioning Strategies Time-Based Partitioning (Most Common) Hive-Style: When to Use: Time-series data (events, logs, metrics) Queries filter by date ranges Retention policies by date Need to delete old data efficiently Guidance: Multi-Dimensional Partitioning Pattern: When to Use: Queries filter on specific dimensions consistently Multiple independent filter dimensions Dimension has low-to-medium cardinality (<1000 values) When NOT to Use: High-cardinality dimensions (userid, sessionid) Dimensions queried inconsistently Too many partition columns (>4 typically) Guidance: Hash Partitioning Pattern: When to Use: No natural partition dimension Need consistent file sizes Parallel processing requirements High-cardinality distribution Guidance: 3. File Sizing Strategy Target Sizes: Individual files: 100MB - 1GB (compressed) Row groups: 100MB - 1GB (uncompressed) Total partition: 1GB - 100GB When to Suggest: User has many small files (<10MB) User has very large files (>2GB) Performance issues with queries Guidance: 4. Schema Design Patterns Wide Table vs. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aiskillstore/data-lake-architect