Agent skill · vasilyu1983
data-streaming
Designs streaming platforms for Kafka, Flink, CDC, and lakehouse ingestion. Use when planning event backbones, CDC pipelines, schema governance, or real-time lakehouse delivery.
What it needs
About 10k tokens when loaded.
What this skill does
Data Streaming Modern Best Practices: choose the event backbone and stream processor separately, treat schemas and replay as product interfaces, default to event-time processing for stateful analytics, and verify managed-service behavior from primary docs before making vendor-specific recommendations. Primary sources live in data/sources.json. Refresh time-sensitive claims against official docs before making definitive recommendations about managed services, version-specific features, limits, or pricing. This skill covers the data platform side of streaming: event backbones, CDC, stateful processing, schema governance, and real-time delivery into lakes, warehouses, search, or serving systems. When to Use Choose between Kafka, Redpanda, Pulsar, Kinesis, or managed Kafka offerings Design topic strategy, partitioning, retention, replay, and ordering guarantees Build or fix CDC pipelines with Debezium, Flink CDC, or managed database-streaming tools Choose between Flink, Kafka Streams, Spark Structured Streaming, or lighter transformation paths Define schema registry, compatibility, contract, and tombstone handling rules Deliver streams into Iceberg, Hudi, Delta, ClickHouse, warehouses, caches, or search systems Review streaming SLOs, lag, checkpointing, reprocessing, and operational failure modes When NOT to Use Lakehouse storage formats, catalogs, or medallion architecture -> Use data-lake-platform OLTP schema tuning or transactional query optimization -> Use data-sql-optimization Event-driven application architecture, CQRS, or domain event design -> Use software-architecture-design BI dashboard automation and Metabase APIs -> Use data-metabase Product instrumentation and attribution strategy -> Use marketing-product-analytics Triage Questions 1. What is the real requirement: operational events, CDC, analytical enrichment, or customer-facing low-latency delivery? 2. What matters most: portability, managed simplicity, geo-replication, cost, or end-to-end latency? 3. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills vasilyu1983/data-streaming