Agent skill · data analytics · aradotso
google-cloud-data-engineering-hub
Production-grade GCP data engineering projects using BigQuery, Dataflow, Beam, Composer, Pub/Sub, Dataproc, and Vertex AI
Why this skill is useful
Provides 54+ complete GCP data engineering projects with executable code and automation scripts for setting up data pipelines.
What it needs
Requires google-cloud-sdk, python installed locally. Requires google-cloud account access. About 7k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Google Cloud Data Engineering Hub Skill Skill by ara.so — Data Skills collection This skill enables AI coding agents to help developers build production-grade Google Cloud Platform (GCP) data engineering solutions using this comprehensive reference repository of 54+ working projects covering BigQuery, Dataflow, Apache Beam, Cloud Composer, Pub/Sub, Dataproc, Gemini AI, and Vertex AI. What This Project Provides A curated collection of complete, runnable GCP data engineering projects. Each project includes: Modular Python code (not scripts) ASCII architecture diagrams Sample data fixtures deploy.sh with GCP setup automation End-to-end working examples tested against live GCP environments Built by Vishal Bulbule (Google Developer Expert, 12x GCP Certified). Installation & Setup Clone the Repository Prerequisites Python 3.10+ Google Cloud SDK (gcloud CLI) Active GCP project with billing enabled Appropriate IAM permissions Environment Setup Project Categories & Key Examples BigQuery Projects (01-07) CSV Ingestion Pipeline (01-bq-csv-ingestion-pipeline) UPSERT/MERGE Pattern (03-bq-upsert-merge-pattern) BigQuery ML (05-bq-ml-train-predict) Cloud Storage Projects (08-10) File Management (08-gcs-file-management) Signed URLs & Lifecycle (09-gcs-signed-urls-lifecycle) Pub/Sub Streaming (31-pubsub-streaming-pipeline) Apache Beam / Dataflow (47-50) Basic Beam Pipeline (47-beam-data-transformation) Beam to BigQuery (48-beam-csv-to-bigquery-load) Advanced Beam Patterns (49-beam-advanced-patterns) Cloud Composer / Airflow (51-54) Basic DAG (51-cloud-composer-dag-basics) BigQuery Pipeline (52-composer-bigquery-pipeline) python from airflow import DAG from airflow.providers.google.cloud.sensors.gcs import GCSObjectExistenceSensor from airflow.providers.google.cloud.transfers.gcstobigquery import GCSToBigQueryOperator from airflow.providers.google.cloud.operators.bigquery import BigQueryInsertJobOperator from airflow.operators.python import PythonOperator from datetime import datetime …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/google-cloud-data-engineering-hub