Agent skill · data analytics · aradotso
harvard-art-museum-etl-pipeline
Build ETL pipelines and analytics dashboards using the Harvard Art Museums API with Python, SQL, and Streamlit
Why this skill is useful
Adds a complete ETL pipeline implementation with executable scripts for data extraction, transformation, and loading specific to the Harvard Art Museums API.
What it needs
Requires mysql-connector-python, streamlit installed locally. Requires harvardartmuseums account access. About 5k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Art Museum ETL Pipeline Skill by ara.so — Data Skills collection. This skill enables you to build end-to-end data engineering and analytics applications using the Harvard Art Museums API. The project demonstrates real-world ETL pipelines, SQL database design, analytical queries, and interactive Streamlit dashboards for museum artifact data. What This Project Does The Harvard Artifacts Collection app provides a complete data engineering workflow: Extract: Fetch artifact data from Harvard Art Museums API with pagination and rate limiting Transform: Convert nested JSON into normalized relational tables (metadata, media, colors) Load: Batch insert data into MySQL/TiDB Cloud with proper foreign key relationships Analyze: Execute 20+ predefined SQL queries for insights Visualize: Display results in interactive Plotly charts via Streamlit The architecture follows: API → ETL → SQL → Analytics → Visualization Installation Configuration API Key Setup Obtain your Harvard Art Museums API key from: https://www.harvardartmuseums.org/collections/api Store it securely: Database Configuration Configure your MySQL/TiDB connection: Database Schema The project uses three normalized tables: Running the Application ETL Pipeline Implementation Extract: Fetch Data from API Transform: Normalize JSON Data Load: Insert into Database Analytics Queries Sample SQL Queries Streamlit Dashboard Implementation Common Patterns Complete ETL Workflow Rate Limiting for API Calls Troubleshooting API Connection Issues Database Connection Failures Missing Data Handling Advanced Usage Incremental Data Loading This skill provides everything needed to build production-ready ETL pipelines for museum data using modern data engineering practices.
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-art-museum-etl-pipeline