Agent skill · data analytics · aradotso
harvard-art-museum-data-pipeline
Build ETL pipelines and analytics dashboards using the Harvard Art Museums API with Streamlit, MySQL, and Python
Why this skill is useful
Adds a complete ETL pipeline with scripts for data extraction, transformation, and loading specific to the Harvard Art Museums API.
What it needs
Requires mysql-connector-python, pandas, plotly, python, python-dotenv, requests, streamlit installed locally. Requires harvard-art-museums, mysql account access. About 6k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Art Museum Data Pipeline Skill by ara.so — Data Skills collection. This project provides an end-to-end data engineering solution for collecting, transforming, storing, and analyzing artifact data from the Harvard Art Museums API. It demonstrates production-ready ETL pipelines, SQL analytics, and interactive visualization using Streamlit. What It Does The Harvard Art Museum Data Pipeline: Extracts artifact data from the Harvard Art Museums API with pagination and rate limiting Transforms nested JSON into normalized relational tables (metadata, media, colors) Loads data into MySQL/TiDB Cloud with batch inserts for performance Analyzes data using predefined SQL queries for business insights Visualizes results through interactive Streamlit dashboards with Plotly charts Architecture Key Components: API integration with secure key management Three-table relational schema: artifactmetadata, artifactmedia, artifactcolors 20+ analytical SQL queries Real-time interactive visualizations Installation Prerequisites Python 3.8+ MySQL or TiDB Cloud account Harvard Art Museums API key (obtain from https://harvardartmuseums.org/collections/api) Setup Required Dependencies Configuration Database Connection Configure your MySQL/TiDB connection in your application: API Configuration Set up the Harvard Art Museums API client: Database Schema Create Tables ETL Pipeline Extract Phase Transform Phase Load Phase Complete ETL Workflow SQL Analytics Queries Common Analytics Patterns Streamlit Dashboard Main Application Structure Running the Dashboard Common Patterns Pagination Handler Error Handling Troubleshooting API Rate Limiting If you encounter rate limit errors: Database Connection Issues Memory Optimization for Large Datasets Handling Missing Data This skill provides everything needed to build production-ready data pipelines using the Harvard Art Museums API with proper ETL practices, SQL analytics, and interactive dashboards.
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-art-museum-data-pipeline