Agent skill · data analytics · aradotso
harvard-artifacts-data-pipeline
Build end-to-end data engineering pipelines using Harvard Art Museums API with ETL, SQL analytics, and Streamlit visualization
Why this skill is useful
Adds executable ETL scripts and SQL database schema for transforming and storing Harvard Art Museums API data.
What it needs
Requires mysql-connector-python, pandas, plotly, python-dotenv, requests, streamlit installed locally. Requires harvard account access. About 5k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Artifacts Data Pipeline Skill by ara.so — Data Skills collection. What This Project Does The Harvard Artifacts Collection Data Engineering & Analytics App is an end-to-end data pipeline that demonstrates professional ETL workflows. It fetches artifact data from the Harvard Art Museums API, transforms it into structured relational tables, stores it in MySQL/TiDB Cloud, and provides interactive analytics through a Streamlit dashboard with Plotly visualizations. Key capabilities: API data extraction with pagination and rate limiting ETL transformations for nested JSON to relational schema SQL database design with proper foreign keys 20+ predefined analytical queries Interactive visualization dashboards Installation Required packages: Configuration API Setup Get your Harvard Art Museums API key from: https://docs.harvardartmuseums.org/ Database Schema The project uses three main tables: Key API Patterns Fetching Artifacts with Pagination ETL Transformation Database Operations Loading Data into MySQL Analytical SQL Queries Example Queries Streamlit Dashboard Basic App Structure Running the Application Common Patterns Error Handling for API Calls Data Validation Troubleshooting API Rate Limiting: Database Connection Issues: Memory Management for Large Datasets:
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-artifacts-data-pipeline