Agent skill · data analytics · aradotso
harvard-artifacts-collection-analytics-etl
ETL pipeline and analytics app for Harvard Art Museums API with SQL storage and Streamlit visualization
Why this skill is useful
Adds a complete ETL pipeline with executable scripts for data extraction, transformation, and visualization that are not commonly found in public documentation.
What it needs
Requires mysql-connector-python, pandas, plotly, python-dotenv, requests, streamlit installed locally. Requires harvard account access. About 5k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Artifacts Collection Analytics ETL Skill by ara.so — Data Skills collection. Overview This project provides an end-to-end data engineering solution for the Harvard Art Museums API. It demonstrates a production-ready ETL pipeline that extracts artifact data, transforms nested JSON structures into relational tables, loads data into SQL databases (MySQL/TiDB), and provides interactive analytics through a Streamlit dashboard with Plotly visualizations. Architecture Flow: API → ETL → SQL → Analytics → Visualization Installation Required packages: Project Structure Database Schema The pipeline creates three relational tables: ETL Pipeline Implementation 1. Extract Data from Harvard API 2. Transform Data 3. Load Data to SQL Analytics Queries Streamlit Application Running the Application Common Patterns Rate Limiting API Requests Handling Missing Data Troubleshooting API Key Error: Ensure HARVARDAPIKEY is set in environment variables. Get your key from https://docs.harvardartmuseums.org/ Database Connection Issues: Verify database credentials and ensure the database exists: Memory Issues with Large Datasets: Process data in batches: Empty Query Results: Check if ETL pipeline has run and data exists in tables:
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-artifacts-collection-analytics-etl