Agent skill · data analytics · aradotso
harvard-art-museums-data-engineering-analytics
Build ETL pipelines and analytics dashboards using the Harvard Art Museums API with Python, SQL, and Streamlit
Why this skill is useful
Adds a complete ETL pipeline with executable scripts for data extraction, transformation, and loading into SQL databases, which the AI cannot generate on its own.
What it needs
Requires mysql-connector-python, pandas, plotly, python-dotenv, requests, streamlit installed locally. Requires harvard-art-museums account access. About 5k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Art Museums Data Engineering & Analytics Skill by ara.so — Data Skills collection. This project demonstrates a complete data engineering workflow: extracting data from the Harvard Art Museums API, transforming it into relational tables, loading into SQL databases, and building interactive analytics dashboards with Streamlit. What This Project Does API Integration: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting ETL Pipeline: Transforms nested JSON into structured relational tables (metadata, media, colors) SQL Storage: Loads data into MySQL/TiDB Cloud with proper schema design Analytics: Executes 20+ predefined analytical queries Visualization: Interactive Plotly dashboards for data exploration Installation Requirements typically include: API Integration Getting API Key 1. Register at Harvard Art Museums API 2. Store key in environment variable or .env file Fetching Artifacts Handling Pagination ETL Pipeline Extract: Parse API Response Transform: Clean and Validate Load: Insert into SQL Database Streamlit Analytics Dashboard Main Application Structure SQL Analytics Queries Common Patterns Environment Configuration Error Handling Troubleshooting API Rate Limiting: Add delays between requests Database Connection Issues: Check credentials and network Missing Data in API Response: Always use .get() with defaults Memory Issues with Large Datasets: Use chunking
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-art-museums-data-engineering-analytics