Agent skill · data analytics · aradotso
harvard-art-museums-data-pipeline
Build end-to-end data engineering pipelines with the Harvard Art Museums API, ETL processes, SQL analytics, and Streamlit visualization
Why this skill is useful
Adds a complete data pipeline implementation with executable scripts for ETL, SQL analytics, and Streamlit visualization that are not commonly found in public documentation.
What it needs
Requires mysql-connector-python, pandas, plotly, python-dotenv, requests, streamlit installed locally. Requires harvard account access. About 6k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Art Museums Data Pipeline Skill by ara.so — Data Skills collection. This skill enables you to build production-ready data engineering pipelines using the Harvard Art Museums API. It covers ETL workflows, relational database design, SQL analytics, and interactive Streamlit dashboards for artifact data visualization. What This Project Does The Harvard-Artifacts-Collection-Data-Engineering-Analytics-App demonstrates a complete data pipeline: 1. Extract: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting 2. Transform: Converts nested JSON into normalized relational tables 3. Load: Batch inserts data into MySQL/TiDB Cloud databases 4. Analyze: Executes analytical SQL queries for insights 5. Visualize: Renders interactive dashboards with Plotly and Streamlit The architecture follows: API → ETL → SQL → Analytics → Visualization Installation Dependencies Configuration Environment Variables Create a .env file in the project root: Get Harvard API Key 1. Visit Harvard Art Museums API 2. Register for a free API key 3. Add to .env file Database Setup API Integration Basic API Request Paginated Data Collection ETL Pipeline Extract and Transform Load into Database SQL Analytics Sample Analytical Queries Streamlit Dashboard Main Application Run the Dashboard Common Patterns Rate Limiting API Requests Error Handling and Retries Troubleshooting API Key Issues Database Connection Errors Memory Issues with Large Datasets Streamlit Caching for Performance
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-art-museums-data-pipeline