Agent skill · data analytics · aradotso
harvard-art-museum-data-engineering
ETL pipeline and analytics app for Harvard Art Museums API using Python, SQL, and Streamlit
Why this skill is useful
Provides a complete ETL pipeline with specific SQL schema and API integration patterns that the AI wouldn't reliably generate on its own.
What it needs
Requires mysql-connector-python, pandas, plotly, python-dotenv, requests, streamlit installed locally. Requires harvard account access. About 5k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Art Museum Data Engineering Skill Skill by ara.so — Data Skills collection. Overview This project provides an end-to-end data engineering solution for the Harvard Art Museums API. It implements a complete ETL pipeline that extracts artifact data, transforms it into relational tables, loads it into SQL databases (MySQL/TiDB), and provides interactive analytics dashboards using Streamlit. Architecture Flow: API → ETL → SQL → Analytics → Visualization Installation Configuration Environment Variables Create a .env file in the project root: Getting Harvard API Key 1. Visit: https://docs.api.harvardartmuseums.org/ 2. Register for a free API key 3. Store it in your .env file Database Schema The project uses three main tables with relational structure: Core Usage Patterns 1. API Data Extraction 2. ETL Pipeline Implementation 3. Database Loading 4. Analytics Queries 5. Streamlit Dashboard Running the Application Complete ETL Workflow Common Troubleshooting API Rate Limiting Database Connection Issues Handling Missing Data Best Practices 1. Always use environment variables for sensitive credentials 2. Implement pagination when fetching large datasets 3. Use batch inserts for better database performance 4. Handle API rate limits with retry logic 5. Validate data before loading to database 6. Create indexes on frequently queried columns 7. Log ETL operations for debugging and monitoring
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-art-museum-data-engineering