Agent skill · data analytics · aradotso
harvard-art-museums-data-engineering-app
End-to-end data engineering and analytics application for Harvard Art Museums API with ETL pipelines, SQL analytics, and Streamlit visualization
Why this skill is useful
Adds a complete end-to-end data engineering application with ETL pipelines, SQL analytics, and interactive visualizations that the AI cannot generate from general knowledge.
What it needs
Requires mysql-connector-python, pandas, plotly, python-dotenv, requests, streamlit installed locally. Requires harvard-art-museums account access. About 5k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Art Museums Data Engineering App Skill by ara.so — Data Skills collection. This project provides an end-to-end data engineering and analytics application built on the Harvard Art Museums API. It demonstrates real-world ETL pipelines, SQL database design, analytical queries, and interactive visualization using Streamlit. What This Project Does The application follows a complete data pipeline: API → ETL → SQL → Analytics → Visualization Collects artifact data from Harvard Art Museums API with pagination and rate limiting Transforms nested JSON into normalized relational tables Loads data into MySQL/TiDB Cloud databases Analyzes with 20+ predefined SQL queries Visualizes results through interactive Plotly dashboards in Streamlit Installation Required dependencies: Configuration API Key Setup Get a free API key from Harvard Art Museums API: Database Connection Database Schema The application uses three normalized tables: ETL Pipeline Extract: Fetch Data from API Transform: Normalize JSON Data Load: Insert into Database Streamlit Application Main App Structure Common Patterns Running the Complete ETL Pipeline Incremental Data Loading Key Commands Run Streamlit App Database Setup Troubleshooting API Rate Limiting If you hit rate limits, increase the delay: Database Connection Issues Missing Data Handling Memory Issues with Large Datasets
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-art-museums-data-engineering-app