Agent skill · data analytics · aradotso
harvard-artifacts-collection-analytics-pipeline
End-to-end data engineering pipeline for Harvard Art Museums API with ETL, SQL analytics, and Streamlit visualization
Why this skill is useful
Adds a complete data engineering pipeline with ETL scripts, SQL queries, and a Streamlit dashboard specifically for the Harvard Art Museums API.
What it needs
Requires mysql-connector-python, pandas, plotly, requests, streamlit installed locally. Requires harvard account access. About 6k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Artifacts Collection Analytics Pipeline Skill by ara.so — Data Skills collection. Overview This project provides a complete data engineering solution for the Harvard Art Museums API, featuring: ETL pipeline for artifact metadata, media, and color data SQL database storage (MySQL/TiDB Cloud) 20+ analytical SQL queries Interactive Streamlit dashboard with Plotly visualizations The architecture follows: API → ETL → SQL → Analytics → Visualization Installation Required Dependencies Configuration Environment Variables Create a .env file in the project root: Database Setup ETL Pipeline Extract: Fetch Data from Harvard API Transform: Process JSON Data Load: Insert into Database SQL Analytics Queries Sample Analytical Queries Streamlit Dashboard Main Application Structure Running the Dashboard Common Patterns Incremental Data Loading Error Handling and Logging Troubleshooting API Rate Limiting Database Connection Issues Memory Management for Large Datasets Key Features Summary ETL Pipeline: Automated data collection with pagination and rate limiting SQL Storage: Normalized schema with foreign key relationships Analytics: 20+ pre-built queries for artifact insights Visualization: Interactive Plotly charts in Streamlit Scalability: Handles batch processing and incremental loads
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-artifacts-collection-analytics-pipeline