Agent skill · data analytics · aradotso
harvard-artifacts-data-engineering-analytics
Build end-to-end ETL pipelines and analytics dashboards using the Harvard Art Museums API with Python, SQL, and Streamlit
Why this skill is useful
Adds a complete ETL pipeline implementation with scripts for data extraction, transformation, and loading specific to the Harvard Art Museums API.
What it needs
Requires mysql-connector-python, streamlit installed locally. Requires harvard-art-museums account access. About 5k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Artifacts Data Engineering Analytics Skill by ara.so — Data Skills collection This project provides an end-to-end data engineering and analytics application for the Harvard Art Museums API. It demonstrates real-world ETL pipelines, SQL database design, analytical queries, and interactive visualization using Streamlit. What This Project Does The application implements a complete data pipeline: Extract: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting Transform: Processes nested JSON into relational database tables (metadata, media, colors) Load: Batch inserts transformed data into MySQL/TiDB Cloud Analyze: Executes 20+ predefined SQL queries for insights Visualize: Renders interactive dashboards with Plotly charts in Streamlit Installation Configuration API Key Setup Obtain an API key from Harvard Art Museums API: Database Configuration Database Schema The project uses three main tables: ETL Pipeline Implementation Extract Phase Transform Phase Load Phase Streamlit Application Main App Structure Analytics Queries Common Patterns Pattern 1: Incremental Data Loading Pattern 2: Error Handling and Logging Running the Application Troubleshooting API Rate Limiting: If you encounter 429 errors, increase the sleep time between requests: Database Connection Issues: Verify environment variables are set: Missing Data Fields: Handle None values in transformations: Memory Issues with Large Datasets: Use batch processing:
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-artifacts-data-engineering-analytics