Agent skill · data analytics · aradotso
harvard-artifacts-data-engineering-pipeline
Build ETL pipelines and analytics dashboards using Harvard Art Museums API with SQL storage and Streamlit visualization
Why this skill is useful
Adds a complete ETL pipeline with executable scripts for data extraction, transformation, and loading specific to the Harvard Art Museums API.
What it needs
Requires mysql-connector-python, pandas, plotly, python-dotenv, requests, streamlit installed locally. Requires harvard-art-museums account access. About 6k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Artifacts Data Engineering Pipeline Skill by ara.so — Data Skills collection. This project provides an end-to-end data engineering solution for collecting, transforming, storing, and analyzing artifact data from the Harvard Art Museums API. It demonstrates production-ready ETL pipelines, relational database design, SQL analytics, and interactive Streamlit dashboards. What This Project Does API Integration: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting ETL Pipeline: Extracts nested JSON, transforms into relational schema, loads into SQL database Database Design: Implements normalized tables (artifactmetadata, artifactmedia, artifactcolors) SQL Analytics: Executes 20+ predefined analytical queries Visualization: Interactive Plotly charts rendered through Streamlit Installation Required Dependencies Configuration API Key Setup Get your API key from Harvard Art Museums API. Create a .env file: Database Setup The project uses MySQL or TiDB Cloud. Create the database schema: Core Components ETL Pipeline Extract: Fetch Data from API Transform: Process JSON to Relational Data Load: Insert into SQL Database Complete ETL Workflow SQL Analytics Queries Sample Analytical Queries Execute Queries Streamlit Dashboard Main Application Common Patterns Batch Processing with Rate Limiting Error Handling in ETL Running the Application Troubleshooting API Connection Issues Database Connection Issues Common Error: Empty Results This skill enables AI agents to guide developers through building production-ready data engineering pipelines using the Harvard Art Museums API with proper ETL practices, SQL analytics, and interactive visualizations.
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-artifacts-data-engineering-pipeline