Agent skill · data analytics · aradotso
harvard-artifacts-collection-data-engineering
End-to-end data engineering and analytics application for Harvard Art Museums API with ETL pipelines, SQL analytics, and Streamlit visualization
Why this skill is useful
Adds a complete ETL pipeline and SQL database setup for Harvard Art Museums API, including executable scripts and interactive visualizations.
What it needs
Requires mysql-connector-python, pandas, plotly, requests, streamlit installed locally. Requires harvardartmuseums account access. About 5k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Artifacts Collection Data Engineering Skill by ara.so — Data Skills collection. This project provides an end-to-end data engineering and analytics application built on the Harvard Art Museums API. It demonstrates real-world ETL pipelines, SQL database design, analytical queries, and interactive visualization using Streamlit. The architecture follows: API → ETL → SQL → Analytics → Visualization. What This Project Does API Integration: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting ETL Pipeline: Extracts, transforms, and loads nested JSON into relational database tables SQL Database: Stores structured data across artifactmetadata, artifactmedia, and artifactcolors tables Analytics: Executes 20+ predefined SQL queries for insights Visualization: Interactive dashboards using Plotly and Streamlit Installation Required packages: Configuration Environment Variables Set up your configuration before running: Database Setup Create the required tables: Key Components and Usage 1. API Data Extraction 2. ETL Pipeline Implementation 3. Complete ETL Workflow 4. SQL Analytics Queries 5. Streamlit Dashboard Running the Application Common Patterns Incremental Data Loading Error Handling in ETL Troubleshooting API Rate Limiting: Add delays between requests Database Connection Issues: Verify credentials and network access Missing Data Fields: Handle null values during transformation Memory Issues with Large Datasets: Use chunking
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-artifacts-collection-data-engineering