Agent skill · data analytics · aradotso
harvard-artifacts-data-engineering-streamlit
Build end-to-end data engineering pipelines with Harvard Art Museums API, ETL workflows, SQL analytics, and Streamlit dashboards
Why this skill is useful
Adds a complete end-to-end data engineering pipeline with executable scripts for ETL, SQL analytics, and Streamlit visualizations that are not commonly found in public resources.
What it needs
Requires mysql-connector-python, pandas, plotly, python-dotenv, requests, streamlit installed locally. Requires harvard-art-museums account access. About 7k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Artifacts Collection Data Engineering & Analytics Skill by ara.so — Data Skills collection. This project provides an end-to-end data engineering and analytics application for the Harvard Art Museums API. It demonstrates real-world ETL pipelines, SQL database design, analytical queries, and interactive Streamlit visualizations for museum artifact data. What This Project Does Data Collection: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting ETL Pipeline: Extracts, transforms, and loads artifact metadata, media, and color information into relational SQL databases SQL Analytics: Runs 20+ predefined analytical queries for insights on artifacts by culture, century, department, and media Visualization: Creates interactive Plotly dashboards in Streamlit for exploring query results Database Management: Manages MySQL/TiDB Cloud schemas with proper foreign key relationships Installation Prerequisites Or use requirements file: Environment Setup Create a .env file in the project root: Get your Harvard Art Museums API key from: https://www.harvardartmuseums.org/collections/api Database Setup Running the Application Access the dashboard at http://localhost:8501 Core Components 1. API Data Collection 2. ETL Pipeline 3. Database Connection 4. Analytical SQL Queries 5. Streamlit Dashboard Common Patterns Batch Processing Large Datasets Error Handling in ETL Custom Query Builder Troubleshooting API Rate Limiting Database Connection Issues Memory-Efficient Processing Advanced Features Data Quality Checks This skill provides comprehensive knowledge for building data engineering pipelines with the Harvard Art Museums API, implementing ETL workflows, and creating analytics dashboards with Streamlit.
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-artifacts-data-engineering-streamlit