Agent skill · data analytics · aradotso
harvard-artifacts-data-engineering-app
Build ETL pipelines and analytics dashboards using the Harvard Art Museums API with SQL storage and Streamlit visualization
Why this skill is useful
Adds a complete ETL pipeline with scripts for data extraction, transformation, and loading into SQL databases, along with interactive visualizations.
What it needs
Requires pandas, plotly, pymysql, python-dotenv, requests, streamlit installed locally. Requires harvard account access. About 5k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Artifacts Data Engineering App Skill by ara.so — Data Skills collection. This project demonstrates end-to-end data engineering using the Harvard Art Museums API. It extracts artifact data, transforms it into relational tables, loads it into SQL databases (MySQL/TiDB), and provides interactive analytics through Streamlit dashboards with Plotly visualizations. What It Does API Integration: Fetches paginated artifact data from Harvard Art Museums API ETL Pipeline: Transforms nested JSON into normalized relational tables SQL Storage: Creates and populates artifactmetadata, artifactmedia, and artifactcolors tables Analytics Queries: 20+ predefined SQL queries for artifact insights Interactive Visualization: Streamlit dashboard with Plotly charts Installation Requirements Configuration Database Setup The application expects a MySQL/TiDB database with three main tables: API Configuration Key Components 1. ETL Pipeline 2. SQL Analytics Queries 3. Streamlit Dashboard Running the Application Common Patterns Batch Processing Error Handling for API Calls Data Quality Checks Troubleshooting API Rate Limits If you encounter rate limit errors: Database Connection Issues Memory Issues with Large Datasets Streamlit Caching Advanced Usage Custom Query Builder Export to Multiple Formats
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-artifacts-data-engineering-app