Agent skill · data analytics · aradotso
harvard-artifacts-collection-analytics-app
Build ETL pipelines and analytics dashboards for the Harvard Art Museums API using Python, SQL, and Streamlit
Why this skill is useful
Adds executable ETL scripts and interactive dashboards for the Harvard Art Museums API that are not available in public documentation.
What it needs
Requires mysql-connector-python, pandas, plotly, python, python-dotenv, requests, streamlit installed locally. Requires harvard-art-museums account access. About 6k tokens when loaded. Last updated 2026-08-05. 5 stars on the source repository.
What this skill does
Harvard Artifacts Collection Analytics App Skill by ara.so — Data Skills collection What This Project Does The Harvard-Artifacts-Collection-Data-Engineering-Analytics-App is an end-to-end data engineering application that demonstrates real-world ETL pipelines, SQL analytics, and interactive visualization. It extracts artifact data from the Harvard Art Museums API, transforms it into relational database structures, and provides interactive analytics dashboards using Streamlit. Key capabilities: Extract artifact metadata, media, and color data from Harvard Art Museums API Transform nested JSON into normalized relational tables Load data into MySQL/TiDB Cloud databases Execute analytical SQL queries Visualize results with interactive Plotly charts Handle API pagination and rate limiting Installation Prerequisites Project Setup Environment Configuration Create a .env file with the following variables: To get a Harvard Art Museums API key: 1. Visit https://www.harvardartmuseums.org/collections/api 2. Request an API key (free) 3. Add it to your .env file Running the Application Database Schema The application creates three main tables: Code Examples 1. API Data Extraction 2. ETL Pipeline - Transform 3. Database Loading 4. Streamlit Dashboard Common Analytical Queries Troubleshooting API Rate Limiting Database Connection Issues Handling Missing Data Best Practices 1. Batch Processing: Process API data in batches to avoid memory issues 2. Error Handling: Wrap API calls and DB operations in try-except blocks 3. Environment Variables: Never hardcode credentials 4. Data Validation: Validate data before inserting into database 5. Logging: Add logging for ETL pipeline monitoring 6. Incremental Loading: Track processed artifacts to avoid duplicates
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aradotso/harvard-artifacts-collection-analytics-app