Agent skill · software engineering · jeremylongshore
firecrawl-reference-architecture
Design a production Firecrawl v2 ingestion system with policy gateway, bounded acquisition, validation, provenance, storage, indexing, observability, and deletion. Use when building a reusable platform. Trigger with "Firecrawl reference architecture", "Firecrawl ingestion system", or "Firecrawl RAG pipeline".
Why this skill is useful
Adds executable scripts for Firecrawl integration that provide specific content ingestion capabilities not commonly found in public documentation.
What it needs
Requires @mendable/firecrawl-js installed locally. Requires firecrawl account access. About 2k tokens when loaded. Last updated 2026-08-07. 2,604 stars on the source repository.
What this skill does
Firecrawl Governed Ingestion Architecture Overview Create explicit trust boundaries from request intake through deletion. Keep source discovery, content retrieval, validation, and downstream publication independently retryable and auditable. Prerequisites The target repository or integration path and the requested operator outcome. The source authorization, data classification, and environment policy. Current Firecrawl documentation, credentials only when needed, and an owner for approvals. Current Contract Firecrawl v2 provides scrape, crawl, map, search, batch scrape, parse, JSON extraction, agentic, and browser surfaces with different async, credit, retention, and availability contracts. The SDK can auto-wait and paginate, while explicit start/status methods support durable orchestration. Provider webhooks supplement but do not replace reconciliation. Authentication For authenticated Cloud operations, inject FIRECRAWLAPIKEY from an approved secret manager. REST requests use Authorization: Bearer with the key. Never print, commit, transmit, or place a key in a URL. Keyless access is suitable only where the current documentation explicitly allows it and the workload accepts its limits; production workflows should make identity and team ownership explicit. Instructions 1. Define tenants, source authorization, freshness, scale, formats, data classification, SLO/RPO/RTO, retention, deletion, and cost constraints. 2. Place authentication, tenant isolation, URL canonicalization, policy versioning, endpoint/format allowlists, budgets, and idempotency at the intake gateway. 3. Separate discovery into map/search, retrieval into scrape/batch/crawl/parse, and model extraction into a validated stage. Use durable job records for asynchronous work. 4. Persist normalized provenance and job/page state before downstream writes. Complete pagination and deduplicate by tenant, canonical source, version, and content hash. 5. …
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills jeremylongshore/firecrawl-reference-architecture