Agent skill · writing content · aiskillstore
firecrawl-crawl
Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction.
Why this skill is useful
Provides specific commands and options for bulk web content extraction that the AI wouldn't reliably generate on its own.
What it needs
Requires firecrawl installed locally. About 1k tokens when loaded. Last updated 2026-08-07. 405 stars on the source repository.
What this skill does
firecrawl crawl Bulk extract content from a website. Crawls pages following links up to a depth/limit. When to use You need content from many pages on a site (e.g., all /docs/) You want to extract an entire site section Step 4 in the workflow escalation pattern: search → scrape → map → crawl → interact Quick start Options Option Description ------------------------- ------------------------------------------- --wait Wait for crawl to complete before returning --progress Show progress while waiting --limit <n> Max pages to crawl --max-depth <n> Max link depth to follow --include-paths <paths> Only crawl URLs matching these paths --exclude-paths <paths> Skip URLs matching these paths --delay <ms> Delay between requests --max-concurrency <n> Max parallel crawl workers --pretty Pretty print JSON output -o, --output <path> Output file path Tips Always use --wait when you need the results immediately. Without it, crawl returns a job ID for async polling. Use --include-paths to scope the crawl — don't crawl an entire site when you only need one section. Crawl consumes credits per page. Check firecrawl credit-usage before large crawls. See also firecrawl-scrape — scrape individual pages firecrawl-map — discover URLs before deciding to crawl firecrawl-download — download site to local files (uses map + scrape)
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aiskillstore/firecrawl-crawl