Agent skill · data analytics · aiskillstore
robots-sitemap-validator
Check robots.txt and sitemap.xml for crawl-blocking mistakes. Use when the user says "check my robots.txt", "validate my sitemap", or "is my site crawlable".
Why this skill is useful
Adds a script for validating robots.txt and sitemap.xml files that the AI wouldn't reliably generate on its own.
What it needs
Requires python installed locally. About 1k tokens when loaded. Last updated 2026-08-07. 405 stars on the source repository.
What this skill does
Robots Sitemap Validator Detect common local robots.txt and sitemap.xml mistakes that can hurt crawling. This detector does not fetch, submit, or change live sites. Workflow 1. Run scripts/scanrobotssitemap.py with a folder, files, or --stdin. 2. Review findings by rule id, severity, file, and line. 3. Treat missing sibling sitemap files as review flags when robots.txt references them. Guardrails Read local files or stdin only. Do not fetch live URLs or submit sitemaps. Do not modify robots.txt or sitemap files. Verification Run tests/testrobotssitemap.py. Risky fixture must produce findings and clean fixture must produce zero findings. Commercial Terms Price: $0 / 0 credits. Free commercial use for personal or internal team workflows. Do not resell or redistribute the package as-is.
How to use it
Reference it in AdaL, Claude Code, Cursor or any coding agent — nothing to install:
@skills aiskillstore/robots-sitemap-validator