← all MCP servers
io.github.jmshinhwa/robots-txt-ai-crawler-audit
Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
install
npm · stdio
npx -y @readystack/robots-txt-ai-crawler-audit@0.1.1
details
- category
- browser
- version
- 0.1.1
- published
- 2026-09-13
last synced 2026-09-13