← all MCP servers
AI Crawler Index
dev.workers.pathwren.www/ai-crawler-index
Identify AI crawlers by user-agent or IP, verify them against operator ranges, build robots.txt.
remote endpoints
- streamable-http https://www.pathwren.workers.dev/c/mcp-registry-official/mcp
details
- category
- browser
- version
- 1.1.0
- published
- 2026-09-01
last synced 2026-09-01