← all MCP servers
polite-crawler
io.github.mark1russell7/polite-crawler
A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.
install
npm · stdio
npx -y polite-crawler@0.1.0
details
- category
- browser
- version
- 0.1.0
- license
- MIT
- stars
- 0
- last commit
- 2026-10-13
- published
- 2026-10-13
last synced 2026-10-14