Website to Markdown — Bounded Crawls for Agents

io.github.kriton-dev/website-to-markdown

Crawl sites to Markdown with sitemaps, globs and depth limits, obeying robots and crawl-delay