janibert1/robots-check

See what your site's robots.txt actually serves -- including whether Cloudflare's AI Crawl Control is silently overriding it.

1 best points · 0 comments · 1 Hacker News post · 1 GitHub stars · View on GitHub

Hacker News discussions

• Robots-check: see what your robots.txt is serving (Cloudflare AI Crawl Control) (1 points, 0 comments, by janibert09, 2026-09-09)