JaydenYoonZK

JaydenYoonZK/ai-crawler-audit

See which AI crawlers your robots.txt allows or blocks (GPTBot, ClaudeBot, PerplexityBot, and 14 more), and generate a policy you actually chose.

0 stars 0 forks 0 watchers JavaScript MIT License
ai-crawlers browser-tool claudebot cli content-rights gptbot llms-txt robots-txt seo web-crawlers
View on GitHub Website
1 Open Issue Need Help Last updated: Jul 6, 2026

Open Issues Need Help

View All on GitHub
Add a WHMCS and WordPress how-to for applying the generated policy 3 months ago
documentation good first issue
JaydenYoonZK/ai-crawler-audit
0

See which AI crawlers your robots.txt allows or blocks (GPTBot, ClaudeBot, PerplexityBot, and 14 more), and generate a policy you actually chose.

JavaScript
#ai-crawlers#browser-tool#claudebot#cli#content-rights#gptbot#llms-txt#robots-txt#seo#web-crawlers