See what your robots.txt tells AI agents.

Enter your domain. We read your robots.txt and show which of 15 AI bots it lets in and which it blocks, with a grade from A to F.

The check is free, and you don’t install anything.

The check looks for these agents.

Each kind visits for a different reason, so a robots.txt can treat them differently.

71%Among top-10,000 sites that block Anthropic’s training crawler, 71% have no rule for Anthropic’s bots that answer people’s questions.Sitedex top-10,000 robots.txt study, July 2026

Agents working for a person

They fetch a page when someone asks ChatGPT, Claude or Perplexity about it.

  • ChatGPT
  • Claude
  • Perplexity

AI search

They index your pages so AI answers can cite and link to you.

  • ChatGPT search
  • Claude search

AI training crawlers

They copy pages in bulk to train models.

  • GPTBot
  • ClaudeBot
  • Anthropic AI crawler
  • PerplexityBot
  • Google AI training
  • Apple AI training
  • Meta AI crawler
  • Amazonbot
  • Bytespider
  • CCBot

The grade reads the rules a site declares in robots.txt for 15 AI bots, the same set as Sitedex’s July 2026 study of the top 10,000 sites (CC BY 4.0). It covers only what the file says. It can’t tell you what your site enforces or which agents actually visit. Read the study.