Free Tool · GEO

AI Bot Access Test

Can the bots of ChatGPT, Claude and Perplexity actually open your page? Even when robots.txt allows them, a firewall or CDN can turn a bot away at the door. The tool requests your page once with each bot's identity, compares the answer with the one a browser gets and shows where any block happens.

The page is requested from the Enar SEO server, first with a browser identity and then once with each bot's identity. Fetched content isn't stored; only the status code, title and amount of text are compared.

Why isn't a robots.txt test enough?

robots.txt is a request telling a bot “don't crawl this page”; the server is what actually opens or closes the door. Security plugins, CDNs' ready-made bot rules and server settings often reject requests whose identity contains an AI bot's name without looking at robots.txt at all. Site owners usually don't notice: everything looks normal in the browser and a robots.txt test says “allowed”, but the bot only sees an error page and the site never shows up as a source in AI answers.

What does the tool check?

  • The server's response: the page is requested with each bot's identity; the status code, any firewall or bot-verification page, redirects and the amount of text are compared with the response a browser gets.
  • robots.txt: whether the tested address is allowed under each bot's own group (or the * group if it has none), with the matching rule and line.
  • Page tags: robots meta tags such as noindex, nosnippet and max-snippet, and the X-Robots-Tag header. To be a source in its AI features (AI Overviews, AI Mode), Google requires a page to be indexed and eligible to be shown with a snippet.
  • Content-Signal: if robots.txt (for example a file managed by Cloudflare) contains preference lines about AI use, they're read.

Test edilen botlar

BotTypeOwner · why it crawlsTest
OAI-SearchBotAI search (citing sources)OpenAI · citing sources in ChatGPT searchHTTP + robots.txt
Claude-SearchBotAI search (citing sources)Anthropic · citing sources in Claude searchHTTP + robots.txt
PerplexityBotAI search (citing sources)Perplexity · citing sources in search resultsHTTP + robots.txt
ChatGPT-UserOpening pages on a user's requestOpenAI · opening a page a user asks for in ChatGPTHTTP + robots.txt
Claude-UserOpening pages on a user's requestAnthropic · opening a page a user asks for in ClaudeHTTP + robots.txt
Perplexity-UserOpening pages on a user's requestPerplexity · opening a page a user asks forHTTP + robots.txt
GPTBotModel trainingOpenAI · model trainingHTTP + robots.txt
ClaudeBotModel trainingAnthropic · model trainingHTTP + robots.txt
CCBotModel trainingCommon Crawl · open dataset (many models are trained on it)HTTP + robots.txt
Google-ExtendedModel trainingGoogle · Gemini training and grounding (doesn't affect Search)robots.txt only
Applebot-ExtendedModel trainingApple · Apple Intelligence trainingrobots.txt only

Google-Extended and Applebot-Extended aren't separate crawlers; Google and Apple collect content with their own crawlers, and these names only state a training preference in robots.txt. Because Anthropic doesn't publish full user-agent strings, the form seen in sites' access logs is used for the Claude bots and for CCBot; since firewalls recognize bots by the name in the identity, the result is still meaningful.

Frequently asked questions

robots.txt izin veriyorsa bot siteme girebilir mi?

Not always. robots.txt is only a request; the real door is kept by the server, a firewall (WAF) or a CDN. Many security plugins and CDN rules reject requests whose identity contains GPTBot, ClaudeBot or PerplexityBot with a 403 without looking at robots.txt. In that case the bot can't read your page and can't show your site as a source in AI answers.

If the result says “can access”, is that certain?

Not certain. The tool sends requests with each bot's identity, but they come from the Enar SEO server. Some services (such as Cloudflare's AI bot block) recognize the real bot by its IP address and a verified bot list, so that block isn't applied to our request. If the site is behind a CDN, the results say so; check the bot settings in your CDN dashboard as well.

Does “server blocks” always mean a bot block?

Usually, but there are two possibilities: either the site deliberately blocks this bot identity, or a “fake bot” protection rejects requests that claim to be a bot but don't come from that company's IPs. In the second case the real bot may be able to open your page. You can tell which by checking your server or CDN logs for requests from the bot's official IP list.

Does blocking model-training bots reduce my visibility?

No. GPTBot, ClaudeBot, CCBot and Google-Extended determine whether your content is used for model training. To be cited as a source in AI answers, access for the search bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot) and the user-requested agents (ChatGPT-User, Claude-User, Perplexity-User) is enough.

Can the test harm the target site?

No. Only one request is sent with each bot identity (eleven requests in total); the result is kept for 10 minutes and repeatedly testing the same site in a short time is limited. The request identity contains the tool's address, so the site's administrator can see where the test came from in their logs.

To review robots.txt rules line by line and try a change before publishing, use the Robots.txt Tester.