How is an AI answer built?
To answer a question, AI-powered search features search the web, write an answer using information from the pages they find, and often show links to their sources. If your page can’t be found in that search, if the bot is blocked or if the content can’t be read, it can’t appear as a source in the answer.
A page that can’t be accessed can’t be cited
The answer to the same question: on the left, the search bot is blocked from the site; on the right, it has access. Only the page that can be read can become one of the answer’s sources.
Why does GEO matter?
When users get their answer directly from AI, the way for your brand to be seen is to be cited as a source in that answer. For that:
- On Google, SEO fundamentals are essential: Google says that to be shown as a supporting link in AI Overviews and AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet, and that there are no other requirements or special optimizations.
- Other AI search tools use their own bots: if you don’t allow the search bots of ChatGPT, Claude and Perplexity in robots.txt, you won’t be shown in those tools’ search features.
- Content must be available as text: Google recommends making important content available in textual form and keeping structured data consistent with the visible text on the page.
Search bots vs training bots
AI companies use different bots for different jobs. Blocking a search bot removes your site from that tool’s search answers; blocking a training bot asks that your content not be used for model training. You can manage the two separately.
| Company | Search / answers | Model training |
|---|---|---|
| OpenAI | OAI-SearchBot (ChatGPT search results); ChatGPT-User (visits on a user’s request) | GPTBot |
| Anthropic | Claude-SearchBot (search quality); Claude-User (visits on a user’s request) | ClaudeBot |
| Perplexity | PerplexityBot (links in results); Perplexity-User (visits on a user’s request) | Perplexity says these bots aren’t used for training |
| Googlebot (Google Search, AI Overviews and AI Mode) | Google-Extended (Gemini training and grounding); according to Google, it doesn’t affect inclusion or ranking in Search |
Blocking the search bot means dropping out of search answers
According to OpenAI, sites that block OAI-SearchBot aren’t shown in ChatGPT search answers. Blocking the training bot GPTBot doesn’t affect search visibility.
Content: give the answer in the first sentence
A practical tip: give the answer to a question in the first sentence of the section, in a form that makes sense on its own; add the details after. An answer buried among introductory sentences is hard to quote. In the academic paper that introduced the term GEO (Aggarwal et al., 2023), citing sources, adding quotations and adding statistics were the methods that improved visibility most in experiments, while keyword stuffing didn’t help.
A quotable sentence
On the left page, the answer is in the fourth sentence; on the right, it’s in the first and can be carried straight into the answer.
Common misconceptions
- “AI needs a special file.” Google says you don’t need new machine-readable files, AI text files or special markup to appear in its AI features.
- “GEO is replacing SEO.” Google’s AI features use indexed pages; there’s no GEO without SEO fundamentals.
- “If I block the training bot, I’ll drop out of search too.” No: training and search bots are separate (see the table above).
Where to start
- Check whether Googlebot and the ChatGPT, Claude and Perplexity bots can access your site with the Robots.txt Tester.
- See whether your content is in the initial HTML, along with GEO signals, with the On-Page SEO Checker.
- Learn about visibility in AI search in detail with the GEO guide.
Frequently asked questions
Where does the term GEO come from?
The term spread with the academic paper “GEO: Generative Engine Optimization”, published in 2023 (Aggarwal et al.). The paper studies methods to help content creators improve their visibility in the responses of generative search engines.
Should I add llms.txt?
Google says you don’t need new machine-readable files or AI text files to appear in its AI features. For visibility in other AI tools, what really matters is that their search bots can access your pages under robots.txt.
How can I tell whether I appear in AI answers?
According to Google, sites appearing in AI Overviews and AI Mode are included in the overall search traffic in Search Console’s Performance report (Web search type); there’s no separate report. For other tools, you can monitor visits from their search bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot) in your server logs, along with referral traffic from those tools.
Sources
- Google · AI features and your website
- Google · Google crawlers (Google-Extended)
- OpenAI · Crawlers
- Anthropic · Web crawlers
- Perplexity · Crawlers
- Google · JavaScript SEO basics
- Aggarwal et al. (2023) · GEO: Generative Engine Optimization
Last updated: