AI search and answer engines
How ChatGPT, Claude, Perplexity and AI Overviews find and cite businesses, how to avoid blocking them by accident, and how to measure whether you appear.
2 min read · updated 24 September 2026
More people now ask an AI assistant instead of scrolling through search results: "who is a reliable roofer in Bristol?", "what should a boiler service include?". The assistant writes an answer and often names businesses and cites pages. Being in that answer is becoming as important as ranking.
How assistants get their information
AI assistants use two kinds of information:
- What the model learned in training, from a large snapshot of the web taken months or years earlier.
- What they look up now. When answering a question about current or local information, assistants search the web and read pages, then cite them.
The second is the one a business can influence quickly, and it runs on crawlers.
Two kinds of AI crawler
- Training crawlers collect pages to train future models: GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended, CCBot (Common Crawl), Applebot-Extended and others.
- Retrieval crawlers fetch a page because someone asked a question just now: OAI-SearchBot and ChatGPT-User (OpenAI), Claude-SearchBot and Claude-User (Anthropic), PerplexityBot and Perplexity-User.
Blocking training crawlers is a reasonable business decision if you do not want your content used to train models. Blocking retrieval crawlers takes you out of the answers. Many sites have pasted in a "block all AI" robots.txt and done the second without meaning to. The AI crawler checker shows which is which on your site in a few seconds.
What helps you get cited
The same things that make a page useful to a person make it easy for an assistant to quote:
- Clear, direct answers near the top of the page, before the detail.
- Specific facts: prices or price ranges, areas covered, response times, qualifications.
- Consistent business details across your site and listings, so the assistant is confident who you are. See local SEO.
- Structured data stating your business name, address, phone and services.
- Reputation elsewhere: reviews, directory listings and mentions on trusted sites are part of what assistants search.
- Pages that load without heavy scripts. Retrieval crawlers generally do not run JavaScript, so content that only appears after scripts run may be invisible to them.
Measuring it
- Referral traffic. Visits from chatgpt.com, perplexity.ai, claude.ai, gemini.google.com and copilot.microsoft.com appear in analytics. Many AI visits arrive with no referrer at all, so treat this as a minimum.
- Prompt monitoring. Ask the questions your customers ask, regularly, and record whether you are named and which pages are cited. AI visibility does this across several assistants and keeps every answer.
- Server logs show which AI crawlers actually visit and which pages they read.
A caution about certainty
Answers vary between people, sessions and days, and the consumer apps add their own context. No tool can see exactly what one person was told. Look for trends across many prompts rather than reacting to a single answer.