AI visibility check

What it checks

1. Which AI crawlers robots.txt lets in

   For 14 crawlers from OpenAI, Anthropic, Perplexity, Google, Apple, Amazon, Meta, Common Crawl and ByteDance, we read the rules for that bot and check your home page is allowed. We group them: AI search, fetches made for a person asking, and model training.

2. Whether the home page may be indexed and quoted

   A noindex, nosnippet or max-snippet:0 in a meta robots tag or the X-Robots-Tag header keeps the page out of Google’s AI Overviews and AI Mode.

3. Whether the text is in the HTML

   We count the words in the page before any script runs. Most AI crawlers do not run JavaScript, so a page that draws its text with scripts looks empty to them.

4. Who is behind the site

   We look for a title, a meta description and Organization JSON-LD with a logo and sameAs links to your profiles, which help answers name your business correctly.

5. Whether there is an llms.txt

   An optional Markdown file at /llms.txt that summarizes the site for AI tools.

It is a quick check of one page, not an audit, and it does not ask any AI model about you. Whether ChatGPT or Perplexity actually mention you is what CMSKite’s AI visibility tracking measures, prompt by prompt.

What actually gets a site into AI answers

• Being indexed and snippet-eligible is the whole technical bar for Google. Its AI features need no special markup and no AI text files; a page has to be indexed and allowed to show a snippet.
• Blocking a training bot is not blocking AI search. GPTBot and Google-Extended gather text for models; OAI-SearchBot, Claude-SearchBot and PerplexityBot find pages to cite. You can block the first group and keep the second.
• Pages that answer one question completely get cited. AI search splits a question into many smaller searches, so a page that answers a narrow question first, with sources for its numbers, is the one quoted.
• llms.txt is cheap, not a ranking lever. Google does not use it, and most AI tools do not read it yet. Add it if you like; fix the other checks first.

User-agent: GPTBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: *
Allow: /

Sitemap: https://example.com/sitemap.xml

What we fetch, and from where

The check runs on our server, as CMSKiteBot/1.0. It reads three public addresses on the standard web ports (/robots.txt, / and /llms.txt), at most 2 MB of the home page, and stops after 30 seconds. We do not store the results. Each visitor can run ten checks an hour.

Sources

• AI features and your website — Google Search Central
• Overview of OpenAI crawlers — OpenAI
• Robots meta tag, data-nosnippet and X-Robots-Tag — Google Search Central