No Rank #1 in ChatGPT: Measure AI Visibility in Percentages

Ask ChatGPT the same question twice and you will usually get a different list of brands. In a study published in January 2026 by Rand Fishkin of SparkToro and Patrick O'Donnell of Gumshoe, there was less than a 1-in-100 chance that ChatGPT or Google's AI returned the same brand list across repeated runs of the same prompt. The chance of the same list in the same order was under 0.1%.

So a single screenshot that shows your brand "ranked first in ChatGPT" proves very little. What can be measured reliably is how often you appear across many runs.

What the study tested

• 2,961 prompts across ChatGPT, Claude and Google AI Overviews (AI Mode where no Overview appeared).
• Twelve brand-recommendation prompts, each run 60 to 100 times per platform, covering categories such as chef's knives, headphones, cancer care hospitals, digital marketing consultants and science fiction novels.
• 142 people wrote their own prompt asking for headphones for a travelling family member. Their prompts shared almost no wording, with an average semantic similarity of 0.081.

Two disclosures from the authors are worth keeping in mind: Gumshoe sells AI tracking software, and the report is not peer-reviewed. Fishkin has called for larger follow-up studies.

What stayed stable

Lists and orders changed almost every time. Presence did not. Even though the 142 people worded their headphone prompts very differently, Bose, Sony, Sennheiser and Apple appeared in 55% to 77% of the 994 headphone answers.

The study also found that a brand's visibility percentage across many runs was more consistent than its position in any one answer. Tight categories, such as cloud computing providers, produced the same leading brands in most answers. Broad ones, such as science fiction novels, were far more scattered.

What this means for how you measure

We agree with the study's main point: "position in AI" as a single rank is the wrong number. This is how we measure instead, and how we would suggest any team measures.

• Count presence, not rank. Report Brand Presence, the percentage of answers that mention you. Where you appear inside an answer still matters to the person reading it, but only as an average across many runs, never as one "position".
• Report your share against competitors. Share of Answer = your brand's mentions ÷ mentions of every tracked brand × 100. Worked example.
• Cover different phrasings of the same intent. Real buyers word questions differently, as the 142-person test showed. A prompt set should include several phrasings of each buying question, and keep that wording fixed from run to run so the trend is comparable.
• Read trends, not single runs. One run is a sample of one. A movement that holds across repeated runs is a real change; a one-day dip is often noise. We wrote about separating the two in How EvidentlyAEO Measures KPI Deltas.

Five questions to ask any AI visibility tool

Fishkin's advice to marketers was to make sure a provider "shows their math". These five questions do that:

• How many times is each prompt run, and on which AI engines?
• Do the prompts stay the same between runs?
• Does the tool report a "rank", or a share of answers?
• Can you see the raw answers behind every number?
• How does it tell normal variation from a real change?

If a tool cannot answer them, its numbers are hard to trust, including ours. For the definitions we use, see What Is Answer Engine Optimization?

Sources

• Search Engine Journal, AI Recommendations Change With Nearly Every Query: SparkToro, 30 January 2026.
• Search Engine Land, AI recommendation lists repeat less than 1% of the time: Study.