Your AI Citation Ranking Is a Distribution, Not a Number
One ChatGPT check is one draw. How much AI citations change between identical runs, and how to log repeated probes so you can spot a real move.
One AI citation check is a sample, not a ranking. Run the identical prompt twice and the engine may cite a different set of sources each time — and sometimes no sources at all. If you react to a single disappearance (or celebrate a single appearance), you're reading noise as signal. The fix is to treat your AI citation ranking as what it actually is: a distribution of outcomes across repeated runs, and to track a citation rate per engine that only counts as a real move when it holds across separate sessions.
How much answers change between identical runs
Three recent studies have tried to quantify the variance, and all three were summarized — not published in full — in a Dev.to post by Robin (2026-10-13). We have not independently opened the underlying primaries, so every figure below is as reported by that post, with the seller's conflict of interest noted — the post's author sells a citation-tracking template and ledger.
Kingy AI Citation Index, Wave One (Curtis Pyke, 2026-10-05; commissioned by Kingy.ai). The same 1,200 questions were run twice through official APIs. When the same provider was compared against itself, the overlap in cited domains (Jaccard similarity) was 0.664 for OpenAI, 0.515 for Claude, and just 0.319 for Gemini. Cross-provider overlap was around 0.06 — essentially, different engines cite nearly different webs. Caveat: the study was commissioned by a company that sells AI-visibility tooling, and an overlap of 0.66 still means a third of cited domains changed between two identical runs.
Is My Brand in AI, AI Answer Evidence Index round two (Minel Gunesoglu, 2026-10-08; the author sells an AI visibility product). On 10 questions repeated multiple times, AI Overviews re-cited 32 of 51 previously cited pages, ChatGPT re-cited 5 of 17, and Gemini 17 of 29. As reported, that means an AI Overviews citation had roughly a 63% chance of surviving to the next check — a coin flip weighted your way, not a ranking. Caveat: tiny question set, and a vendor with skin in the game.
cloro AI Search Index (freshness check 2026-10-07; cloro sells a search data API). This one measures a different axis: how often answers cite any source at all. As reported, Gemini gave answers with no sources 31.8% of the time over the 30 days to October 7, and ChatGPT's any-source rate swung from 99.6% in July to 79.6% and back to 97.6%. Caveat: a data-API vendor's own index, and a source-rate that can move 20 points month to month is itself evidence that single checks are noisy.
Three studies with three different conflicts of interest agree on one thing: variance between identical runs is large, engine-specific, and sometimes the engine cites nothing. Any of these numbers is a reason to build a logging routine, not a benchmark to quote.
Three reasons a citation disappears
When yesterday's check showed you cited and today's doesn't, exactly one of three things is true:
- The engine cited nothing that run. Per the cloro figures above, source-free answers are common enough that a "no citation" result may not be about you at all.
- It drew a different handful of sources. Identical prompts sample differently — the Kingy and Is My Brand in AI numbers both show substantial turnover in cited domains between runs.
- You actually lost the placement. Real regressions exist. But one check cannot tell you which of the three happened — and the first two are, in our experience, the most common.
How we log it at Tacavar
We run a daily GEO citation pass over a fixed keyword set per site — a fixed panel, so every day-over-day comparison is apples to apples. The probes go through four retrieval and search engines (exa, tavily, DataForSEO, and general web search), each row recording whether the site was cited, at what rank, and against which competing domains.
Be clear about what that is: these are retrieval and search APIs, a proxy for what ChatGPT, Gemini, or AI Overviews actually answer. We don't probe the answer engines' apps directly. Proxies correlate with answer-engine behavior, imperfectly — which is exactly why the routine below exists.
The core rule: every day-over-day loss is re-probed live on all four engines before anyone acts. The reason is rotation noise. Each keyword is typically sampled by one engine per day from the ring, so a keyword whose carrying engine doesn't come up today looks "lost" without ever being re-tested. In our logs, the overwhelming majority of apparent losses re-cite on the re-probe — on 2026-10-14, 14 of 15 day-over-day losses re-cited and were logged as engine rotation, not a content regression; on 2026-10-15, 9 of 10 did.
Two standing guards keep real regressions from being buried in that noise:
- WATCH rule: a keyword dark on all four engines in a single day goes on a watch list and is re-probed the next day — not "fixed." Two consecutive dark days, confirmed across engines, is the threshold for treating it as real.
- 48-hour window: a recently edited page gets roughly two days of index-propagation grace before any dark reading counts. Search indexes lag deploys by about that much, and a fix-tweet panic on day one measures the index, not the content.
A worked example from our own tracker: the query "langgraph vs crewai for production multi-agent systems" against our comparison post. That keyword has been probed 79 times since late July. It was cited exactly once before August 31 — at rank 9 on one engine, against a different URL — then went uncited again. Then on October 8 it came back at rank 1 on exa for the dedicated post, while the other three engines' rows that day did not cite it at all. Three readings: most probes miss, one early hit was a different surface, and the "win" was real but engine-specific. A single check at any point in that arc would have told a completely wrong story.
The same "don't trust one reading" logic applies to model regressions, not just citations — see /blog/silent-llm-regression-tripwires/.
A logging template you can copy
Whether you use a tracker or a spreadsheet, log per run, not per day:
| Column | What it captures |
|---|---|
| date, engine, prompt id, run # | Which draw this is — engine and repeat number must never be collapsed |
| any-citation | Did the answer cite anything at all? (The cloro lesson: "no" may mean nothing.) |
| named / cited | Was your brand named in prose, merely listed as a source, or neither? These are different outcomes. |
| rank | Position in the cited list, if cited |
| figures-correct | If your numbers were quoted, were they right? |
| competing domains | Who else got cited — this is your real competitive set, and it rotates |
Report the result as "cited in X of N runs" per engine per prompt — a rate — never as a single position. Keep engines separate; never average ChatGPT and Gemini behavior into one blended score, because the Kingy overlap figures (as reported) show the engines are citing nearly disjoint sets of sources.
When to call it a real move
Our recommendation, distilled from the routine above:
- Same direction, twice, in different sessions. A move counts when ≥2 fresh runs across ≥2 separate sessions agree. One session can be a bad index day.
- Per engine, never blended. A Gemini loss with a stable ChatGPT rate is a Gemini event.
- Wait out the post-edit window. No citation verdicts for ~48 hours after publishing or significantly editing a page.
If all three conditions hold and the rate is still down, then you have a regression worth investigating — and even then, check the cited URL: engines sometimes shift which of your pages they cite before they stop citing you at all. That's the pattern behind much of what we cover in /blog/ai-search-visibility-agencies-2026/ and /blog/who-ai-engines-cite-wound-care-2026/: visibility snapshots look stable week to week only because they aggregate many noisy draws.
If you want a baseline read on where you stand before building a log, run the free GEO scan at tacavar.com/scan/.
What citation logs can't tell you
Even a perfect citation log measures visibility, not visits. A citation that never gets clicked produces no traffic, and AI referrals behave differently from search referrals. Track referral traffic separately — UTM tagging on any links you can influence, and segment AI-referred sessions in analytics. The citation rate tells you the engine knows you exist; only the referral log tells you whether that's worth anything.
FAQ
Why did ChatGPT cite my site once and then stop? Most likely, nothing changed. Identical prompts sample different sources between runs — one check is one draw. Re-run the same prompt several times across separate sessions and look at the cited-rate, not the single result, before concluding you lost anything.
How many times should I run the same prompt before trusting the result? Enough to compute a rate: at minimum two fresh runs per engine per session, and only trust a direction that holds across two separate sessions. Two agreeing runs in one sitting still share that session's conditions.
Should I combine ChatGPT, Gemini and Perplexity into one AI visibility score? No. Third-party overlap data (as reported by the DEV post summarized above) shows different engines cite substantially different sources — a blended score averages away exactly the information you need. Track per-engine rates.
What should an AI citation log include? Date, engine, prompt id, run number, any-citation, named-vs-cited, rank, figures-correct, and the competing domains cited alongside you. Report "cited in X of N runs" per engine.
How long after editing a page should I wait before re-checking AI citations? About 48 hours. Search and retrieval indexes lag deploys, and a dark reading inside that window usually measures index propagation, not your content.
Does being cited by an AI engine mean I'll get traffic? Not necessarily. Citation rate measures visibility; clicks depend on whether the engine's users follow sources. Measure referral traffic separately from citations.
Out of scope for this post: ranking guarantees (nobody can promise a citation rate) and tool reviews. The method is the point.