How to Measure AI Rankings Without Faking the Data

The problem with AI ranking reports

AI search created a new kind of report: the citation report. It tells you whether ChatGPT, Perplexity, Gemini, or Claude mentioned your site. The problem is that many of these reports are not measurements at all. They are simulations — a model asked to guess what another model would say — presented with the confidence of a measurement.

A simulated report misleads in a specific way: it tells you to optimize for a mirage. You rewrite pages to fix gaps that never existed, and you celebrate citations that never happened. The work feels productive while the real rankings stay exactly where they were.

The fix is not a better model. It is a stricter definition of what counts as a measurement.

What a real measurement is

A real measurement is a recorded observation: on a specific date, for a specific question, on a specific engine, was your site cited, at what position, and with what quoted text? That is all. Nothing estimated, nothing generated, nothing inferred from a model's knowledge of your brand.

Manual recording counts. Asking the engine yourself and writing down what it said is a genuine measurement, as long as you label it manual. What does not count is a report generated without asking the engine — no matter how realistic it looks.

Step 1: Build a prompt set

Start with the questions your buyers actually ask. A good prompt set has 8 to 15 questions, mixing three kinds: "best X" questions ("best CRM for small law firms"), "how to choose" questions ("how to choose a CRM tool"), and comparison questions ("X vs alternatives"). If a question has no buyer behind it, it does not belong in the set — it will only dilute the trend.

Write the questions exactly as a buyer would type them. Do not optimize the wording for your own site; the point is to measure what buyers see, not what flatters you.

Step 2: Record a run

A run is one pass over the whole prompt set on one date. Ask each question on each engine you track, and record cited yes or no, position, and any quoted text. Record every question, including the ones where you were not cited — a run with only the wins recorded is not a run, it is a highlight reel.

One date equals one run. That discipline is what makes step 3 possible: without dated runs, you have anecdotes, not trends.

Step 3: Compare runs

After fixes (step 4), record a new run and compare it with the previous one, prompt by prompt and engine by engine. Each prompt-engine pair falls into one of six states: first measurement, new citation, lost citation, improved position, worsened position, or unchanged. The citation rate — the share of measurements where you were cited — summarizes the run in one number, but the per-prompt table is where the decisions live.

A lost citation is as informative as a new one. It tells you a page changed, a competitor improved, or an engine's answer shifted. Treat losses as work items, not as noise.

Step 4: Fix from the gaps

Fixes follow directly from the measurements. Not cited at all: the engine found no page that answers the question well, so build one — a direct answer page, an FAQ with FAQ schema, a plain definition of the core terms, or a facts page with checkable claims. Cited below first place: move the direct answer to the top of the page, ahead of background and storytelling. Cited first: leave the page alone. Rewriting a winning page is the most common self-inflicted loss in this loop.

Then re-measure. The trend table is the proof a fix worked — or the signal that it did not, and something else is wrong.

What not to do

Do not present estimates as measurements, even with a footnote. Do not buy citation reports you cannot verify against the engines yourself. Do not rewrite a page that is already cited first. And do not measure once and declare victory: one run is a snapshot, and snapshots do not show direction.

The loop

Measure, fix, re-measure. That is the whole system. It is slower than a generated report and it is honest, which is why its trends are worth acting on. See how the AI Citation Tracker approaches this, or start a free trial and run your first prompt set this week.