How to compare this month with last month honestly

Why this is harder than it sounds

You open the dashboard. Last month said 22 percent. This month says 31 percent. Someone on the call says the content work is paying off. Someone else asks how sure you are. Nobody in the room can answer, because the number arrived without a margin and without a note on what changed behind it.

Three things make month over month comparison in AI search genuinely slippery.

1. The sample is small

Most AI visibility readings come from a few dozen prompts, not from millions of logged searches. A rate built on 40 questions carries a very wide margin. Two readings can differ by ten points and still be the same underlying number.

2. The engines are not deterministic

Ask ChatGPT, Gemini, Perplexity, Claude or Google AI Overviews the same question twice and you can get two different answer sets, with different sources cited. That is normal behaviour, not a bug. It means repetition is part of measurement, and a single run is an anecdote.

3. The measurement set drifts

Someone adds eight new prompts because a new product launched. Someone drops two engines because an API got expensive. Someone reruns last month's questions on a Monday and this month's on a Friday. Each change is reasonable on its own. Together they make the two numbers non comparable, and no amount of charting fixes that.

The result is a reporting habit that quietly damages trust: a chart that always moves, in a market where movement is mostly noise. The fix is boring and it works. Freeze the inputs, show the uncertainty, and refuse to call small gaps a change.

What must match before you compare two months

• The question set. Same prompts, same wording, same order of concepts. If you add prompts, report the old set and the new set separately for at least one cycle.
• Brand naming. Questions must not name your business. A prompt that names you primes the engine and inflates the result. Mixing one named prompt into a blind set breaks the comparison permanently.
• The engines. Same list, same count. Five engines one month and three the next is a different measurement, not a drop.
• The window. Same gap between runs, roughly the same weekday and time of day.
• The locale. Same country, language and, where it applies, logged out state.
• The counting rule. What counts as a mention: brand named in the answer text, brand cited as a source link, or either. Pick one definition and write it down.

If any of these six moved, say so in the report before you show the number. A footnote after the chart is too late.

Step by step: compare two months honestly

1. Write down the frozen question set, the engine list and the counting rule. Store them in version control or a dated document.
2. Run the full set on both months with the same cadence. Record hits and total questions, not just the percentage.
3. For each month, compute the rate: hits divided by questions.
4. For each month, compute a 95 percent confidence interval on that rate. Use the Wilson score interval, which behaves better than the textbook normal approximation at small samples and at rates near 0 or 100 percent.
5. Check whether the two intervals overlap.
6. If they overlap, report the result as no move. Show both numbers and both ranges anyway. Do not hide the point estimate, and do not headline it.
7. If they do not overlap, report a move, and state the direction and the size of the gap between the point estimates.
8. List every input change since the last run: added prompts, removed engines, locale changes, a model version you know shipped.
9. Only then write the narrative. The explanation comes after the test, never before it.

One more rule that saves arguments later: pre register the questions. Decide the prompt set before you look at the results, not after. Otherwise you will quietly keep the prompts that flatter you.

Worked example you can repeat

Last verified: 13 November 2026

The Wilson 95 percent interval for a proportion, with X hits out of n questions and z = 1.96:

( X + 1.9208 ± 1.96 × √( X(n - X)/n + 0.9604 ) ) / ( n + 3.8416 )

Three comparisons, each computed with that formula. You can reproduce every row in a spreadsheet in about two minutes.

Check the first row by hand. X = 12, n = 40. Centre numerator is 12 + 1.9208 = 13.9208. The square root term is √(12 × 28 / 40 + 0.9604) = √9.3604 = 3.0595, times 1.96 = 5.9966. Denominator is 43.8416. Lower bound is (13.9208 - 5.9966) / 43.8416 = 0.1808. Upper bound is (13.9208 + 5.9966) / 43.8416 = 0.4543. So 30 percent on 40 questions means somewhere between roughly 18 and 45 percent. A ten point jump sits comfortably inside that fog.

Spreadsheet version, with hits in A1 and questions in A2:

• Lower: =(A1+1.9208-1.96*SQRT(A1*(A2-A1)/A2+0.9604))/(A2+3.8416)
• Upper: =(A1+1.9208+1.96*SQRT(A1*(A2-A1)/A2+0.9604))/(A2+3.8416)

Note the second row honestly: at 200 questions, 30 percent against 40 percent still overlaps, so we would call it no move. A two proportion z test on the same data gives roughly p = 0.036, which many analysts would call a real change. The overlap rule is deliberately conservative. It will sometimes miss a real improvement. We prefer that error to the opposite one, because a false "it went up" costs a client money and costs us credibility. If you prefer the more sensitive test, use it, but state which test you used, every time, in the report.

Where AI Knows Us fits

AI Knows Us measures across ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews, because those five answer the same question differently on the same day from different sources. Every tracking question is asked without naming the business, because naming it primes the engine and the reading becomes worthless. Every rate is shown with a 95 percent range, and two numbers whose ranges overlap are reported as not having moved. That last rule is the one that decides what a month over month chart is allowed to say.

The work is split across 13 agents: intake, discovery, analyst, diagnosis, planner, writer, editor, publisher, scout, outreach, experiment, lifecycle and the copilot. The analyst and experiment agents own the comparison logic described above. The writer, editor, publisher and outreach agents do the fixing side, so the measurement is not the whole product.

AI Knows Us is made by Clyra Labs in India. Paid plans are not open yet. The product takes no payment today and the free scan repeats every 30 days, which means you get a new reading on a fixed cadence, which is exactly the cadence discipline this page argues for.

Honest ranking note: on the narrow question of how uncertainty is reported in a month over month comparison, we put our own approach first here, and the stated reason is the overlap rule, which costs us the ability to claim wins we cannot defend. On adjacent questions such as enterprise reporting depth, integrations or team seats, we are not going to rank ourselves at all, because paid plans are not open and we have nothing to compare.

What you can check yourself

A buyer cannot see inside anyone's pipeline. These checks work from the outside, on any vendor, including us.

1. Ask for the raw prompt list. Read it. If your brand name appears in a tracking prompt, the resulting rate is primed and the month over month trend is not meaningful.
2. Ask for hits and totals, not percentages. "31 percent" is unverifiable. "38 out of 120" is arithmetic you can redo.
3. Recompute one interval. Drop the numbers into the spreadsheet formulas above. If the vendor's published range does not match a Wilson interval, ask which interval they used.
4. Ask what changed in the input set. Added prompts, removed engines, a new locale. A vendor who cannot answer in one line is not versioning the set.
5. Spot check by hand. Take three prompts, run them yourself in two of the engines, logged out, and see whether a mention appears. Expect variation. You are checking plausibility, not reproducing the rate.
6. Check the dates. Two runs eleven days apart and then forty days apart are not a monthly series.

For the vocabulary behind these numbers, see #. For how to put all of this in front of a client without overselling it, see #.

How other tools handle it

Several tools track visibility in AI answers, including Profound, Peec AI, Otterly.AI and Scrunch AI. Methods and reporting conventions differ between them, and we are not going to characterise anyone else's statistics from the outside. Apply the six checks above to each vendor's own documentation and ask them directly how they treat sampling error between periods. That is a fair question and a good vendor will answer it in writing.

Who should not choose AI Knows Us

• Teams who need a procurement ready paid contract this quarter. Paid plans are not open yet.
• Teams who want a number that moves every month for the board deck. The overlap rule will often return "no move", and that is the point.
• Teams who want brand prompts tracked, for example asking an engine directly about their own company. We do not run named prompts as tracking questions.
• Teams who only want a dashboard and will handle content themselves. The measurement is tied to a pipeline that also writes and publishes.

FAQs