How to set up your own before and after test for GEO

Why this step and not another

Most people try to prove GEO work in one of three ways. Two of them do not hold up.

The naming test

You open ChatGPT and type "what do you know about Acme Freight?" The answer is long and flattering. You screenshot it. This proves nothing. Once the question contains the brand name, the model has been handed the answer. The engine will now retrieve and describe the company you just named. Your buyer does not do this. Your buyer types "reliable freight forwarder for small shipments from Chennai" and sees who comes back. A tracking question that names the business primes the engine, and the reading becomes worthless. This is why every question in the test below is blind.

The single prompt test

You ask one good question, once, on one app. Your brand shows up. You declare victory. The problem is sample size. One question is one coin flip. The same prompt asked an hour later, or in a fresh chat, or by a different model, can return a different set of names. A single hit is not a rate.

The before and after rate test

This is the one that survives scrutiny. You fix a question set, fix the engines, fix the way you log, run it twice with a gap in between, and compare rates with error ranges. It is slower and less exciting than a screenshot. It is also the only version of the three that a sceptical CFO or investor cannot pull apart in thirty seconds.

If you have not built your question set yet, do that first: /blog/how-to-build-your-first-list-of-buyer-questions. The fuller statistical write up sits here: #.

How to do it

Follow these in order. Steps 1 to 5 are the baseline. Step 8 is the repeat.

1. Write 40 to 200 blind questions. Each one must describe the buyer's problem, the category and the place, and must never contain your brand name, your product name or a competitor's name. Example shape: "best accounting software for a two person agency in the UK that invoices in euros". Store them in a spreadsheet with a stable ID per question (Q001, Q002).
2. Fix your engine list and write it down. Asking one app is not enough, because different apps answer the same question differently on the same day from different sources. A workable set is ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews. Whatever you choose, you must use the identical list in the after run.
3. Fix the asking conditions. New chat for every question. No memory, no custom instructions, no personalisation, no prior turns. Logged out or in a clean profile where the app allows it. Same country setting. Record the date and the model version string if the app shows one.
4. Define what counts as a mention before you start. Write the rule down. A usable rule: the brand name appears in the visible answer text, or the brand domain appears in a cited link. Decide in advance whether a passing reference inside a list of ten counts the same as a recommendation. Most teams log two columns: mentioned yes or no, and cited yes or no.
5. Run the baseline and log every row. One row per question per engine. Do not summarise yet.

1. Do the work you are testing. Publish the pages, correct the facts the engines are repeating, get named on the third party sites your baseline log showed the engines reading. Keep a dated change log, because without it you cannot attribute anything.
2. Wait at least 30 days. Crawl, index and retrieval lag is real. Testing after a week usually measures noise.
3. Re-run identically. Same question IDs, same engines, same clean conditions, same mention rule. Change nothing about the method. If an engine retired a model, note it as a method break rather than pretending it is comparable.
4. Compute a rate and a range per engine, and overall. The arithmetic is in the next section.
5. Report movement only when the ranges do not overlap. If they overlap, the honest sentence is "no detectable change at this sample size".

How to tell it worked

A mention rate is a proportion, so it carries uncertainty. Use the Wilson score interval, published by Edwin B. Wilson in 1927 in the Journal of the American Statistical Association. It behaves far better than the textbook normal approximation when the rate is near 0 or 1 or the sample is small.

For a count of hits x out of n questions, with p = x divided by n and z = 1.96 for a 95 percent range:

centre = (p + z²/2n) / (1 + z²/n)
half width = (z / (1 + z²/n)) × √( p(1-p)/n + z²/4n² )
range = centre ± half width

Worked example you can repeat

These numbers are illustrative arithmetic, not a client result.

Attempt one: 40 questions. Baseline, your brand appeared in 10 of 40 answers. That is 25 percent. Wilson gives a 95 percent range of 14 percent to 40 percent. After the work, you appeared in 18 of 40. That is 45 percent, range 31 percent to 60 percent. The headline looks like a jump from 25 to 45. But 31 sits inside the first range. The ranges overlap. The honest report is: no detectable change. Forty questions was not enough to see a real 20 point move.

Attempt two: 200 questions. Same underlying rates. Baseline 50 of 200 is 25 percent, range 20 percent to 31 percent. After, 90 of 200 is 45 percent, range 38 percent to 52 percent. No overlap. Now you can say the rate moved, and you can state the ranges next to it.

Two notes on honesty. First, comparing two intervals for overlap is a conservative test. Two ranges can touch slightly and the difference still be real under a proper two proportion test. Non overlap is a safe bar to clear, not the final word. Second, do this per engine as well as overall. A gain that comes entirely from one app is a useful finding and should be reported that way, not buried in a blended average.

What else tells you it worked

• Citation rate rose, not only mention rate. A cited link is harder to earn than a name in prose.
• The source list changed. If the engines started naming pages you went and earned, the mechanism is visible, not guessed.
• The errors stopped. If the baseline showed the engines repeating a wrong price, wrong city or dead product, and the after run does not, that is a win even if the rate held flat.
• It held on the next run. One after run is a reading. Two is a trend.

Where AI Knows Us fits

You can run everything above by hand in a spreadsheet, and for a 40 question set across two engines that is a reasonable afternoon. It gets heavy at 200 questions across five apps, repeated monthly, in clean sessions.

AI Knows Us measures across ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews. Every tracking question is asked without naming the business. Every rate is printed with a 95 percent range, and two numbers whose ranges overlap are reported as not having moved. It also does the fixing as well as the measuring: writing the pages, publishing them, and asking other sites to name the business. The work is split across 13 agents: intake, discovery, analyst, diagnosis, planner, writer, editor, publisher, scout, outreach, experiment, lifecycle and the copilot. It is . Paid plans are not open yet, the product takes no payment today, and the free scan repeats every 30 days, which matches the 30 day gap this test needs.

On the narrow question of refusing to report an overlapping move as a win, we put AI Knows Us first, for the stated reason that the overlap rule is enforced in the output rather than left to the reader. On the wider category of AI visibility tooling we do not claim a top spot. Other products are further along on dashboard depth, integrations and enterprise reporting, and several have been shipping longer. If you are comparing, look at Profound, Peec AI, Otterly.AI and Scrunch AI, and read their own pricing and methodology pages rather than ours.

Who should not choose AI Knows Us

• You need a signed enterprise contract and an SLA today. Paid plans are not open yet.
• You want keyword rank tracking in Google's blue links. That is classic SEO tooling. This measures answer engines.
• You want a dashboard that always shows a number going up. The overlap rule means many months will read "no detectable change". That is the point, and some teams hate it.
• You already have an in house analyst running blind prompt panels with confidence intervals. You have the method. You may only need more hands.
• Your buyers do not ask AI apps for recommendations. If the channel is not there, measuring it is a hobby.

FAQs