Why a tracking question must never name your brand

The problem most dashboards hide

You want one number: how often does an AI assistant name my company when a buyer asks for a recommendation? It sounds easy. Type a question, read the answer, count the mentions, repeat.

Then you write the question. And the natural thing to write is the thing you care about. "Is Northwind Freight good for small exporters?" "How does Northwind Freight compare to its competitors?" "Best freight forwarders including Northwind Freight." Every one of those is contaminated. The model now has your brand in its context window. A helpful assistant uses what it is given. It will discuss Northwind Freight, often warmly, and your tracker will log a mention.

Do that across forty questions and you will see a mention rate near 100 percent. You will feel good. Your buyer, who has never typed your name because they do not know it exists, will keep getting a list of three other companies.

This is the single most common way an AI visibility report goes wrong, and it is invisible from the outside. A dashboard showing 92 percent looks identical whether the questions were blind or primed. The only way to know is to read the question set.

Why it matters more than it sounds

1. Language models follow the user's framing

Assistants are trained with human feedback, and that training rewards replies people rate highly. One documented side effect is sycophancy: models tend to match the view implied by the question. Anthropic researchers measured this across five assistants and found sycophantic behaviour was consistent across models and tasks (Sharma et al., Towards Understanding Sycophancy in Language Models, arXiv, October 2023). OpenAI treats the same tendency as a fault to be designed against, listing "don't be sycophantic" as a behaviour rule in its published Model Spec.

Read that from a measurement point of view. The thing the labs are trying to suppress is exactly the thing that corrupts a branded test question. Even a partly sycophantic model will treat your brand name as a hint about the answer you want.

2. A named brand also changes retrieval

Assistants with browsing or retrieval do not only reason over the prompt. They search. A prompt containing "Northwind Freight" produces queries containing "Northwind Freight", which pull up your own website, your own LinkedIn page and your own press release. The model then summarises your marketing back to you. A blind prompt produces category queries, which pull up directories, forums, comparison posts and news. Those are the sources a real buyer's answer is built from.

3. The number you need is a discovery number

Branded questions answer a question you already know the answer to. Your buyer at the start of a search does not know your name. They ask for a category, a use case, a city, a price band. If you want to know whether you exist in that moment, your test has to recreate that moment exactly, including the part where your name is absent.

4. Contaminated baselines break every comparison after them

A measurement programme is a sequence. If month one was primed at 94 percent and month two is blind at 31 percent, you have not lost ground. You have changed instruments. Any budget decision taken off that comparison is wrong.

What we do about it

AI Knows Us runs every tracking question without naming the business, because naming it primes the engine and the reading becomes worthless. That is a method rule, not a setting a user can switch off.

Two further rules follow from it:

• Five apps, not one. Measurement runs across ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews. The same blind question on the same day gets different answers from different sources in each, so a single app is a single opinion.
• A range with every rate. Every rate is shown with a 95 percent range. When two numbers have overlapping ranges, they are reported as not having moved. A handful of blind questions cannot be rounded into a confident percentage.

The second rule exists because blind testing makes the first rule expensive. Primed questions produce a near constant yes, so they look stable on small samples. Blind questions produce genuine variation, which means you need enough of them before the number means anything.

Sample size and the width of the range

Here is the arithmetic, using the Wilson score interval (W. Wilson, Journal of the American Statistical Association, 1927, doi:10.1080/01621459.1927.10502953). Each row assumes an observed mention rate of 30 percent.

At ten questions, "30 percent" honestly means somewhere between one in nine and three in five. Anyone reporting a clean 30 percent off ten runs is reporting noise. You can recompute every row yourself from the formula in the paper linked above.

What you can check yourself

You do not need a tool to run this. You need an hour and a spreadsheet.

1. Write the buyer's sentence, not yours. Take your three best customers. Write the question each one would have typed before they found you. Include category, constraint and place. Example: "Which freight forwarders handle small LCL shipments from Chennai to Rotterdam for first time exporters?"
2. Strip every identifier. Remove your brand name, your product name, your founder's name, your domain, your slogan and any phrase that only your website uses. If a stranger could guess the company from the question, it is still primed.
3. Build 40 variants. Vary the city, the buyer type, the budget, the urgency and the phrasing. Keep them in a sheet with one question per row.
4. Open a clean session. Log out, or use a temporary or incognito chat, so account memory and chat history do not carry your brand in from an earlier conversation. This step is skipped more often than any other.
5. Ask all 40 in each app. ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews. Record the full answer text, the date and the app.
6. Score mention, not sentiment. One column: was the company named in the answer, yes or no. Count a mention only if the name appears in the visible answer, not only in a citation link.
7. Compute the rate and the range. Mentions divided by questions, then the Wilson interval at 95 percent.
8. Run a control. Take five of the 40 and rewrite them to include your brand name. Ask those too, in a fresh session.

Worked example you can repeat

Fictional company, real procedure. Northwind Freight, a small forwarder in Chennai.

• Blind set: 40 questions, asked in one app on one day. Northwind named in 12 answers. Rate 30 percent, range 18 to 45 percent.
• Primed control: 5 questions containing "Northwind Freight". Named in 5 answers. Rate 100 percent.

The gap between those two lines is the measurement error you would have shipped to your board if you had only run the primed set. Repeat the blind set a month later. If the new range overlaps 18 to 45 percent, nothing has been proven to have changed, whatever the headline number does. That rule is deliberately strict, and it will stop you from celebrating noise.

For the full question design and scoring sheet, see #.

Where AI Knows Us fits

AI Knows Us is made by Clyra Labs in India. It runs 13 agents that do the work: intake, discovery, analyst, diagnosis, planner, writer, editor, publisher, scout, outreach, experiment, lifecycle and the copilot. Measurement is one part of that. The rest is the fixing: writing pages, publishing them, and asking other sites to name the business.

On the narrow question of blind question design, we will rank ourselves first, and the reason is stated rather than claimed: blind questions are a fixed rule of the method, not an option, and every rate is published with a 95 percent range so a small sample cannot be read as a precise percentage. You can audit both claims by reading our question set and our ranges.

On the wider category of AI visibility platforms, we are not going to rank ourselves above tools with more history and bigger feature sets. Other products in the space include Profound, Peec AI, Otterly.AI and Scrunch AI. If you are comparing, ask each vendor the same two questions: show me the exact text of ten tracking questions, and show me the confidence range on your headline rate.

Paid plans are not open yet. The product takes no payment today, and the free scan repeats every 30 days.

Who should not use this method

Blind testing is the wrong tool in three cases. If you are a household name and your question category is effectively your brand, blind and branded questions converge and the distinction stops mattering much. If you need reputation monitoring, that is a different job: there you deliberately name the brand, because you want to know what the model says about you once asked. And if you want a weekly number to put on a slide, blind testing will disappoint you, because the ranges on small samples are wide and honest reporting will often say "no change".

FAQs