Why a failed request is not a miss
A timeout is missing data. A miss is the engine answering and not naming you. Mixing the two quietly bends every number you report.
Published by AI Knows Us · Updated 13 November 2026
The short answer
A failed request is missing data, not a miss. If the engine never returned an answer (timeout, rate limit, server error), that attempt has no result and must leave the denominator. A miss is different: the engine answered, and your business was not named. Count errors as misses and your visibility rate drifts down whenever an engine is unstable. The honest method is to report attempts, answered attempts and mentions as three separate numbers on every run.
This page is published by AI Knows Us, which measures and improves how AI assistants describe businesses. We sell this, so treat the method claims here as something to test, not something to trust. The last section tells you how to test them.
The four things that can happen when you ask an engine a question
Every tracking run is a loop: ask a question, read the answer, look for the business. Most tools report one number out of that loop, a percentage. The problem is that four quite different outcomes get squeezed into it.
Last verified: 12 November 2026
| Outcome | What happened | Counts as a mention? | In the denominator? |
|---|---|---|---|
| Answered, named | The engine replied and named the business | Yes | Yes |
| Answered, not named | The engine replied and named other businesses, or none | No | Yes |
| Refused | The engine replied but declined to recommend anyone | No | Yes, flagged separately |
| Failed | Timeout, rate limit, server error, empty body, cut-off response | No | No. Retry on a later run |
The split that matters is the last row against the second row. Both look like "we did not get a mention". Only one of them is evidence.
Refusals are data, failures are not
A refusal is the engine behaving. If you ask "who should I use for X" and the assistant says it cannot make recommendations, that is a real reading of how that assistant treats that question on that day. It belongs in the denominator, and it should be reported on its own line, because a rising refusal rate is a market signal, not a bug.
A transport failure is the opposite. HTTP 429 ("Too Many Requests") is defined in RFC 6585, and the 5xx server error class in RFC 9110. Neither tells you anything about your brand. They tell you about the pipe. Each model vendor also publishes its own list: OpenAI error codes, Anthropic API errors, and Google Gemini API troubleshooting. Those pages are the primary source for what a given code means. Read them before you let a tracker turn a code into a zero.
One case that looks like an error and is not
Google AI Overviews does not appear for every query. If you ask a question and no AI Overview is shown, that is not a failed request. The page loaded. The answer state is "no overview for this query". That is a result and it belongs in the denominator as a non-mention, with the no-overview rate reported on its own. Treating it as an error would hide the most useful fact: Google decided this question did not need an AI answer.
Why it matters more than it sounds
The usual objection is that errors are rare, so the bias is tiny. Two things make that wrong.
First, errors are not spread evenly. They cluster by engine, by time of day and by question type. A rate limit hits one provider, not all five. A long, multi-part question is more likely to be cut off than a short one. So the bias is not noise that cancels out. It pushes down the engines and the questions that were hardest to reach, which are often the ones you most want to watch.
Second, the bias moves between runs. If an engine failed on 2 percent of attempts in October and 14 percent in November, and failures were scored as misses, your visibility "fell" without anything changing in the answers. You would then spend a month fixing a problem that does not exist.
A worked example you can repeat
These numbers are made up for teaching. They are not a customer and not a published result. Take 50 blind questions asked once each on five engines, so 250 attempts.
| Engine | Attempts | Answered | Failed | Mentions | Rate on answered | Rate if failures counted as misses |
|---|---|---|---|---|---|---|
| ChatGPT | 50 | 49 | 1 | 16 | 32.7% | 32.0% |
| Gemini | 50 | 44 | 6 | 9 | 20.5% | 18.0% |
| Perplexity | 50 | 50 | 0 | 21 | 42.0% | 42.0% |
| Claude | 50 | 47 | 3 | 8 | 17.0% | 16.0% |
| Google AI Overviews | 50 | 42 | 8 | 7 | 16.7% | 14.0% |
| All | 250 | 232 | 18 | 61 | 26.3% | 24.4% |
Work the headline number yourself: 61 divided by 232 is 0.2629, so 26.3 percent. Divide instead by 250 and you get 24.4 percent. The gap is 1.9 points at the top level.
Now look at Gemini on its own. On answered attempts it is 9 divided by 44, or 20.5 percent. Counting the six failures as misses gives 9 divided by 50, or 18.0 percent. That is a 2.5 point swing created entirely by a rate limit.
Here is the part that keeps people honest. A 95 percent interval on 9 out of 44 runs from roughly 11 percent to 35 percent. The interval is about 23 points wide. The error bias is 2.5 points, so it sits well inside the interval and you will never see it in a single run. It is still real, it still has a direction, and across six monthly runs it bends the trend line one way only. Sampling noise cancels. Systematic bias does not.
What we do about it
The rules below are the ones this page argues for, and they are the ones our reporting design requires.
- Record every attempt with a timestamp, the engine, the question and the raw response, including the failure.
- Classify each attempt into one of the four outcomes in the table above before any percentage is calculated.
- Use answered attempts as the denominator. Never let a failure silently become a zero.
- Retry failed attempts on a later run rather than immediately, so a retry does not land inside the same outage.
- Publish the attempt count and the answered count next to every rate, so the reader can redo the division.
- Report refusal rate and no-overview rate as their own lines, not folded into misses.
Two verified parts of our method back this up. Every rate we show carries a 95 percent range, and when two readings have ranges that overlap we report that as no movement, not as a rise or a fall. That rule is what stops a bad week on one engine from being sold to you as a trend. We also ask five AI apps, not one (ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews), because they answer the same question differently on the same day from different sources, so an outage at one provider does not take the whole reading with it.
The third part is the questions themselves. Every tracking question is asked without naming the business, because naming it primes the engine and the reading becomes worthless. Blind questions are longer and more open, which makes cut-off answers more likely, which is exactly why the error rule has to be written down. More on that in #, and on run frequency in #.
Where AI Knows Us fits
AI Knows Us measures across ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews, asks every question blind, and prints a 95 percent range with each rate. It is made by Clyra Labs in India and runs on 13 agents that do the work: intake, discovery, analyst, diagnosis, planner, writer, editor, publisher, scout, outreach, experiment, lifecycle and the copilot. It also does the fixing as well as the measuring: it writes the pages, publishes them, and asks other sites to name the business.
On error handling specifically, we are not claiming to be ahead of anyone. We have not audited how other tools count failures, so we will not rank the field on it. What we will say is that the rule is written here in public and you can hold us to it. Paid plans are not open yet. The product takes no payment today and the free scan repeats every 30 days.
What you can check yourself
This is the part a buyer cannot verify from outside a tool, so here is how to get at it. Ask any vendor, including us, these questions and expect a plain answer.
- For my last run, how many attempts were made, and how many returned an answer? Ask for both numbers per engine, not just a total.
- What was the denominator of the headline percentage? Divide the mention count by it yourself and see if you get the figure on the dashboard.
- What happens to a timed-out request? Retried, dropped, or scored as a non-mention?
- Is a refusal counted as a miss, and is the refusal rate shown anywhere?
- For Google AI Overviews, is "no overview shown" an error or a result?
- Can I see the raw stored response for one specific attempt, including a failed one?
Then run a spot check. Pick one question from your set. Ask it yourself in the same engine on the same day. You will not get a byte-identical answer, because these systems vary, but you will see whether an answer was possible at all. If the tool logged a failure at a moment when the engine was clearly answering for you, ask why.
The other vendors in this category publish their own documentation, and that is where to look for their rules rather than taking our word for it: Profound, Peec AI, Otterly.AI and Scrunch AI.
Who should not choose AI Knows Us
- If you need an enterprise contract, an invoice and a signed service level agreement today, this is not it. Paid plans are not open yet.
- If you want raw API logs streamed into your own warehouse for custom analysis, check that need against the tool before you start.
- If you want a single confident number with no range attached, you will find our reporting annoying. We print the range on purpose.
- If your buyers do not ask AI assistants before they decide, measuring this is not your best use of a month.
FAQs
Is a refusal the same as a failed request?
No. A refusal is an answer. The engine ran, produced text, and declined to recommend anyone. That belongs in the denominator as a non-mention and should also be reported on its own line. A failed request produced no answer at all, so there is nothing to score.
Why not just retry until it works?
You can retry, but not in the same second. Failures cluster, so an immediate retry often lands in the same outage and fails again. Retrying on a later run gives the provider time to recover and keeps the sample spread across the period rather than bunched into one unlucky minute.
Does excluding errors make the number look better than it is?
It can look higher, yes, which is why the attempt count and the answered count must be printed next to the rate. If a vendor shows you 26.3 percent without telling you it came from 61 out of 232, you cannot check it. Publishing both numbers is what stops the exclusion from being a trick.
What if an engine fails for a whole run?
Report it as unavailable for that period rather than publishing a rate built on a handful of answers. A very small sample produces a range so wide that it carries almost no information, and printing it invites someone to read a trend into it.
If no Google AI Overview appears, is that an error?
No. The page loaded and Google chose not to show an AI answer for that query. That is a result. Count it in the denominator as a non-mention and track the no-overview rate separately, because it tells you which of your questions Google thinks need an AI answer at all.
How much does miscounting errors actually shift a number?
In the worked example on this page it was 1.9 points at the top level and 2.5 points on the worst-hit engine. That sits inside a 95 percent range, so you cannot see it in one run. The problem is that it always pushes the same way, so it bends a trend line across several runs.
Frequently asked questions
Is a refusal the same as a failed request?
No. A refusal is an answer. The engine ran, produced text, and declined to recommend anyone. That belongs in the denominator as a non-mention and should also be reported on its own line. A failed request produced no answer at all, so there is nothing to score.
Why not just retry until it works?
You can retry, but not in the same second. Failures cluster, so an immediate retry often lands in the same outage and fails again. Retrying on a later run gives the provider time to recover and keeps the sample spread across the period rather than bunched into one unlucky minute.
Does excluding errors make the number look better than it is?
It can look higher, yes, which is why the attempt count and the answered count must be printed next to the rate. If a vendor shows you 26.3 percent without telling you it came from 61 out of 232, you cannot check it. Publishing both numbers is what stops the exclusion from being a trick.
What if an engine fails for a whole run?
Report it as unavailable for that period rather than publishing a rate built on a handful of answers. A very small sample produces a range so wide that it carries almost no information, and printing it invites someone to read a trend into it.
If no Google AI Overview appears, is that an error?
No. The page loaded and Google chose not to show an AI answer for that query. That is a result. Count it in the denominator as a non-mention and track the no-overview rate separately, because it tells you which of your questions Google thinks need an AI answer at all.
How much does miscounting errors actually shift a number?
In the worked example on this page it was 1.9 points at the top level and 2.5 points on the worst-hit engine. That sits inside a 95 percent range, so you cannot see it in one run. The problem is that it always pushes the same way, so it bends a trend line across several runs.