Build the run so it can be restarted, and a server error costs you one reading instead of the dataset

Decide before the first call that the runner writes one row per reading the moment it arrives, and that a restart skips every row already on record. A timeout, a rate limit or a server error then costs one reading rather than a day of collection, and the run can be driven to completion in passes instead of in one unbroken session. Our own two citation runs sit either side of that decision: on 15 October 2026 we attempted 612 readings and used 597, and on 16 October 2026 we attempted 1,260 and used all 1,260, with the larger run finishing clean because it could be restarted into its own gaps.

This is an engineering choice rather than a methodological one, and it is the choice that decides whether the methodology is reportable at all. We sell this measurement taken on a schedule, which means the collector is ours to get right before any figure it produces is worth quoting.

The two counts only exist if the rows do

A study that reports both what it attempted and what it used is telling a reader something a single total cannot. 597 of 612 is a different claim from a run whose output file holds 597 rows and no memory of the rest. The pair is not a reporting habit bolted on afterwards: it is a by product of having stored every attempt, including the ones that failed, as its own row with its own status.

Run	Attempted	Usable	What the gap was
15 October 2026	612	597	15 readings came back without the source list the run existed to record
16 October 2026	1,260	1,260	No gap

The losses are also locatable, which is only true because each row carried its engine. ChatGPT finished 306 readings of its 306. Gemini finished 291. Every one of the 15 fell in one column, which is a different and more useful statement than a run wide loss rate, and it tells the next run where to add retries.

What one row has to carry

The row is the unit of the whole design. Five fields make it restartable and the first one does most of the work.

• A key that identifies the reading, not the attempt. Question identifier, surface, market. A restart looks for that key and moves on if it finds it, which is what makes re running the whole command safe.
• The full answer text, saved verbatim. Not a score and not a judgement. A stored answer can be recounted under a new rule next month; a stored verdict cannot.
• Every cited address exactly as it came back. Hostnames get resolved and cleaned later, before anything is divided, and that is only possible if the raw address is still there.
• A status, with the reason. Usable, empty, truncated, no source list, refused. A row marked with its reason is a row you can re attempt selectively.
• A timestamp per row. Not one date for the run. Two readings taken four hours apart in the same file become arguable later if nothing records which was which.

Batch the run so a failure has a boundary

The second run asked 210 questions per market, 630 in total, and it was pushed through in 21 batches. A batch is the unit you re attempt and the unit you reason about when something goes wrong halfway. It also gives a failure a name: batch eleven is a thing somebody can look at, while a run that stopped somewhere is not.

Keep the batch small enough that losing one is cheap and large enough that the overhead of starting one is not the dominant cost. The useful test is whether you would be willing to re run a whole batch rather than argue about which rows inside it are suspect, because that is the decision you will actually be making at the time.

Name the rows you could not use, then re attempt only those

A row that failed is not a finding, and treating it as one is a specific and expensive mistake. A policy file reading of ours taken on 20 October 2026 covered 26 hosts and parsed 24 of them. The other two answered with an HTTP 403, and the published result calls them not usable instead of calling them refusals, because one of those two hosts has already been shown in this programme to serve that status when it is busy rather than when it objects.

Two named rows out of 26 is a cheap second attempt on another day. The same two rows folded silently into a blocked column would have been a published claim about somebody's policy, built on a response that was about traffic. Keeping a status and a reason on every row is what holds those two outcomes apart.

The arithmetic to do before the first call

The size of a run is questions multiplied by surfaces multiplied by repeats, and it grows faster than people expect. Our second run was 630 questions on two surfaces, asked once each, which is 1,260 readings. Ask each question three times instead of once and the same bank needs three passes of that size, with no additional coverage and considerably more confidence in each rate.

That multiplication is also how metered tooling bills you, which is worth working out before a plan is chosen rather than after the first invoice, and the same arithmetic in its rank tracking form is set out in why a plan limit is keywords multiplied by how often you check them. Decide the multiplier, then build the runner, then start.

What a resumable run does not fix

It does not make two readings comparable. Rows collected across two days are rows from two days, and the timestamp on each one is what lets somebody say so rather than assume otherwise.

It does not improve a bank that leans one way. A set of questions that only asks one kind of thing produces the same slanted result whether it was collected in one pass or in six.

And both runs above collected from two products. Three of the five surfaces we read for a customer contributed nothing to either figure, and neither bank holds a rate by trade, because no question in either one named a trade.

Questions about restarting a long measurement run

How does the runner know a reading is already done?

By the key, written before anything else. Question identifier plus surface plus market, looked up in the rows already stored. If it is there, the row is skipped, so running the same command twice does nothing the second time, which is the property you want at three in the morning.

Should a failed row be deleted on a retry?

No. Keep it with its status and write the successful attempt as its own row. The failure is part of the record that lets you report the attempted count honestly, and it is also the only evidence you will have about how often a surface times out.

When is a reading unusable, and when is it simply a zero?

An empty answer, a truncated one, or one with no source list where the whole point of the reading was the source list. A reading that answered fully and named nobody is a result and belongs in the rate. The 15 readings set aside in our first run are the first kind.

Is it worth storing the whole answer rather than the parts I need?

Yes, and the reason is a change of rule later. The same stored rows from the 16 October run produce 20,143 citations counted as one domain per reading and 36,576 counted as raw addresses. Only one of those was the published measure, and both were available because the text was kept rather than scored on arrival.

How large can a run be before this matters?

It matters at the size where losing the run would make you reschedule the day. For a run of a few dozen readings a single pass is fine. At 1,260 readings across 21 batches it is the difference between a dataset and an afternoon spent working out how far it got.

The collector decides what the study is allowed to say

Every honest sentence in a published reading traces back to something the runner wrote down at the time: the attempted count, the loss by engine, the raw addresses that let a proxy be resolved before a share was calculated, the timestamp that stops two days being read as one. A collector built to emit one summary file can only ever support the summary. Somebody else having built and maintained that collector is part of what a customer is buying from us, which is a duller sales argument than most in this field and a more honest one. Ours produced the search figures at aiknowsus.com/proof, every one of them computed from the file rather than entered by hand.