The verification debt: checking what a model tells you about a company

Here is a useful way to think about an unchecked statement from an AI tool: it is borrowed. The sentence arrives with the confidence of a primary source and the provenance of a rumour, and until you have traced it, the difference between what you know and what you merely hold is exactly the distance to the original document. The desk calls that gap verification debt. Like any debt, it compounds quietly and comes due at the worst moment.

Rule one: the primary document decides

For companies with public listings, primary material is refreshingly easy to reach: annual and quarterly filings, earnings-call transcripts, investor-relations pages, regulatory announcements, the prospectus itself. Search snippets, aggregator profiles, news summaries and model answers are all secondary; they can be a day old, a year old, or a confident blend of both. The habit that pays: when a claim matters, name the primary document it should live in before you ask anyone — human or model — to confirm it.

The desk's checking pass

When a model hands the desk a claim about a company, the claim goes through four questions before it is allowed inside a note:

• Where would this live? Identify the primary source the claim must trace to — a filing, a transcript, a regulatory page. If no primary home exists, the claim is treated as unfiled, full stop.
• Does the period match? A model blends training data from multiple years with perfect fluency. Check the reporting period, the fiscal-year convention and the date on the claim before comparing any figure.
• Do the units survive? Currency, thousands-versus-millions, per-share versus total: paraphrase kills units quietly, and a percentage of the wrong base is still wrong.
• Does the wording survive? Open the primary page and compare the exact words. “Discussed the possibility of” and “announced” are different sentences with different consequences.

Any claim that fails one of the four is either struck or explicitly re-labeled as unverified in the note. The label travels with the sentence, so a future reader re-checks instead of re-trusting.

Red flags worth more than a suspicious mind

Some patterns make the debt visible. Confident tone with no source named. A number that is conveniently round or suspiciously precise. A claim about a company’s plans that reads like a claim about its results. Two sources that turn out to be the same story retold twice, dressed as independent confirmation — this last one is common around fast-moving news, where secondary coverage clones itself faster than primaries can be published.

Queuing the debt sensibly

Verification is not free, so spend it where it counts. A detail inside an example does not deserve the same scrutiny as a fact you are about to rely on. The desk’s rule of thumb: the more a claim changes an action, the closer to primary it must be traced before the action — and if there is no time to trace it, the honest status of the claim is “believed, not checked”, in writing, until it is.

That single sentence, kept in your notes, is what separates using an AI tool from owing it.