Not every document needs the same scrutiny. Most of what people receive at work is read once and never used again, and that is fine. The trouble starts with the few documents where a wrong figure would actually cost something: the ones about to go out with your name on them, the ones you are about to quote, and the ones you published a while ago and have not looked at since.
We spent a lot of time with those documents, and the same four situations keep appearing. What we have learned is that each one fails in its own way. A document you are about to publish usually includes figures with no sources cited for them. A document someone sent has sources that do not support what it claims. An AI-generated document may cite sources that do not exist. And an old document has figures that were true once and are not any more. So the question to ask is different for each.
We built EMBER (Evidence Mapping and Benchmarking of Existing Research) to run those checks for us. EMBER takes an existing document, tests every factual claim against evidence outside it, and returns the text with a trust marker on each claim and a note on what to fix. But knowing which situation you are in, and what to look for, is useful with or without the tool.
1. The document you are about to publish
A client report, a conference deck, and a whitepaper for the website. Once it is shared, it cannot be recalled, and every statistic in it speaks for the organisation. A figure that does not hold up invites a public correction, and once a reader has caught an error, they tend to stop trusting the figures around it, including the ones that were right.
The check here is about completeness. Every number, date and comparison needs a source you could produce if challenged, and every source needs to actually contain what the document says it does. The common failures are a figure carried over from an earlier draft whose source was dropped, and a statistic the writer remembered rather than looked up. Most fixes are small: a stronger source, a softened figure, a caveat where the evidence is thin. This is the situation where we lean on EMBER most, because it turns “check everything” into a short list of what still needs attention.
2. The document someone sent you
A third-party research piece, a vendor’s pitch deck, an ebook someone downloaded. Its numbers are about to be quoted in a strategy paper or repeated to a client, and once that happens, they are the organisation’s numbers, with our name on any mistake.
The check here is about the citations, because that is where third-party research most often fails. The footnote looks authoritative, but the page it points to either does not contain the figure attributed to it, contains a different one, or cites the author’s own earlier work as if it were independent. Before quoting a figure from a document you did not write, open the reference and read the sentence it is meant to support. EMBER does this for every citation and flags the mismatches separately from how well the claim is supported, because a claim can be true and still be badly cited.
3. The document an AI tool wrote
An AI tool writes just as confidently when it is wrong as when it is right, and a fluent sentence and a fabricated citation look identical on the page. Most people do not look twice: a KPMG and University of Melbourne global study found that 66 per cent of AI users rely on its output without evaluating accuracy, and more than half said it had already caused them to make a mistake at work.
The check here is about existence. Does the source exist at all? Does the figure appear anywhere outside the document? When an AI tool invents a reference, it tends to give a broken link that goes nowhere. Unlike a paywalled link, this suggests the source did not exist in the first place. If the link works, check further that its contents do indeed support the claim it acts as a citation to.
The habit is to treat every citation in an AI-written document as unproven until it has been verified. EMBER applies the same test to AI-written claims as to human-written ones. It looks only at whether the evidence supports the claim, so a sentence gets no credit for sounding sure of itself.
4. The document you published a while ago
Documents stay static after they get published, but the world keeps moving. Last year’s figures get superseded, cited pages move or vanish, and a claim that was solid at publication quietly stops being true. An out-of-date document does not look wrong, which is the problem, and an old figure cited as current is our mistake now, not the original source’s.
The check here is about time. Which figures were current at the time of writing and have since been updated? Which cited pages still resolve? Is each figure dated in the text, so a reader can tell it was true in 2024 rather than true now? The practical method is to recheck at an interval and compare against the last pass, so that what changed is visible. Each EMBER run is a dated verdict on the document as it stood that day, which makes the comparison straightforward.
The one thing the four have in common
What these four types of documents have in common is that they all look trustworthy. The outgoing deck is polished, the incoming report has footnotes, the AI draft reads fluently, and the old whitepaper was accurate when it was written. Nothing about them signals a problem, so nothing prompts anyone to check.
Since appearance is not a good guide for initiating a check, a good rule of thumb is to use the situation as a guide. Is this document about to go out, has it just come in, was it written by an AI, or has it been sitting unread for a year? The answer points to a different thing to look for, and in practice, the check is shorter than people expect: a handful of figures without sources, a few footnotes to open, a couple of links to test, and a date to add. That is a small amount of work set against what a wrong figure costs once it has gone out with your name on it.
Each of these four situations is one we have run EMBER on ourselves. How the check works, and what the report that comes back contains, is here.
