How to Verify AI Answers and Citations

How to Verify an AI Answer Against Its Citations

Verify a cited AI answer claim by claim. Open the exact source, read the cited passage in context, decide whether it supports the wording, look for missing or contrary evidence, check the date and authority, then rewrite or reject the claim.

A citation makes inspection possible. Its presence does not make the sentence true.

flowchart LR
    A["AI answer"] --> B["Split into claims"]
    B --> C["Open exact citation"]
    C --> D["Check passage and context"]
    D --> E["Search for conflicts and omissions"]
    E --> F["Check date and authority"]
    F --> G["Support, qualify, rewrite, or reject"]

Start with claims, not paragraphs

One sentence can contain several claims.

"The product is private, accurate, and approved for schools" contains at least three. Each claim may need a different source and reviewer. A vendor privacy page cannot establish accuracy. A citation to an education page cannot establish that a particular district approved the product.

Copy the answer into a claim-audit record. Give each testable statement its own row.

Do not begin by asking whether the whole answer "looks right." That invites the polished language to set the confidence level.

Confirm the source identity

Open the artifact itself.

Record title, author, publisher, edition, version, effective date, and access date. Confirm that the citation points to the intended document rather than a summary, old copy, similarly named page, or extracted fragment.

When a notebook contains multiple editions, the answer may cite a real passage from the wrong period.

For a web source, determine whether the page is current, archived, or silently updated. For an uploaded file, compare its hash or version with the source of record when that matters.

Check the passage

Find the cited words and compare them with the answer.

The passage may directly support the claim. It may support only part of it. It may contain the same topic without entailing the conclusion.

Google's current NotebookLM chat guide says citations can reveal quoted material and navigate to its source location. That feature shortens the inspection path, but the reviewer still decides what the passage means.

If the cited text is not available, do not assign confidence from the citation marker alone.

Read enough context to find the boundary

Read the surrounding paragraph, section heading, definitions, footnotes, table notes, exceptions, and limitations.

A source may say a control is available only to a paid plan, a feature works only in one language, a rule applies after a date, or a study used a narrow population. Removing the condition can turn a true passage into a false general claim.

Check whether the notebook import preserved the relevant material. Google's current source-import record notes that some comments, footnotes, embedded material, and nested pages may not enter the notebook representation.

When the context is missing, return to the original artifact.

Separate source fact from model inference

An answer often combines direct facts with synthesis.

Mark which words are stated by the source and which connect or interpret them. Then ask whether the inference follows.

Suppose one source says public sharing is available and another says viewers can ask questions. It does not follow that every underlying source remains hidden. The inference needs its own evidence.

The model may create a useful explanation. It should not borrow the citation's authority for an unstated conclusion.

Search for evidence the answer omitted

The cited passage can be accurate while the answer remains incomplete.

Search the approved source set for exceptions, newer guidance, contrary evidence, alternative interpretations, and failure cases. Ask which source would be most likely to change the conclusion.

A one-sided pack can produce perfectly traceable one-sided answers.

Use [[How to Build a High-Quality Source Pack for an AI Tutor]] to record conflicts and known gaps before treating the answer as representative.

Check currency

Match the source date to the decision date.

Product features, prices, limits, terms, regulations, policies, and support states can change. An old source may be correct history and wrong current guidance.

Preserve both statements when the change matters. "Google introduced Audio Overviews as experimental in September 2024" is a historical claim supported by the launch record. Current behavior belongs to the current Audio Overview help page.

Do not replace history with the current page or present an old launch page as the current product.

Match authority to the question

Ask whether the source has the authority to answer this kind of claim.

A vendor can state its intended feature and terms. A school decides its policy. A researcher reports a study. A regulator or statute carries a different form of authority. A user can report an experience.

No single source automatically owns every layer.

For a high-impact decision, identify the relevant professional, institution, official record, and review process. A claim audit supports that process. It does not replace it.

Use a support classification

Classify the result before rewriting it.

This is more useful than a single confidence score because it tells the editor what to do next.

Work a low-risk example

Consider the claim: "A NotebookLM chat-only public link prevents viewers from reaching the notebook's sources."

The current public-notebook documentation warns that the chat view hides material for focus but may not prevent access to underlying sources and artifacts.

The original claim is contradicted by the current first-party record.

A safer revision is: "A chat-focused view changes the default interface, but the notebook owner should assume viewers may still reach underlying material and should share only approved sources."

That revision keeps the evidence, limitation, and action together.

Escalate consequential questions

Do not use this method to certify medical, legal, financial, employment, educational, safety, security, or other consequential advice.

The NIST Generative AI Profile provides a broader risk framework for confabulation, privacy, information integrity, human oversight, and governance. A claim audit can supply evidence inside that process.

For a related episode about claims that failed under reproduction, see [[How to Reproduce a Language Model Benchmark]].

This guide was developed with AI assistance from the immutable E035 transcript, current Google product records, NIST's Generative AI Profile, and the linked citation-audit framework. Dalton Anderson remains the author. Research, education, legal, product, current-source, and founder review are mandatory before publication. Publication is not authorized.