How to verify an AI answer and its sources

An AI answer with links can still be wrong. A citation proves only that a link was produced. It does not prove that the page exists, says what the answer claims, is current, or is appropriate evidence.

Verification begins by treating the answer as a list of claims, not as one polished block of text.

Prioritise claims by consequence

Do not spend equal time checking every sentence. Prioritise claims involving:

• law, visas, tax or contracts;
• health and safety;
• prices, schedules and deadlines;
• product compatibility;
• commands that delete or overwrite data;
• security and privacy;
• named studies, statistics or quotations;
• decisions that are expensive or difficult to reverse.

The checking effort should match the consequence of an error. Health, legal, financial and safety decisions need qualified, current sources and may require professional advice rather than an AI summary.

Split compound claims

AI often compresses several statements into one confident sentence. Break it apart.

“The device supports the standard, works with every router and receives updates until 2030.”

That contains at least three checks:

1. Which version of the standard?
2. What compatibility limitations exist?
3. Where is the support end date stated?

One source may support only the first part.

Open the citation

For every important cited claim:

• confirm the page loads;
• check the publisher and author;
• search the page for the relevant wording;
• read enough surrounding context to understand conditions;
• check publication and update dates;
• distinguish a final rule from a proposal;
• confirm the page refers to the same product, country or population.

If the source cannot be opened, do not quietly treat the citation as evidence.

Prefer the source closest to the fact

Use:

• legislation or government guidance for legal rules;
• the manufacturer manual for product specifications;
• the transport operator for current schedules;
• the original research paper for a study result;
• a recognised public-health body for health guidance;
• the project’s own documentation for software behaviour.

A secondary summary adds interpretation and may omit limitations. Use the original study when it is available and relevant, then use good secondary reporting for context where helpful.

Primary does not mean infallible. A manufacturer is authoritative about connector specifications and conflicted about whether its product is “best.” Match the source to the claim.

Check the date twice

First check when the page was published or updated. Then check whether the underlying fact has a separate effective date.

A 2026 article can describe a rule that starts in 2027. An old manual can remain correct for an old product. A search result snippet may show the crawl date rather than the publication date.

Write the verification date beside unstable facts while researching. This makes later updates possible.

Search against the answer

Do not only search the phrasing the AI supplied. Look for failure conditions:

• [claim] exception
• [product] compatibility limitations
• [rule] effective date
• [study] criticism
• [software feature] removed
• [country] official guidance

This is not cynical searching. It is how conditions hidden by a smooth summary become visible.

Recalculate numbers

Check units, denominators and time periods. Common errors include:

• confusing percentage points with percent change;
• mixing monthly and annual costs;
• converting miles and kilometres twice;
• quoting relative risk without absolute risk;
• applying a per-device figure to a whole household;
• using a national average as an individual prediction.

Recreate simple calculations in a spreadsheet or calculator. For complex analysis, find the original method and assumptions.

Ask AI to help, but not to certify itself

AI can help extract claims, propose counter-questions or format a verification table. It should not be the only judge of whether its own answer is true.

A useful table is:

Mark “partial” honestly. A source supporting a general trend does not support an exact number.

A worked claim-splitting example

Suppose an answer says: “This router supports Wi-Fi 6E, works with every older device and will receive security updates until 2030.” Do not search that whole sentence as one claim.

The corrected answer may be narrower than the original. That is the point: verification changes unsupported wording instead of merely adding links to it.

Why confident errors happen

NIST calls false or confidently presented erroneous generative-AI output “confabulation.” These systems generate likely output from patterns; factual accuracy is not guaranteed by fluent wording.

Tone is not a reliability signal. A hesitant correct answer and a confident false answer can come from the same system.

The short verification pass

When time is limited:

1. isolate the decision-changing claims;
2. open every important citation;
3. replace secondary sources with primary ones;
4. check date, jurisdiction and product version;
5. search for exceptions;
6. recalculate numbers;
7. remove any claim that cannot be supported.

The last step matters. If a decision-changing claim cannot be supported, remove it, narrow it or state the uncertainty explicitly.

Sources consulted

• NIST Generative AI Profile
• NIST AI Risk Management Framework
• NIST AI Resource Center