Fact-Checking AI Output

Confidence isn't the same as accuracy

An AI model states a wrong fact in the same calm, fluent tone it uses for a right one. There is no visual cue, no hesitation, no "I'm not sure" you can rely on. A made-up statistic, a quote that nobody said, or a book that does not exist can all look exactly as polished as the real thing. This is sometimes called hallucination, and the practical lesson is simple: how sure an answer sounds tells you nothing about whether it is true.

What to check, in order of priority

You do not need to verify every sentence. Spend your effort where a mistake would cost you something:

• Numbers: prices, fares, percentages, dates and deadlines. These go out of date fastest and are the easiest to get subtly wrong.
• Names and quotes: who said it, who wrote it, what a law or rule is actually called.
• Sources and links: if the answer cites a study, article or web page, open it and confirm it exists and says what is claimed.
• Anything recent: models learn up to a cutoff date, and anything close to or after it is where they are most likely to guess.
• Anything you will act on: money, health, legal and travel decisions deserve a second look even when the answer seems reasonable.

A check I ran on a real answer

In the prompt lesson I asked ChatGPT to plan an eight-day jungle safari trip from Pune for four people on a ₹40,000 budget. The plan looked complete, with dates, a day-by-day table and a budget that added up. To see how much of it I could trust, I asked Claude to check the key claims against the official Tadoba booking page.

Here is what came out of it:

• Confirmed: the official page says safaris can be booked up to 120 days in advance, as the plan stated.
• Found but not in the plan: the same page says the core zone is closed on Tuesdays and the buffer zone on Wednesdays. The plan never mentioned this. Its safari days happened to avoid those closures, but a different set of dates might not have.
• Could not confirm: the safari price. The plan quoted ₹6,000 for a weekday buffer-zone safari, but the fee details on the official page sit behind a tab that did not show up when it was read. A resort's website listed ₹5,000 for the same thing. Two sources, two numbers, and I could not settle it from the pages I could see.

The takeaway is that the plan was mostly sound and still had gaps. The price is exactly the kind of figure I would confirm on the official booking page myself before paying for anything.

A simple process you can follow

1. Ask for sources. "Where did you get that?" If the model cannot point to something real and checkable, treat the claim as unverified.
2. Go to the primary source. The official site, the original document, the railway or government page. A second chatbot is not a check, because different models can share the same blind spots.
3. Compare, don't just confirm. If two sources disagree, as they did on the safari price, that is useful information. Find out why before choosing one.
4. Note the date. Fares, fees and rules change. A correct answer from last year may be wrong today.

Does web search fix it?

It helps a lot for current information, because the model can read a real page instead of relying on memory. It does not remove the need to check. The page it read may be out of date, a third-party summary, or partly hidden behind a tab or login, as in the example above. Open the source it cites and make sure it really says what the answer claims.

A useful habit: ask it to flag its own doubts

Adding "flag anything here you are not fully confident about" to a prompt will not make the answer accurate, but it often brings out caveats the model would otherwise state as plain fact. It costs you one sentence, and it is worth doing for anything moderately important.

Frequently asked

← Back   Next: AI for Writing →