Why "probably right" isn't good enough

Ask a good AI assistant a question and you get a fluent, confident answer. Ask it again in a fresh chat and you may get a different one: different wording, sometimes different facts. Most of the time nobody notices, because most of the time nobody asks twice.

This lesson is about the moment that stops being harmless.

What run-to-run variance is

A language model writes its answer one word at a time, choosing each word from a list of likely candidates. Some randomness in that choice is deliberate: it is what makes the output read naturally instead of robotically. The side effect is that the same input can produce different outputs on different runs. That is run-to-run variance.

Even when the randomness is turned down to its minimum (a setting called temperature 0), production systems can still drift. Requests are batched together on shared hardware, providers update their models, and the context sent with each question shifts slightly. The deterministic AI explainer covers the mechanics. For this lesson, the point is simple: a language model is not guaranteed to give the same answer twice.

When variance is harmless, and when it is risk

Variance is not a defect in itself. What matters is what happens to the answer next.

How the answer is used	Does variance matter?
Drafting, brainstorming, summarising for yourself	Rarely: variety can even help
Internal work a colleague reviews before it goes anywhere	A little: the reviewer catches it
An answer a customer reads and relies on	Yes: two customers can be told two different things
A price, a refund, an eligibility or credit decision	Critically: money and fairness depend on consistency

The line is crossed when someone acts on the answer without a person checking it first. Below that line, a varying answer is a style issue. Above it, it is a business decision being made differently each time.

You own what your AI says

In February 2024, a Canadian tribunal decided Moffatt v. Air Canada. The airline’s website chatbot had told a customer he could apply for a bereavement fare discount after travelling. That wasn’t the airline’s actual policy. Air Canada argued, in effect, that the chatbot was a separate legal entity responsible for its own actions.

The tribunal rejected that. It held that the chatbot was part of Air Canada’s website and that “it makes no difference whether the information comes from a static page or a chatbot”. The airline was ordered to pay the customer CA$812.02 in total.

The amount was small. The principle wasn’t: a business is responsible for what its AI tells customers. If two customers can get two different answers, the business owns both.

The audit problem

There is a second, quieter risk. When a customer, a regulator or your own leadership asks “why did the system decide this?”, the honest answer has to come from running the same case again and seeing the same result.

If the decision came from a fresh model generation, running it again may produce a different outcome. Then nobody can show how the original decision was made. For anything regulated (credit, insurance, employment, eligibility), that is a serious gap.

This is part of why the market is cooling on unconstrained AI. Gartner predicts that over 40% of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value or inadequate risk controls.

Try it yourself: a ten-minute consistency test

You don’t need engineers for this. Pick one question your team really asks an AI tool, ideally one where the answer matters: a policy question, a pricing rule, a summary of a contract clause.

1. Open a fresh chat and ask the question exactly as written. Copy the answer into a spreadsheet.
2. Repeat 10 to 20 times, each time in a new chat, with the identical wording.
3. For each answer, mark the parts a customer or decision would rely on: numbers, yes/no conclusions, deadlines, eligibility.
4. Count how many distinct versions of those important parts you got.

If the important parts are identical every time, that workflow is consistent today (re-test after any tool or model change). If they differ even once, you’ve found a place where “probably right” is being treated as right.

What comes next

The fix isn’t to stop using AI. It is to move the decision itself out of the model’s hands while keeping the model where it helps. The next lesson defines deterministic AI, the term for doing exactly that.