You ran the same prompt twice. Same wording, same model, same afternoon. The first answer was sharp. The second wandered off, invented a statistic, and formatted itself differently for no reason you can see. Nothing changed on your end, so it feels like the tool is being moody.
It is not moody. It is doing arithmetic, and once you understand what kind, the inconsistency stops being mysterious and starts being something you can design around.
The model is not retrieving an answer
It is worth saying plainly: a language model does not look anything up. It predicts the next fragment of text, over and over, based on everything before it. Each fragment shifts the ground for the one after it.
That means there is no stored answer being fetched more or less accurately on different days. There is a fresh chain of predictions every time. Small differences early in that chain compound into large differences by the end.
Why the same prompt gives different answers
- Sampling. At each step the model has a spread of plausible next fragments. Most interfaces deliberately pick with some randomness rather than always taking the single likeliest one, because always taking the likeliest produces flat, repetitive text.
- Compounding. One different word in the second sentence changes what is probable in the fifth. By paragraph three you are on a different path entirely.
- Ambiguity in your prompt. If your instruction leaves room for several readings, the model is choosing between them each run. You are seeing your own vagueness reflected back with variation.
- Hidden context. System instructions, prior turns and retrieved documents all sit in the same stream. Change any of them and you have changed the prompt, even if the part you typed is identical.
Why it invents things
A model has no internal signal that separates I know this from this sounds like the kind of thing that would be true. It optimises for plausible continuation, not for accuracy. A fabricated citation is generated by exactly the same process as a correct one.
This is why asking a model to be more careful helps far less than people expect. Care is not the missing ingredient. Constraint is.
What actually reduces variation
You cannot make a probabilistic system deterministic by asking nicely. You can narrow the space it is choosing from.
- Specify the output shape. A prompt that demands a named structure has far fewer valid paths than one that says write a summary. Structure is the cheapest reliability you can buy.
- Give the decision rule, not just the goal. Say what to do when information is missing, when two sources conflict, when the input is empty. Otherwise the model improvises, differently each time.
- Move stable content to the top. Rules, definitions and reference material belong early. Variable input belongs at the end.
- Lower the temperature when the task has one right answer. Classification and extraction do not benefit from creative sampling.
- Show one worked example. A single concrete example constrains format more effectively than three paragraphs describing the format.
Test with more than one run
The habit that catches most of this: never judge a prompt on a single output. Run it five times against the same input. If the five answers differ in ways that matter, the prompt is underspecified, and you now know exactly where.
Then run it against input designed to break it. Empty input. Nonsense. Input that contradicts itself. A prompt that only works on well-behaved examples is not finished, it is untested.
The shift that makes it click
Stop thinking of the prompt as a request and start thinking of it as a specification. A request hopes for a good outcome. A specification defines what an acceptable outcome looks like and leaves less room for anything else.
The AI Prompt Engineering book covers how these models read instructions and why they break, and the seven lesson mini-course walks through the same system one exercise at a time.