Deterministic vs probabilistic AI: specify what must repeat

Write the acceptance condition before choosing a label. Is the requirement identical text, the same calculated amount, the same approval decision or the same confirmed action? Each needs its own comparison.

What do deterministic and probabilistic mean here?

This reference uses deterministic for repeatable results within a declared execution boundary and fixed relevant conditions. Probabilistic describes representing possible outcomes with probabilities or selecting outcomes using those probabilities. The terms can describe different aspects of one system.

Thinking Machines Lab separates sampling randomness from the numerical behavior of inference. Its explanation shows why a model’s use of probabilities and a serving system’s repeatability need separate examination. Read the technical account.

Illustrative example: suppose a fixed computation always returns probabilities of 0.8 for category A and 0.2 for category B. That vector represents uncertainty and is repeatable. Always selecting the larger value returns A; sampling from the vector permits either category. This is an invented mathematical example, not a measured model result or a confidence threshold recommendation.

Which layer should you compare?

The following is our engineering review framework. It builds on the inference distinctions above and LangChain’s distinction between predetermined workflows and dynamically directed agents. LangChain documentation.

Predetermined control flow does not by itself establish repeated model output. LangChain’s workflow examples include successive LLM calls; our recommendation is to test those calls separately from the graph’s permitted paths. Workflows and prompt chaining.

Why is temperature zero insufficient evidence?

Thinking Machines Lab explains that removing sampling randomness still leaves numerical execution to consider, including changes associated with batching. Its analysis distinguishes repeatability for a fixed batch from results that remain unchanged across batch sizes. This is a documented mechanism, not proof that every serving system behaves identically. Thinking Machines Lab.

PyTorch states that reproducibility is not guaranteed across releases, commits or platforms, and may differ between CPU and GPU even with identical seeds. It also warns that deterministic operations are often slower. Measure the relevant performance tradeoff on the intended workload instead of assuming a universal cost. PyTorch reproducibility guidance.

The LLM determinism guide covers the inference boundary in more detail.

How can probabilistic interpretation feed deterministic execution?

Proposed invoice design, not a tested implementation: use a model to propose fields from an invoice, validate those fields against the source and approved business records, then execute a versioned rule on the accepted values.

For an invented routing rule, amounts above 1,000 units require review and amounts at or below 1,000 take the standard route. The rule should act on a checked amount and currency. If a fresh extraction proposes a different amount, the execution layer has received a different input; identical routing is no longer the same-input test.

Make the handoff explicit:

1. Retain the candidate fields and supporting record.
2. Establish which values the workflow accepts and why.
3. Apply the approved rule version to those values.
4. Pause unresolved cases for authorized human review.
5. Retain the decision and confirmation of any downstream action.

This design places a testable contract around the accepted inputs. Whether the whole workflow satisfies a broader determinism claim still requires evidence about extraction, validation, external state and exception handling. The companion RAG versus deterministic execution guide explains the evidence side of that handoff.

What tests should settle the comparison?

Use three separate tests in the review. These are our recommended acceptance checks, not a certification standard.

• Repeatability: hold the declared inputs, state and versions fixed; run fresh executions and compare the specified result. Report saved-result reuse separately.
• Correctness: compare decisions against approved reference cases, including rule boundaries and missing inputs. A repeatable application of the wrong rule remains wrong.
• Auditability: ask a reviewer to reconstruct one decision from its source records, accepted inputs, rule version, exception resolution and action outcome.

Then change one declared condition deliberately—such as a policy version—and test the expected new behavior. Keep this change test separate from fixed-condition repetition.

Start with the deterministic AI definition, run the testing protocol, and collect the records in the Deterministic AI Checklist. A useful comparison ends with a result someone can inspect and a clear account of what remains untested.