Read the evidence before counting AI failures

An audit needs the finding behind a number. The AI Reliability Incident Register records bias, deception and harmful conduct, with evidence status kept separate from category. Its 90-record release demonstrates why a single “AI failures” total is an incomplete description.

What does the evidence mix show?

This analysis freezes Register v1.12, released October 16, 2026. Every count comes from its versioned JSON; each ID is counted once. A research study is one record, regardless of its model or trial count.

43 of 90 records are studies: 47.8% of this release. Describing the full collection as 90 proven real-world harms would erase that distinction, along with the 14 allegation records and nine reports. A finding is an institutional confirmation within the scope of a record; it does not necessarily mean a court found liability.

Why should a reader filter before making a claim?

The evidence mix differs across categories. 14 of 17 deception records are studies (82.4%). In harmful conduct, 11 of 39 records are allegations (28.2%). These are shares of the selected records in each category, not estimates of how frequently either failure occurs.

The difference has a practical consequence: a team choosing a deception example is often looking at a controlled experiment, while a harmful-conduct example may require careful treatment of a contested complaint. Read the setup, observed result, response and unresolved consequence before translating either into a control requirement.

A zero means no included record occupies that cell. The absence of a deception allegation in this snapshot cannot establish the absence of such allegations elsewhere. Nor can these counts rank vendors or compare the severity of categories.

What do the evidence labels establish?

Our methodology uses four labels. Study identifies research with a stated setup. Allegation preserves the contested status of a lawsuit or complaint. Report attributes an account without upgrading it to an institutional finding. Finding identifies what a court, regulator, auditor or organization confirmed. Consult each record’s source and response to learn what was actually established.

These distinctions also matter across databases. The OECD abstract separates actual harms from potential dangers. The AI Incident Database editor’s guide distinguishes incidents, issues and reports, and excludes certain theoretical studies from incidents. Our Register explicitly retains labelled studies. A shared word such as “incident” does not make collections interchangeable.

How should the records inform a control test?

Select a record whose failure mechanism matches the workflow, then define an observable test result. Our proposed review sequence is: identify the source claim and its uncertainty, name the control boundary, introduce a controlled failure, and retain both the expected outcome and actual evidence.

Reproducing an output can help investigate a failure. It does not establish that an output is fair, an action authorized or an allegation true. For a discrimination question, evaluate appropriate group outcomes and proxies; for an action question, inspect permissions and destination state. The Deterministic AI Checklist keeps these evidence requirements separate, and its filled review example shows how to leave untested areas visible.

How can you reproduce this table?

Download the Python script and frozen JSON. With Python 3.9 or later, run:

python3 reproduce.py ai-reliability-incidents-v1.12.json register-analysis

The script verifies the input hash, checks unique IDs and counts all category/status combinations, including zero cells. It writes a JSON summary, CSV table and credited chart. It makes no network requests. The CSV uses 90 as the denominator of every percentage. The category-specific percentages above instead use the stated row totals, 17 and 39; all article percentages round to one decimal.

The snapshot SHA-256 is 1cd10bf6caf2e253bdaccd41818dfdec2f8ab27b15cb76a7aac3a88ea31d8d61. Later additions do not change this pinned analysis. This is a curated collection with uneven coverage, not a representative sample, time trend or measure of model accuracy. No controls were tested by counting the records.

How to cite Deterministic AI. Read the evidence before counting AI failures. October 16, 2026. Analysis of Register v1.12. Link this article and the frozen release; include your access date. Original analysis, chart and script: CC BY 4.0; credit the publication and identify changes.

For invented facts and other false assertions, use the companion Tracker composition analysis.