How to Read Autonomous-Vehicle Safety Claims

How to Read Autonomous-Vehicle Safety Claims

An autonomous-vehicle safety comparison is credible only when it aligns the system, automation state, operational design domain, geography, time, exposure, outcome, reporting threshold, human benchmark, adjustments, and uncertainty. A crash count or impressive percentage without those fields is not enough.

The narrow conclusion should match the narrow evidence.

flowchart TD
    A["Write the exact safety claim"] --> B["Name system, version, and ODD"]
    B --> C["Align outcome and exposure"]
    C --> D["Inspect reporting and benchmark"]
    D --> E["Reproduce adjustments and uncertainty"]
    E --> F["State the bounded conclusion"]

Write the claim as a comparison

"Robotaxis are safer" leaves both sides undefined.

A usable claim names the system and software grouping, automation state, location, operating conditions, observation period, exposure, crash outcome, and comparison population.

For example, a study might compare police-reported injury crashes per million rider-only miles for one system in selected cities with an adjusted human-driver benchmark over a stated period.

That sentence is less exciting and much easier to audit.

Identify the exposure denominator

Raw incident counts cannot compare programs of different scale.

Common denominators include vehicle miles traveled, passenger trips, operating hours, intersections, or another measure suited to the outcome. Miles alone can still conceal differences in road type, speed, time, weather, traffic density, and vulnerable road-user exposure.

NHTSA's current Standing General Order data page explicitly warns that summary incident reports are not normalized by vehicles, miles, or operational design domain.

If a source presents NHTSA counts as a company ranking without adding aligned exposure, stop.

Align the outcome

"Crash" can mean property damage, airbag deployment, police report, injury, hospital transport, serious injury, fatality, contact, or an event meeting a company-specific threshold.

Determine whether the measure counts crash involvement regardless of fault, preventable events, insurance claims, police reports, or another outcome. Check whether severity is based on medical coding, police assessment, vehicle damage, or a proxy.

Do not compare an injury-only numerator with an all-crash denominator. Do not compare company-recorded contact events with a benchmark limited to police-reported crashes.

NHTSA's Crash Report Sampling System estimates a national picture from a probability sample of police-reported crashes. Its inclusion rules and sampling design matter when it is used to construct a human benchmark.

Match the operational domain

A system operating at low speeds on mapped urban streets faces a different exposure mix from national human driving across highways, rural roads, snow, night, and every region.

The benchmark should match geography, road type, speed, time, weather, and other domain fields as closely as the data permit.

Dynamic adjustment can improve alignment. It can also introduce modeling choices that need documentation and sensitivity testing.

Read [[What an Operational Design Domain Really Means]] before accepting a comparison that treats all miles as interchangeable.

Understand the reporting system

NHTSA requires identified entities to report certain crashes involving ADS and Level 2 ADAS. The criteria differ between those categories.

The agency warns that entities have different access to crash information. A fleet operator with continuous telemetry may learn about minor incidents that another manufacturer knows only through consumer reports. Initial filings can be incomplete, updated, or duplicated.

The 2025 amendment also changed reporting requirements. Data before and after that change should not be merged without accounting for the rule.

The NHTSA file is a regulator-collected reporting dataset. It is not a final cause determination for every event.

Inspect company-authored evidence

Waymo's Safety Impact hub provides miles, crash cases, outcomes, locations, benchmark material, methods, and downloadable data. That is far more useful than an unsupported marketing sentence.

It remains company-authored evidence. Check who wrote the analysis, which releases were grouped, how miles and outcomes were selected, how the human benchmark was adjusted, how uncertainty was calculated, whether results are peer reviewed, and whether the files reproduce the headline.

Waymo's March 2026 company update said its analysis covered more than 170 million fully autonomous miles and reported large reductions in serious or fatal injury crashes, airbag deployments, and injury crashes relative to its human benchmark.

The correct public phrasing is "Waymo reports that its analysis found..." followed by system, scope, period, method, and limitations. Independent review may strengthen or challenge that result.

Separate partial automation from ADS

Level 2 driver assistance requires continuous driver supervision. A driverless Level 4 passenger service assigns the driving task differently.

Mixing their incident data can produce a category error. NHTSA has reported that entities sometimes misclassified systems in earlier filings and continues to improve data quality.

NTSB's investigative outcomes also show why role matters. Its partial-automation findings emphasize human monitoring limits, while advanced-automation investigations focus on system limitations, testing risk management, safety drivers, and oversight.

Look for uncertainty and rare outcomes

Fatal and serious-injury crashes are rare relative to miles. That is good for society and difficult for statistics.

Small counts can create wide uncertainty. A percentage reduction may look precise while the confidence interval remains broad. Zero observed events does not prove zero risk.

Ask for event counts, exposure, intervals, model assumptions, sensitivity analyses, and the pre-specified outcome. Be wary of selecting only the metric with the strongest result.

Reproduce the narrow result

The analysis record should identify every data file, version, transformation, exclusion, match, adjustment, formula, and code artifact. Another qualified analyst should be able to reproduce the table.

Then test alternatives. Change geographic matching, reporting assumptions, outcome definitions, software groupings, and exposure windows. A conclusion that reverses under a reasonable assumption needs stronger qualification.

State what the evidence supports

A disciplined conclusion might say that a defined system had a lower rate of a defined reported outcome than an adjusted benchmark within the studied ODD and period, under the published method.

It should not become "autonomous cars are safer than humans everywhere."

For one-event research, use [[How to Build an Autonomous-Vehicle Incident Evidence Record]]. For the product roles behind the data, read [[Levels of Driving Automation Explained]].

This guide was developed with AI assistance from E025, current NHTSA, NTSB, CRSS, and Waymo sources, and the linked comparison framework. Dalton Anderson remains the author. It is not a safety conclusion, statistical review, legal advice, or insurance advice. Editorial, research, statistical, technical, safety, legal, source, accessibility, and founder review are required before publication. Publication is not authorized.