Research Note: A Matched Protocol for Robotaxi Safety Claims

Research Note: A Matched Protocol for Robotaxi Safety Claims

Question

The safety guide needs a method that can test crash, injury, intervention, disengagement, or mileage claims without converting unlike records into a ranking.

Federal data boundary

NHTSA's Standing General Order page publishes reported ADS and Level 2 ADAS incidents and a detailed limitations section. The third amended order became effective June 16, 2025. NHTSA says reporting requirements differ between ADS and Level 2 ADAS, reporting entities have different access to crash information, duplicate reports can occur, and the public data should not be assumed to represent all crashes statistically.

The third amended order defines the current reporting threshold and timing. A report is evidence that an incident met the reporting rule as understood by the reporting entity. It is not by itself a fault finding, a fleet exposure record, or a safety rate.

Raw report counts cannot compare operators unless the analyst also has compatible exposure, operating conditions, event definitions, and reporting completeness.

Comparison fields

Every claim should disclose the following fields.

Intervention and disengagement require an additional definition. A company can count human steering takeover, high-level route guidance, a safety-driver precaution, a system-requested handoff, or a simulation event differently. "Miles per intervention" has little meaning until the event definition and supported domain are visible.

A useful study and its limits

A 2025 Waymo-affiliated retrospective study compared 56.7 million rider-only miles with human benchmarks matched by vehicle type, road type, and location. It used NHTSA incident records, company mileage, crash-type groups, severity outcomes, and statistical intervals. The authors reported statistically lower rates for several outcomes.

The study is useful because it demonstrates better comparison design. It still studies one operator, a defined period, selected locations, specific outcomes, and company-supplied exposure. It does not prove every robotaxi, every Waymo software version, or every city is safer than every human driver.

Waymo's living Safety Impact hub provides downloadable methods and current company analysis. Its July 2026 time-and-location research summary argues that roadway, city, day, and time materially change the human baseline. Both remain operator-affiliated sources and should be labeled that way.

Decision rule

If the numerator, denominator, domain, baseline, reporting rule, severity, and uncertainty cannot be matched, the correct result is "not comparable from the disclosed evidence." That is not the same as saying the systems are equally safe. It is a statement about the evidence.

Editorial use

The public guide should show how to build a matched claim table and when to stop. It should not rank Tesla and Waymo from raw NHTSA counts or repeat the transcript's 10,000-versus-444-mile comparison.