Research Note
Real-Condition Product Test Evidence Model
A demonstration supports only the claim that the observed behavior occurred under the shown conditions. A product-readiness claim requires testing that resembles the inte
Real-Condition Product Test Evidence Model
Decision
A demonstration supports only the claim that the observed behavior occurred under the shown conditions. A product-readiness claim requires testing that resembles the intended operating environment, duration, intensity, variation, support load, and consequence of failure.
Test dimensions
| Dimension | Evidence question |
|---|---|
| Duration | Does performance change after minutes, hours, repeated cycles, fatigue, heat, drift, or wear? |
| Intensity | What happens under the actual load, pace, concurrency, force, traffic, or cognitive demand? |
| Variation | Which people, objects, starting states, environments, and edge conditions were included? |
| Intervention | How often did a person reset, guide, repair, approve, or rescue the system? |
| Failure | What unsafe state, degraded behavior, or incomplete outcome occurred? |
| Recovery | Can the system detect, contain, communicate, and recover from failure? |
| Support | What setup, maintenance, training, monitoring, and escalation are required? |
| Change | Which software, model, hardware, data, or environment changes force reevaluation? |
NASA's systems engineering handbook distinguishes verification from validation and asks whether a system works in its intended operational environment. NIST's AI Resource Center provides testing, evaluation, verification, and validation resources tied to AI risk management. OSHA notes that many robot accidents occur during non-routine work such as programming, maintenance, testing, setup, or adjustment.
These sources do not create one universal test duration. They establish why the intended use and operating conditions must shape the evidence.
Cross-domain boundary
Software, wearables, medical devices, industrial robots, consumer products, and research prototypes require different methods. A general framework can identify missing dimensions, but it cannot replace a regulated protocol, safety standard, accessibility review, security assessment, clinical study, or domain specialist.
Publication use
The public essay should help a reader turn a claim into a testable operating record. It should not imply that longer testing alone proves safety or quality. Duration and intensity matter alongside task definition, measurement quality, affected users, controls, support, and accountability.
Sources
Follow the evidence.
- NVIDIA Rubin announcementnvidianews.nvidia.com
- SIMA 2 technical reportstorage.googleapis.com
- NIOSH Center for Occupational Robotics Researchcdc.gov
- OSHA robotics overviewosha.gov
- NIST AI Resource Centerairc.nist.gov
- Google DeepMind SIMA 2 announcementdeepmind.google
- NVIDIA Blackwell Ultra announcementnvidianews.nvidia.com
- BMW Figure 02 trialpress.bmwgroup.com
- Spotify episode recordpodcasters.spotify.com
- Google DeepMind SIMA announcementdeepmind.google
- Figure news indexfigure.ai
- Figure 03 introductionfigure.ai
- NASA Systems Engineering Handbooknasa.gov
- Figure Helix 02figure.ai
- NVIDIA Blackwell launchinvestor.nvidia.com
- BMW Figure 03 projectpress.bmwgroup.com
- SIMA technical reportstorage.googleapis.com
- Figure company pagefigure.ai