One-Shot Evidence Ladder Research Note

One-Shot Evidence Ladder Research Note

For E098, one shot means one builder instruction before judging the initial result. It does not mean one sentence, no prior work, no clarification, or no setup. The shared input contained about 122 pages of product thinking, schemas, integrations, and implementation direction. Dalton explicitly described that material as documentation built over time rather than a casual prompt.

The first output can provide meaningful evidence. It can show whether the builder recognizes entities, proposes a useful information architecture, creates navigable screens, follows a visual direction, and approximates a workflow. It can also expose failure modes quickly. A broken route, fictional metric, or inappropriate domain workflow is evidence too.

The evidence ladder has seven levels.

Evidence does not automatically flow upward. A beautiful render does not prove navigation. A working workflow does not prove that the same action is authorized correctly for another user. A stable deployment does not prove that customers want the product.

Google's Firebase Studio documentation makes the same general boundary explicit: generated output can appear plausible while being inaccurate, generated code should be tested before production, and user data should not be entered into the prototyping chat. Replit's Agent guidance also tells users to review and test, add context, plan, and use checkpoints.

E098 reached different levels by platform. Base44, Lovable, and Replit showed navigable or workflow-like behavior in the recordings. Emergent showed mostly render-level evidence. Firebase Studio generated several screens but showed setup and reliability friction. None of the preserved evidence reaches secure multi-user operation or validated user outcomes.