Decision Quality Is Not the Same as Outcome Quality

Decision Quality Is Not Outcome Quality

A good decision can fail. A bad decision can succeed. If that sounds like an excuse, it is because process language is often used after the fact to defend a preferred story.

The solution is not to judge only by results. It is to preserve the process before the result arrives and then review process and outcome as separate evidence.

flowchart TD
    A["Ex ante process"] --> B["Decision"]
    B --> C["Outcome"]
    D["Luck and hidden information"] --> C
    A --> E["Process review"]
    C --> F["Outcome review"]
    E --> G["Updated rule"]
    F --> G

Why outcomes distort judgment

Baron and Hershey's 1988 outcome-bias research asked participants to evaluate decisions under uncertainty. When the outcome was favorable, people rated the thinking and decision maker more positively than when the outcome was unfavorable, even though the decision-time information was held constant.

A later replication and extension found evidence consistent with the original effect. Knowing the outcome changes how people judge what came before.

That distortion can affect performance reviews, incident investigations, product bets, medical judgments, investments, and public criticism.

What decision quality includes

Decision quality begins with a correct problem statement. The team should identify the owner, deadline, affected people, governing authority, constraints, and real alternatives.

It should gather relevant evidence, distinguish facts from assumptions, seek base rates where useful, and record important unknowns. Probabilities should correspond to defined events and dates.

The process should examine foreseeable harms, conflicts, required controls, dissent, reversibility, recovery, and stop conditions. A decision with a polished spreadsheet but no authority or harm analysis is not automatically sound.

The standard rises with consequence. A reversible experiment and an irreversible safety decision should not require the same review.

What outcome quality includes

Outcome quality asks what happened. Did the predicted event occur? Which benefits and harms appeared? Was the result inside the expected range? Who was affected? Were recovery and stop rules effective?

Results are evidence. One result may be noisy, but repeated results can expose a bad model, missing variable, poor calibration, weak execution, or changed environment.

The error is using the outcome to rewrite what was known. A surprise should update the next decision, not create a false memory that everyone saw it coming.

Four combinations

A good process with a good outcome is easiest to accept, but the result may still contain luck.

A good process with a bad outcome may be an expected loss inside an uncertain distribution. Review whether the risk was understood and whether the consequence was acceptable.

A bad process with a good outcome is dangerous because success rewards the wrong behavior. If a team ignored a critical control and nothing happened, the absence of harm does not validate the choice.

A bad process with a bad outcome needs both accountability and learning. Calling it unlucky avoids the process defect.

Accountability remains

Separating process from outcome does not excuse negligence. A person is accountable for available evidence, professional duties, required controls, truthful disclosure, foreseeable harm, and decisions within their authority.

Process review can reveal that the decision was not reasonable even if the exact outcome was uncertain. It can also reveal that a reviewer is punishing someone for an unforeseeable result.

The point is fairer accountability, not weaker accountability.

Preserve the ex ante record

The strongest protection against hindsight is a decision journal written before commitment. Record the alternatives, evidence, uncertainty, probabilities, assumptions, dissent, harms, authority, chosen option, reasons, review date, and stop rule.

Do not overwrite it later. Append the outcome and review.

This is the practical extension of Annie Duke's book argument. The journal turns a memorable distinction into evidence.

Review in the right order

First, hide the outcome when practical. Ask reviewers to judge the process from the contemporaneous record.

Second, reveal the outcome and inspect what it adds. Separate luck, execution, missing evidence, and changed conditions.

Third, update a future rule. The review should change a threshold, evidence requirement, control, monitoring plan, owner, stop condition, or forecast.

Without the final update, a review becomes storytelling.

The durable rule

Do not ask only whether the decision worked. Ask whether the process deserved confidence at the time, whether the outcome revealed new evidence, and what should change next.

Decision quality and outcome quality are different. Learning requires both.

The separation also improves portfolio review. One decision may be dominated by chance, while a series can reveal whether estimates, controls, and execution repeatedly fail. Review individual fairness and aggregate performance together.

AI assisted with research organization, structure, drafting, and validation. Dalton Anderson remains the attributed author and final editorial authority. The transcript and linked public sources control factual claims. Publication remains unauthorized.