Back to the episode map

Article

How to Review a Decision Without Outcome Bias

Reconstruct what was known before the outcome, review the process and result separately, identify learning, and record what should change next time.

Aug 4, 20264 min readBy Dalton Anderson

How to Review a Decision Without Letting the Outcome Rewrite It

Reconstruct the decision using what was known at the time. Review the process before discussing the result. Then bring the outcome back as new evidence and decide what should change.

This method helps a team learn without pretending outcomes do not matter or allowing hindsight to rewrite the original choice.

1. Confirm the purpose of the conversation

Ask whether the person or team wants support, a quick debrief, formal analysis, or a decision review.

This matters in personal conversations and at work. Someone who is still absorbing an event may not be ready for a hard critique. A regulated or high-impact event may require a formal incident, legal, compliance, safety, or human-resources process instead of an informal review.

Do not use a casual decision review to bypass the required path.

2. Reconstruct the decision record

Before discussing the outcome, write down the decision that was made, the objective, the decision owner, the date, the options considered, and the constraints.

Capture the information available at the time, including its source and freshness. Record the important assumptions, uncertainties, estimates, dissent, dependencies, and conditions that would have triggered a different choice.

If no contemporaneous record exists, label the reconstruction. Memory after an outcome is not the same as a record created before it.

3. Hide the outcome when possible

Have reviewers examine the reconstructed decision before they learn what happened. This is sometimes called an outcome-blind review.

Ask which option they would have selected and why. Record the reasoning before revealing the result. The exercise cannot remove every bias, but it makes it harder for the known ending to determine the entire judgment.

When the outcome is already known to everyone, ask reviewers to cite only information that was available before the decision.

4. Evaluate the process

Examine whether the objective was clear, the right people were involved, credible alternatives were considered, evidence quality was understood, risks and downstream effects were addressed, and the decision stayed within its authority.

Check whether the process met the standard appropriate to the decision. A small reversible experiment and an irreversible high-impact commitment should not require the same review.

Name omissions directly. A process is not sound merely because it contains a meeting, a spreadsheet, or a written rationale.

5. Describe the outcome separately

State what happened without immediately assigning praise, blame, luck, or skill.

Include the expected and unexpected effects, timing, affected people, costs, benefits, control failures, and information that became available afterward. Separate measured results from impressions and secondhand reports.

One outcome may be noisy. A repeated pattern across comparable decisions carries more weight.

6. Compare expectation with result

Ask whether the outcome fell inside a range the decision maker had considered. If probabilities or scenarios were recorded, compare them with what occurred without pretending one observation can validate the estimate.

Look for surprise. Was a known risk underestimated? Did an assumption fail? Did implementation differ from the approved decision? Did an external event dominate the result? Did the process ignore something that was reasonably knowable?

The purpose is not to divide the world perfectly into luck and skill. It is to identify which parts can be improved.

7. Decide what changes

Write the specific update to the next decision. That may be a better data source, another reviewer, a narrower permission, an earlier test, a clearer threshold, a new control, or no process change at all.

Assign an owner and a review date when the change matters operationally. If the lesson cannot be observed in future behavior, it is probably too vague.

8. Preserve the review

Keep the original decision record, the outcome record, and the review as separate but linked artifacts. Do not overwrite the earlier rationale with what is known now.

That history allows future reviewers to see whether judgment improved, whether estimates became better calibrated, and whether the same failure returned.

The boundary

This is a general learning method. It does not replace investigation, audit, clinical review, legal analysis, financial controls, insurance or underwriting standards, safety management, employment procedures, regulatory obligations, or the accountable authority for a consequential decision.

The standard remains simple: judge the decision using the evidence available then, use the outcome as evidence now, and document what the next decision should do differently.

Sources

Follow the evidence.

  1. WSOP report on the 2010 heads-up championshipwsop.com
  2. Outcome-bias replication and extensionspmc.ncbi.nlm.nih.gov
  3. Brier probability-forecast verification paperjournals.ametsoc.org
  4. WSOP official media guidewsop.com
  5. Thinking in Bets publisher recordpenguinrandomhouse.com
  6. WSOP Tournament of Champions historywsop.com
  7. Prospective hindsight studyonlinelibrary.wiley.com
  8. Outcome bias in decision evaluationpubmed.ncbi.nlm.nih.gov
  9. NFL account of the Malcolm Butler interceptionnfl.com
  10. Spotify episodeopen.spotify.com
How to Review a Decision Without Outcome Bias