Conditional probability

"When the facts change, I change my mind. What do you do, sir?" — Attributed to John Maynard Keynes¹

You're at a party, and someone suggests a game. They'll flip a coin twice, hidden from view. Then they'll tell you one piece of information about the outcome, and you have to guess the probability of various scenarios. "The first flip was heads," they say. What's the probability that both flips were heads?

If you answered 1/2 or 50%, congratulations—you intuitively understand conditional probability. But while this example seems trivial, similar reasoning patterns underlie everything from medical diagnoses to judicial verdicts to scientific breakthroughs. They're the hidden foundation of rational thought itself.

Most of us move through life accumulating beliefs through a messy, haphazard process. We believe things because our parents taught us, because we read them on a website with an authoritative design, or because believing them helps us fit in with our social group. We rarely update these beliefs in a systematic way when new evidence arrives. This is, to put it mildly, less than optimal.

Bayesians² argue that ideally, we should think of our beliefs as probabilistic statements about the world—statements that should be systematically updated when new evidence arrives. The mathematical tool for this updating process is conditional probability: the probability of an event occurring given that another event has already occurred.

Formally, the conditional probability of event A given event B is written as P(A|B) and calculated as:

P(A|B) = P(A ∩ B) / P(B)

Where P(A ∩ B) is the probability of both A and B occurring, and P(B) is the probability of B occurring. But what does this actually mean?

Shrinking Your Universe

When we condition on an event, we're effectively shrinking our universe of possibilities. If I tell you "it rained yesterday," all possibilities where it didn't rain yesterday are eliminated from consideration. We're now operating in a smaller universe—the universe where it rained yesterday.

To return to our opening example: - Before any information: the universe contains four equally likely outcomes - (H,H), (H,T), (T,H), (T,T) - After learning "first flip was heads": the universe shrinks to just (H,H) and (H,T) - Within this new, smaller universe, (H,H) occurs in 1 out of 2 cases, so P(both heads | first flip heads) = 1/2

This "universe shrinking" perspective makes conditional probability intuitive. You're not calculating anything mysterious—you're just counting possibilities in a restricted universe.

The Medical Test Paradox

But humans often get conditional probability catastrophically wrong. Consider the classic example:

1. A rare disease affects 1% of the population
2. A test for this disease has a 90% true positive rate (sensitivity)
3. The test has a 90% true negative rate (specificity)
4. Your test comes back positive

What's the probability you have the disease?

Most people—including many doctors³—guess around 90%. The actual answer is just 8.3%.

Why such a dramatic difference? Because we're neglecting crucial information: the base rate of the disease. Let's work through it:

• Population: 1,000 people
• Disease prevalence: 1% → 10 people have the disease, 990 don't
• Of the 10 disease carriers, 9 test positive (90% sensitivity)
• Of the 990 healthy people, 99 test positive (10% false positive rate)
• Total positive tests: 9 + 99 = 108
• Probability of disease given positive test: 9/108 ≈ 8.3%

This demonstrates why conditional probability is so counterintuitive. Our minds naturally focus on P(positive test | disease) when what we actually need is P(disease | positive test)—two entirely different quantities.

This problem is so common that we have a specific formula to handle it: Bayes' theorem. It allows us to "reverse" conditional probabilities:

P(A|B) = [P(B|A) × P(A)] / P(B)

In our medical example: - P(disease | positive) = [P(positive | disease) × P(disease)] / P(positive) - P(disease | positive) = [0.9 × 0.01] / [0.01 × 0.9 + 0.99 × 0.1] - P(disease | positive) = 0.009 / 0.108 ≈ 0.083 or 8.3%

This is perhaps the single most important formula for thinking clearly in an uncertain world. It's the mathematical foundation of changing your mind when confronted with evidence.

But Bayes' theorem isn't just useful for medical tests and coin flips. It provides a framework for thinking about belief updating in general:

P(hypothesis | evidence) = [P(evidence | hypothesis) × P(hypothesis)] / P(evidence)

Or in plain English:

Your belief after seeing evidence = How well the hypothesis predicts the evidence × Your prior belief / How likely that evidence was overall

This gives us a prescription for rational belief:

1. Start with a prior probability based on background knowledge
2. Calculate how likely the observed evidence would be under different hypotheses
3. Update your beliefs accordingly

When someone says "extraordinary claims require extraordinary evidence," they're essentially making a Bayesian argument: if P(hypothesis) is very low, then P(evidence | hypothesis) needs to be extraordinarily high to make P(hypothesis | evidence) substantial.

Living in Bayesworld

What would it look like to systematically apply conditional probability to everyday life? Consider:

• Politics: Instead of asking "Does this evidence support my side?" ask "How likely would I be to see this evidence if my view was wrong?"
• Relationships: When interpreting someone's behavior, consider both P(behavior | they're upset with me) and P(behavior | they're not upset with me)
• Career: Don't just consider P(success | taking this job), but also P(learning valuable skills | taking this job) × P(future success | learning those skills)

Even for decisions where formal calculation is impractical, the structure of conditional probability provides guardrails for clearer thinking.

Of course, there are limits to this approach. In the real world:

1. We rarely have precise probabilities to work with
2. Our hypotheses are vague and overlapping, not mutually exclusive
3. The evidence itself is often ambiguous
4. Our cognitive resources are finite

But these limitations don't invalidate the underlying principles. Even approximate Bayesian reasoning is far superior to the alternative: stubbornly clinging to prior beliefs regardless of evidence, or wildly oscillating between certainties based on the most recent thing we heard.

As Julia Galef points out in her book The Scout Mindset, the goal isn't perfect Bayesian updating—it's developing the habit of treating your beliefs as hypotheses that could be wrong, and approaching evidence as an opportunity to refine them rather than a threat to be neutralized.

The principles of conditional probability extend far beyond individual reasoning:

• Scientific method: The entire structure of hypothesis testing can be viewed as a formalized way of updating conditional probabilities
• Machine learning: Bayesian networks and probabilistic programming allow computers to reason about uncertainty in ways similar to ideal Bayesian reasoners
• Financial markets: Options pricing models incorporate conditional probabilities of various market outcomes
• Public health: Epidemic models use conditional probabilities to predict disease spread under various intervention scenarios

Perhaps most importantly, understanding conditional probability helps us recognize when others are misusing statistics, either through innocent confusion or deliberate manipulation.

The study of conditional probability isn't just an abstract mathematical exercise—it's a practical framework for navigating an uncertain world. By thinking explicitly about how new evidence should update our beliefs, we can become less wrong over time.

As you go through life, try asking yourself:

If my belief about X were wrong, would I be likely to encounter the evidence I'm seeing?

It's not a perfect fix for our cognitive biases, but it's a start. And in a world where being slightly less wrong can mean the difference between success and catastrophic failure, it's worth the effort.

In closing, let me paraphrase E.T. Jaynes⁴: Probability theory isn't arbitrary. It's the unique extension of deductive logic into the realm of uncertainty. Conditional probability isn't just one way to reason about evidence—properly understood, it's the only coherent way.

And that's a proposition I'd assign a high probability to being true.