A patient tests positive for a rare disease. The test is 99% accurate. What's the probability they actually have the disease? Most people instinctively answer '99%.' The correct answer is often closer to 9%—or even lower. This counterintuitive result stems from Bayes' Theorem, the fundamental rule for updating beliefs when new evidence arrives.
The Theorem
Bayes' Theorem states: P(A|B) = P(B|A) × P(A) / P(B). In words: the probability of hypothesis A given evidence B equals the likelihood of observing B if A were true, times our prior probability of A, divided by the overall probability of observing B. The four components each carry meaning. P(A) is the prior—our initial belief before seeing evidence. P(B|A) is the likelihood—how probable the evidence is if A is true. P(B) is the marginal probability. P(A|B) is the posterior—our updated belief after seeing evidence.
The Medical Test Example
Suppose a disease affects 1 in 1,000 people (prior = 0.001). A test has 99% sensitivity (true positive rate) and 99% specificity (true negative rate). Given a positive test: P(disease|positive) = 0.99 × 0.001 / [(0.99 × 0.001) + (0.01 × 0.999)] = 0.00099 / 0.01098 ≈ 0.09, or about 9%. The disease is so rare that even with a highly accurate test, most positives are false. This explains why mass screening programs for rare conditions require careful analysis—the false positive burden may outweigh the true positive benefits.
Why Prior Probability Matters
This example illustrates a crucial principle: the prior probability dramatically affects the posterior. When the prior is very small (rare disease), even strong evidence can leave the posterior surprisingly low. In populations with higher baseline rates—symptomatic patients referred to specialists, for instance—the same test performs very differently because the prior is much higher. This is why diagnostic accuracy statistics quoted out of context can be deeply misleading: a test's performance depends on who is being tested.
Bayesian Reasoning in Practice
Bayesian reasoning extends far beyond medical tests. Spam filters use it to classify emails: given certain words appear, what's the probability the email is spam? Scientists use it to update theories as new experimental data arrives. In machine learning, Naive Bayes classifiers apply the theorem to categorize documents and images. Financial analysts update probability estimates for economic events as new data emerges. The key insight in all these applications is the same: new evidence should update our beliefs in proportion to both the strength of the evidence and our existing knowledge.
The Frequentist vs. Bayesian Debate
Bayesian probability is philosophically controversial. The frequentist school holds that probability represents long-run frequencies of events, making it inappropriate to assign probabilities to fixed parameters. The Bayesian school treats probability as a degree of belief, allowing priors and posteriors over any uncertain quantity. This divide has practical consequences: Bayesian methods require specifying priors (which critics call subjective), while frequentist methods avoid priors but struggle with questions like 'what is the probability this hypothesis is true?' Both approaches have genuine strengths and remain active areas of statistical practice.
Conclusion
Bayes' Theorem is the mathematics of rational belief updating. Its most powerful lesson is that evidence doesn't exist in a vacuum—its interpretation depends entirely on what we already know. A positive test means something very different in a healthy population versus a symptomatic patient. By quantifying how evidence should shift beliefs, Bayes' Theorem provides a rigorous foundation for learning from data and a corrective for the intuitive reasoning that leads us to dramatically overestimate the meaning of a single positive result.