Thinking in probability

Before the theorems comes a habit of mind. This chapter fixes what a probability actually is — a long-run frequency, a degree of belief, and above all a number you should be willing to bet at — and then installs the working practices that make the rest of the series land: name the experiment, write the sample space, translate percentages into natural frequencies, and simulate whenever intuition and algebra disagree. None of this is decoration. Most wrong answers in probability are not calculation errors; they are answers to a different, unstated question.

Two readings of one number#

The frequentist reading: probability is long-run relative frequency. Say the experiment can be repeated independently under identical conditions — flips of a coin, hands of a shuffled deck. Then $P(A)$ is the value the running fraction of occurrences settles toward:

The Bayesianreading: probability is a degree of belief held by a particular agent with particular information — perfectly meaningful for one-off events (“this strategy has a real edge”, “this election goes to the incumbent”) where no repetition is available. Different agents with different information may legitimately hold different probabilities for the same event.

The crucial fact is that both readings obey the same arithmetic — the Kolmogorov axioms, stated and mined in axioms, conditioning & Bayes. Frequencies obey them because proportions do: counts of disjoint events add. Beliefs obey them because betting prices that don’t can be exploited (next section). So the machinery of this series works verbatim under either interpretation, and we will move freely between them: frequencies when we simulate, beliefs when we price an uncertain one-off. The bridge between the two — the guarantee that frequencies converge to the number beliefs should track — is the law of large numbers, proved in LLN & the CLT and visible below.

Odds, fair bets, and implied probability#

Betting language predates probability theory and remains the fastest way to move numbers around. Odds of $a:b$ against an event mean: stake $b$ to win $a$. The conversions to memorise cold:

A bet is fair when its expected profit is zero: staking $p$ to win $1-p$ on an event of probability $p$ has expectation $p(1-p)-(1-p)p=0$. Run the logic backwards and any quoted price becomes an implied probability: a contract paying $1 if the Fed cuts, trading at $0.62, is the market asserting $P(\text{cut})\approx 0.62$. Odds of 3:1 against a horse assert $P=\frac{1}{4}$.

Name the experiment, write the sample space#

The single most protective habit in probability: before computing anything, say precisely what is being done — the experiment — and list what can happen — the sample space $\Omega$. Most famous “paradoxes” are not deep; they are two parties silently using different experiments. “A family has two children and one is a boy” means one thing if a child was met at random and another if the family was selected for containing a boy — different experiments, different answers ($\frac{1}{2}$ versus $\frac{1}{3}$), both computed honestly in the conditioning chapter. Until the experiment is named, the question has no answer.

Where raw intuition fails — and why#

Human probability intuition was trained on small samples and vivid stories, and it fails in patterned, predictable ways. Knowing the failure modes by name is half the cure:

• The law of small numbers.We expect small samples to look like the population. They don’t: 10 flips of a fair coin give 70%+ heads about 17% of the time. A hospital averaging 15 births a day will record far more 60%-boy days than one averaging 45 — smaller samples fluctuate more, a fact quantified by the $1/\sqrt{n}$ scaling in LLN & the CLT. Every “hot streak” read off a 20-trade sample commits this error.
• Representativeness.HTHTTH looks “more random” than HHHHHH, so people call it more likely. Both are single outcomes of the same experiment, probability $(1/2{)}^6=1/64$ each. The intuition is answering a different question — how likely is a sequence of that general appearance — and Problem 4 shows that question has a respectable answer the intuition is crudely tracking. The trap is swapping questions without noticing.
• Stories over frequencies. Adding detail makes a scenario more plausible but strictly less probable: “a crash triggered by a failed bond auction” is a subset of “a crash”, yet the vivid version gets the higher number from most people (the conjunction fallacy). Monotonicity — $A\subseteq B\Rightarrow P(A)\le P(B)$, proved from the axioms in the next chapter — is the two-line refutation.
• Base-rate neglect.A striking piece of evidence (a positive test, a stellar backtest) swamps the boring background rate of the thing being tested for. The full accounting is Bayes’ theorem, dissected with numbers in axioms, conditioning & Bayes; the preview is that when genuine cases are rare, most alarms are false, no matter how good the alarm.

Think in natural frequencies#

Percentages are a compressed format the brain decodes badly; whole numbers of cases decode cleanly. The repair, due to Gigerenzer: restate every percentage problem as counts in a concrete population — “out of 10,000 cases…” — and watch the confusion evaporate. Take the classic screening setup (1% prevalence, 99% sensitivity, 5% false-positive rate) that defeats a majority of physicians when posed in percentages. In natural frequencies:

• Out of 10,000 people, 100 have the condition; 9,900 do not.
• Of the 100 with it, 99 test positive.
• Of the 9,900 without it, 5% — 495 — also test positive.
• So the positives number $99+495=594$, of whom $99/594=\frac{1}{6}\approx 16.7\%$ actually have the condition.

No formula was needed — the frequency format isthe computation, because it carries the base rate (100 versus 9,900) in plain sight instead of hiding it inside a conditional percentage. The habit for this series: whenever conditional probabilities tangle, drop to a population of 10,000 and count. Bayes’ theorem, when we prove it in axioms, conditioning & Bayes, will simply be this table turned into algebra.

Simulation as ground truth#

This series has a standing rule: when intuition and algebra disagree, simulate. A Monte Carlo run is an experiment — probability practised as an experimental science — and it is astonishingly cheap insurance: a five-line script settles arguments that verbal reasoning can sustain for hours. The error of a simulated probability shrinks like $\sqrt{p(1-p)/n}\le \frac{1}{2\sqrt{n}}$ (the CLT again, and the same $1/\sqrt{n}$ band as the coin-flip figure), so $n=10^5$ trials pins any probability to about $\pm 0.003$ — ample to catch every conceptual error, which tends to be off by far more. The full craft — variance reduction, convergence diagnostics, when Monte Carlo beats quadrature — is developed in Monte Carlo methods; here we establish the reflex.

A demonstration on two awkward-feeling questions. First: roll a die 10 times — what is the chance some adjacent pair of rolls is equal? Intuition flails; the algebra is a clean chain-rule product (each of the 9 gaps independently avoids a match with probability $5/6$): $1-(5/6{)}^9\approx 0.8062$. Second: what is the fair price of this game — roll a die, then either keep the face value in dollars or re-roll once and keep the second roll? Optimal play re-rolls 1, 2, 3 (anything below the re-roll’s expected value $3.5$), giving $\frac{4+5+6}{6}+\frac{3}{6}\cdot 3.5=2.5+1.75=4.25$. The simulation checks both:

import numpy as np

rng = np.random.default_rng(7)
trials = 200_000

# 1) P(at least one equal adjacent pair in 10 die rolls)
rolls = rng.integers(1, 7, size=(trials, 10))
has_pair = (rolls[:, 1:] == rolls[:, :-1]).any(axis=1)
exact = 1 - (5 / 6) ** 9
print(f"adjacent pair: sim {has_pair.mean():.4f}   exact 1-(5/6)^9 = {exact:.4f}")

# 2) Fair price of: roll a die, keep it, or re-roll once and keep that
first = rng.integers(1, 7, trials)
second = rng.integers(1, 7, trials)
payoff = np.where(first >= 4, first, second)   # optimal: re-roll 1, 2, 3
print(f"re-roll game:  sim {payoff.mean():.4f}   exact 17/4  = {17 / 4:.4f}")

# Monte Carlo standard error ~ sqrt(p(1-p)/n): here about 0.0009 for the
# probability — a conceptual error (wrong sample space, wrong conditioning)
# is typically 10-100x larger, so the sim catches it instantly.

Note the division of labour: the simulation does not replace the exact answer — it audits it. When the two agree to three decimals you have independent confirmation of both the algebra and the code; when they disagree, one of your two models of the experiment is wrong, and finding which is where the real learning happens. Problem 5 makes you run this loop yourself.

Calibration — making “70% confident” mean something#

If probabilities are prices, a forecaster can be audited like a market-maker. A forecaster is calibratedwhen, among all the occasions she says “70%”, the event happens about 70% of the time — and likewise at every stated level. Calibration is what elevates “70% confident” from mood to measurement: it is a frequentist property of your Bayesian utterances, the two readings of probability shaking hands.

The standard audit is the Brier score — mean squared error between forecasts $f_i\in [0,1]$ and outcomes $o_i\in \{0,1\}$:

Worked micro-example: forecasts $0.9,0.7,0.3$ with outcomes $1,1,0$ score $\frac{0.01+0.09+0.09}{3}\approx 0.063$ — far better than the 0.25 of a permanent fence-sitter. The Brier score is a proper scoring rule: your expected score is minimised by reporting your true belief, so it cannot be gamed by strategic exaggeration or hedging — honesty is the optimal policy, by construction. It also decomposes into calibration (are your 70%s really 70%?) plus resolution (do your forecasts actually separate the 1s from the 0s?), so the audit tells you whichskill you lack. A trading model’s predicted win probabilities deserve exactly this audit before its P&L is believed — a theme the applied series returns to with real backtests.

The road from here#

The series builds in deliberate order. First the grammar: counting & combinatoricsmakes “equally likely outcomes” a computational tool, and axioms, conditioning & Bayesturns today’s informal rules — prices must cohere, learning shrinks the sample space, base rates dominate — into theorems. Then the objects: random variables, expectation and variance, the named distributions, and joint behaviour. Then the payoff, the intuition toolkit of Part II: symmetry arguments that answer in one line what integrals answer in ten, indicator tricks, conditioning strategies, and the limit theorems — LLN & CLT — that explain when averages can be trusted. Throughout, simulation rides shotgun as the experimental check on every claimed answer. The destination is not a formula sheet; it is the ability to look at an uncertain situation, name the experiment, and price it.

Practice problems#

Five problems to test the habits before the machinery arrives — each is solvable with nothing but today’s tools: honest sample spaces, odds arithmetic, natural frequencies, and simulation.

Next:the “equally likely outcomes” promise gets its engine — permutations, combinations, and the art of counting without listing: Counting & combinatorics.