Possible Worlds and the Axioms

Roll two dice. How many different things can happen?

Most people answer 21, by counting combinations: a 2 and a 5 is the same roll as a 5 and a 2. The right answer is 36, and the gap between those two numbers is the whole reason this lesson exists. Get the list of things that can happen wrong, and every probability you compute afterwards is wrong with it.

Nothing here needs any probability you do not already have. There are two rules, and everything else in the lesson is derived from them in front of you.

Possible worlds

Russell and Norvig frame probability in terms of possible worlds: one complete way things could turn out. Where a logical statement rules some worlds out, a probabilistic statement says how likely each one is.

Write the set of all of them as $\Omega$, the capital Greek letter omega, called the sample space. A single world in it is $\omega$, lowercase omega. The worlds have to be:

• mutually exclusive, meaning no two of them can both happen, and
• exhaustive, meaning one of them definitely does.

Back to the dice. If you can tell the two dice apart, a world is an ordered pair: which number the first die shows, then the second. That gives

$$(1,1),\ (1,2),\ \dots,\ (6,6), \qquad 6 \times 6 = 36 \text{ worlds}.$$

The count of 21 treats $(2,5)$ and $(5,2)$ as one world. They are two, and lumping them together quietly flattens a real difference: a mixed pair like $\{2,5\}$ really is twice as likely as a double like $(3,3)$, yet the 21-world model hands both the same $1/21$. This is the single most common slip in the subject, and it happens before any formula is written down.

The two axioms

$$0 \le P(\omega) \le 1 \quad\text{for every }\omega, \qquad \sum_{\omega \in \Omega} P(\omega) = 1 .$$

Read them in words. The first says no world is less likely than impossible or more likely than certain. The second says the probabilities of all the worlds add up to one, which is a formal way of saying something happens.

That is the entire foundation. If the dice are fair, then by symmetry every world carries the same probability, and the second axiom forces that shared value to be $1/36$.

Events

An event is just a set of worlds: "the total is 7" is the event containing the six worlds $(1,6), (2,5), \dots, (6,1)$. Its probability is the sum over the worlds inside it:

$$P(a) = \sum_{\omega \in a} P(\omega) .$$

This is the definition. The school rule, "favourable outcomes over total outcomes", is only the special case where every world happens to be equally likely. A loaded die still has a perfectly good probability model; it just assigns unequal $P(\omega)$, and the sum above still works.

Deriving the complement rule

Here is the first derivation, and it is two lines. Let $\neg a$ mean "$a$ does not happen". The worlds where $a$ holds and the worlds where $\neg a$ holds share nothing and between them cover all of $\Omega$. So splitting the second axiom's sum into those two groups gives

$$P(a) + P(\neg a) = 1 \quad\Longrightarrow\quad P(\neg a) = 1 - P(a) .$$

Nothing was assumed beyond the axioms. That is the pattern for everything that follows: derive, do not memorise.

The addition rule

What about "$a$ or $b$", when the two can both happen at once? Adding their probabilities counts the worlds they share twice, so subtract that overlap exactly once:

$$P(a \text{ or } b) = P(a) + P(b) - P(a \text{ and } b) .$$

Worked example. Draw one card from 52. There are 13 hearts and 4 Kings, so $P(\text{heart}) = 13/52$ and $P(\text{King}) = 4/52$. Exactly one card is both, the King of Hearts, so

$$P(\text{heart or King}) = \frac{13}{52} + \frac{4}{52} - \frac{1}{52} = \frac{16}{52} = \frac{4}{13} .$$

Adding naively gives $17/52$. Count the cards by hand and you find 16, not 17: the King of Hearts was counted once as a heart and once as a King.

Count them yourself. The figure opens on this example, with every card drawn and the King of Hearts highlighted as the one world the naive sum counts twice. Change Event A and Event B, or switch to Two dice: 36 worlds, and each time the direct count of "A or B" is set against the addition rule.

Conditioning and independence

Learning that $b$ happened shrinks the set of worlds still in play, and probability has to be renormalised over what is left. That is what conditional probability is:

$$P(a \mid b) = \frac{P(a \text{ and } b)}{P(b)}, \qquad P(b) > 0 .$$

Read $P(a \mid b)$ as "the probability of $a$ given $b$". The denominator is the renormalisation: you are now living inside the worlds where $b$ holds. Rearranging gives the multiplication rule $P(a \text{ and } b) = P(b)P(a \mid b)$, which is always true.

Events are independent when learning $b$ tells you nothing about $a$, that is when $P(a \mid b) = P(a)$. Only then does the multiplication rule collapse to the familiar $P(a)P(b)$.

Check independence, never assume it. Multiplying probabilities is valid only under independence. When a common cause drives many outcomes at once, the events are strongly dependent, and multiplying badly understates the chance that many of them occur together.

That last sentence is not a technicality. It is the failure mode behind mispriced insurance portfolios and underestimated system outages, and you will meet it again in Bayes and the Base Rate.

See it move. Reshape a distribution and watch where its mass sits: the same axioms, very different pictures.

Try it live. Sample from two standard families, check their moments, and watch the central limit theorem arrive:

Before the quiz

Make sure you can, without looking back: count the worlds for two dice and say why it is 36 rather than 21, derive the complement rule from the axioms, and say what the addition rule subtracts and why. The longer treatment, with more worked examples, is in Probability from Zero.