Discrete Probability Space

probability-theory

Definition

Discrete Probability Space

A discrete probability space is a probability space $(\Omega ,\mathcal{P}(\Omega ),P)$ whose sample space is finite or countably infinite. Writing $\Omega =\{{\omega }_i\mid i\in I\}$ for $I\subseteq \mathbb{N}$, it is determined by the point masses

$$p_i=P(\{{\omega }_i\})\ge 0,\sum _{i\in I}{p}_i=1,P(A)=\sum _{{\omega }_i\in A}{p}_i.$$

Why Point Masses Determine Every Event

Because $\Omega$ is countable, every event $A\subseteq \Omega$ is a countable disjoint union of singleton outcomes:

$$A=\bigcup _{{\omega }_i\in A}\{{\omega }_i\}.$$

Countable additivity then gives

$$P(A)=P\left(\bigcup _{{\omega }_i\in A}\{{\omega }_i\}\right)=\sum _{{\omega }_i\in A}P(\{{\omega }_i\})=\sum _{{\omega }_i\in A}{p}_i.$$

Thus no additional probabilities need to be specified once the singleton masses are known.

Construction from Weights

Conversely, suppose numbers $(p_i{)}_{i\in I}$ satisfy

$$p_i\ge 0,\sum _{i\in I}{p}_i=1.$$

Then

$$P(A)=\sum _{{\omega }_i\in A}{p}_i$$

defines a unique probability measure on $\mathcal{P}(\Omega )$. Non-negativity follows from $p_i\ge 0$, normalisation follows by taking $A=\Omega$, and countable additivity follows because disjoint events contain disjoint collections of outcomes.

Contrast with a Discrete Random Variable

Warning

A discrete probability space has a finite or countably infinite underlying sample space $\Omega$. A discrete random variable instead has a finite or countable image $X(\Omega )$. A discrete random variable may therefore be defined on an uncountable probability space.

Example

Tosses until the first head

Let

$$\Omega =\{1,2,3,\dots \},$$

where outcome $n$ means that the first head occurs on toss $n$. For a fair coin, assign

$$P(\{n\})=2^{-n}.$$

These masses are normalised because

$$\sum _{n=1}^{\infty }{2}^{-n}=1.$$

The probability that the first head occurs on an even-numbered toss is

$$P(\{2,4,6,\dots \})=\sum _{k=1}^{\infty }{2}^{-2k}=\sum _{k=1}^{\infty }{4}^{-k}=\frac{1}{3}.$$