Conditional Probability

probability-theory

Definition

Conditional Probability

Conditional probability is the probability of an event after restricting attention to the case where another event is known to occur.

In a probability space $(\Omega ,\mathfrak{S},P)$, let $A,B\in \mathfrak{S}$ be events with $P(B)>0$. The conditional probability of $A$ given $B$ is

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)}.$$

The event $B$ becomes the new effective sample space. Within that restricted world, $A$ happens exactly on the overlap $A\cap B$.

Derivation

Conditioning on $B$ means replacing the original world $\Omega$ by the smaller world $B$. Any event outside $B$ is now irrelevant, because it contradicts the information that $B$ occurred.

For an event $A$, the part of $A$ that remains possible after conditioning on $B$ is therefore

$$A\cap B.$$

To turn this restricted probability into an ordinary probability again, the new world $B$ must have total probability $1$. So we rescale all probabilities inside $B$ by the same normalising factor $c$:

$$P_B(C)=cP(C\cap B).$$

The normalisation condition is

$$P_B(B)=1.$$

Substituting $C=B$ gives

$$1=P_B(B)=cP(B\cap B)=cP(B).$$

Since $P(B)>0$, this forces

$$c=\frac{1}{P(B)}.$$

Thus the probability of $A$ in the restricted world $B$ is

$$P_B(A)=\frac{P(A\cap B)}{P(B)}.$$

This restricted probability measure is written as $P(A\mid B)$, giving

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)}.$$

Independence

If two events $A$ and $B$ are independent, i.e. $A\perp B$, and $P(B)>0$, then:

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)}=\frac{P(A)\cancel{\cdot P(B)}}{\cancel{P(B)}}=P(A)$$

Therefore:

$$P(A\mid B)=P(A)⟺A\perp B$$