Bayes Theorem

probability-theory bayesian-statistics

Definition

Bayes Theorem

Rewriting the definition of the conditional probability for $P(A)>0$ and $P(B)>0$ gives the multiplication theorems:

$$\begin{array}{rl}P(A\cap B)& =P(A\mid B)\cdot P(B)\\ P(B\cap A)& =P(B\mid A)\cdot P(A)\end{array}$$

which leads to Bayes Theorem:

$$P(A\mid B)=\frac{P(B\mid A)\cdot P(A)}{P(B)}$$

The Bayes Theorem describes the probability of an event based on prior knowledge (before observation) and allows to update the prior knowledge after observation to the posterior (after observation).

$$\text{underbracket}{P(A\mid B)}_{\text{Posterior}}=\frac{P(B\mid A)\cdot \text{overbracket}{P(A)}^{\text{Prior}}}{P(B)}$$

Law of Total Probability

Updating Beliefs

Combing the law of total probability gives a partition $H_1,H_2,\dots$ of $\Omega$ with Bayes Theorem, we get the alternative form:

$$P(H_i\mid B)=\frac{P(B\mid H_i)\cdot P(H_i)}{P(B)}=\frac{P(B\mid H_i)\cdot P(H_i)}{{\sum }_{j=1}^{\infty }P(B\mid H_j)\cdot P(H_j)}$$

where the events $H_i$ are often referred to as Hypothesis. We call $P(H_i)$ the prior probability and $P(H_i\mid B)$ the posterior probability.