Central Limit Theorem

probability-theory

Definition

Central Limit Theorem

Let $X_1,X_2,\dots$ be a sequence of identically distributed random variables with finite variance

$$\mathbb{V}ar(X_i)<\infty \forall i\in \{1,2,\dots \}$$

and $\overline{X_n}$ be the sample mean of the first $n$ random variables, then:

$$\sqrt{n}\cdot (\overline{X_n}-\mathbb{E}[X_i])\stackrel{d}{\to }\mathcal{N}(0,\mathbb{V}ar(X_i))$$

Note that $(\overline{X_n}-\mathbb{E}[X_i])$ is the first central moment of the sample’s distribution.

Normal Distribution

In practice, for large $n$, the sample mean is approximately normal-distributed:

$$\overline{X_n}\approx \mathcal{N}\left(\mu ,\frac{{\sigma }^2}{n}\right)$$

Similarly, for the sum $S_n={\sum }_{i=1}^n{X}_i=n\cdot \overline{X_n}$, we have:

$$S_n\approx \mathcal{N}(n\mu ,n{\sigma }^2)$$

Thus, the standardised $\overline{X_n}$ and $S_n$ are approximately standard normal distributed:

$$\begin{array}{rl}\frac{\overline{X_n}-\mu }{\sigma /\sqrt{n}}& \approx \mathcal{N}(0,1)\\ \frac{S_n-n\mu }{\sqrt{n}\sigma }& \approx \mathcal{N}(0,1)\end{array}$$