Probability Distribution

math probability statistics

Definition

Probability Distribution

A probability distribution is a mathematical construct that assigns probabilities to the measurable subsets of a sample space. Formally, for a random variable $X$ defined on a probability space $(\Omega ,\mathcal{F},P)$, the distribution is the probability measure $P_X$ on the measurable space $(\mathcal{X},\mathcal{B})$ defined as the pushforward measure:

$P_X(B)=(X_{\ast }P)(B)=P(X^{-1}(B))$

for all $B\in \mathcal{B}$. It represents a theoretical model of the likelihood of outcomes, satisfying the Kolmogorov axioms.

Representation Complexity

For a joint distribution over $n$ independent binary (Boolean) random variables, the number of free parameters required to specify the distribution is $2^n-1$. This exponential growth motivates the use of Bayesian Networks to factorise the distribution using conditional independence.

Classification of Distributions

Discrete Distributions: These are characterised by a probability mass function (PMF), where the random variable takes values from a countable set.

Continuous Distributions: These are characterised by a probability density function (PDF), where the random variable takes values from an uncountable interval, and probabilities are obtained via integration.