Sigmoid

math analysis machine-learning

Definition

Sigmoid Function

The sigmoid function (specifically the logistic function) is an S-shaped function that maps any real-valued input to the interval $(0,1)$. Formally:

$$\sigma (z)=\frac{1}{1+e^{-z}}=\frac{e^z}{1+e^z}$$

Properties

Range and Limits: The function is bounded such that $\sigma (z)\in (0,1)$, with ${\lim }_{z\to \infty }\sigma (z)=1$ and ${\lim }_{z\to -\infty }\sigma (z)=0$.

Derivative: The derivative is elegantly expressed in terms of the function itself: ${\sigma }^{\prime }(z)=\sigma (z)(1-\sigma (z))$, which facilitates efficient backpropagation in neural networks.

Symmetry: The function satisfies the symmetry property $1-\sigma (z)=\sigma (-z)$.