Hyperbolic Tangent

math analysis deep-learning

Definition

Hyperbolic Tangent

The hyperbolic tangent () is a non-linear activation function that maps real-valued inputs to the interval . Formally:

where is the sigmoid function.

Properties

Zero-Centring: Unlike the sigmoid function, is zero-centred, meaning its output range is symmetric around the origin. This property often leads to more efficient gradient-based optimisation during the training of deep artificial neural networks.

Vanishing Gradients: Similar to the sigmoid, saturates for large absolute values of , where the derivative approaches zero. This can impede training in very deep architectures, leading to the preference for ReLU.