Hebbian Learning

neuroscience machine-learning cognition

Definition

Hebbian Learning

Hebbian learning, introduced by Donald Hebb in The Organization of Behavior (1949), is the principle that synaptic connections between neurons are strengthened when the pre- and postsynaptic neurons are repeatedly active together. In Hebb’s words:

“When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.”

The principle is often summarised as cells that fire together wire together. In its simplest mathematical form:

$$\Delta w_{ij}=\eta \cdot x_i\cdot y_j$$

where $w_{ij}$ is the weight from neuron $i$ to neuron $j$, $x_i$ and $y_j$ are their activations, and $\eta$ is a learning rate.

Limitations

Hebbian learning in its pure form is unstable. Stronger connections drive more firing, which strengthens them further — a positive feedback loop with no bound. There is no competition between inputs, no mechanism for weight reduction, and no forgetting. Left unchecked, weights grow without limit.

Variants

Later modifications introduced stability:

• Weight normalisation — rescaling weights to prevent unbounded growth.
• Sanger’s rule (generalised Hebbian learning) — distributes an input’s contribution incrementally over multiple outputs, introducing competition.
• Oja’s rule — adds a decay term that normalises the weight vector, converging to the first principal component of the input.

Role

Hebbian learning bridges the McCulloch–Pitts neuron and the connectionist machine. The McCulloch–Pitts neuron could compute but not learn — its connections were fixed. Hebb gave the synapse plasticity: a modifiable weight whose strength reflects the correlation history of the neurons it connects. This single mechanism underlies learning in biological brains and, in modified form, weight updates in artificial neural networks.