Residual Block

machine-learning

Definition

Residual Block

Residual blocks are skip-connection blocks that learn residual functions with reference to the layer input, instead of learning unreferenced functions.

Formally, denoting the desired underlying mapping as $\mathcal{H}(x)$, we let the stacked nonlinear layers fit another mapping of $\mathcal{F}(x):=\mathcal{H}(x)-x$. The original mapping is recast into $\mathcal{F}(x)+x$. The $\mathcal{F}(x)$ acts like a residual, hence the name ‘residual block’.

¹

Footnotes

1. https://paperswithcode.com/method/residual-block ↩