It is tempting to imagine an artificial neural network as a stack of boundary-drawing machines: the first layer draws one boundary, the second layer draws a better boundary, and the last layer draws the final boundary.
That picture is a little too neat. A layer usually does not draw one boundary. It holds many perceptrons or neuron-like units, and each unit performs its own small test.
For a perceptron-like hidden layer,
Each has its own hyperplane . The layer does not make one cut through the space. It lays down a bundle of cuts, one per neuron.
This is the gentler picture: a layer is not one pen stroke. It is a small bundle of rulers laid across the space. Each ruler asks whether the point is on one side or the other.
A layer returns a code
After the tests are made, the layer returns a vector of answers:
This vector is a new description of the same point. The original point might have been described by coordinates . After the hidden layer, it is described by which hidden tests fired.
The next layer does not see the old drawing directly. It sees this code. It draws its own hyperplanes in the space of previous activations.
That is the shift that makes depth powerful. The first layer cuts the original ground. The next layer cuts a map made from those cuts. Deeper layers cut maps of maps.
The final boundary belongs to the whole network
The final decision boundary is where the network’s output changes class. It is produced by the composition of all layers, not by one hidden layer alone.
A layer does not usually draw one decision boundary. It draws many feature boundaries or tests, one per neuron, and the following layers combine them into the network’s final decision boundary.
This keeps the geometry honest. Neurons make local tests. Layers collect those tests into representations. The network as a whole draws the boundary that matters for classification.