Definition
Hidden Layer
A hidden layer is any layer of a multi-layer perceptron that sits between the input and output layers. It consists of hidden neurons whose activations are not directly supervised by the training data.
Hidden layers are the source of the network’s expressive power — they compose intermediate representations that the output layer combines into the final prediction. They are also the source of the credit assignment problem: no training label exists for any hidden neuron.