Definition
Underfitting
Underfitting is a phenomenon in machine learning where a model is unable to capture the underlying structure of the data, resulting in poor performance on both the training dataset and unseen data. This typically occurs when the hypothesis class is too simple or when the training process is insufficient. Formally, underfitting is characterised by high bias.
Relation to Complexity
In the context of the bias-variance tradeoff, underfitting represents the low-complexity regime where the model’s approximation error dominates. Unlike overfitting, adding more data typically does not resolve underfitting; instead, the model capacity must be increased (e.g., using a higher-degree polynomial in regression).