Overfitting vs Underfitting: How ML Models Fail

Overfitting and underfitting are the two basic ways a machine learning model can fail to generalize: an overfit model memorizes the training data too closely, including its noise, and performs poorly on anything new; an underfit model fails to capture even the real pattern in the training data, and performs poorly everywhere, training and test alike. Nearly every model-tuning decision — how much data to collect, how complex a model to use, when to stop training — is ultimately about landing between these two failure modes.

What each failure looks like

The clearest way to tell them apart is by comparing performance on the data the model trained on versus data it hasn’t seen.

Underfitting: the model performs poorly on the training data itself. It hasn’t learned the underlying relationship at all — imagine fitting a straight line through data that clearly curves. The model’s error is high everywhere, because it’s too simple, or wasn’t trained long enough, to capture the actual pattern.

Overfitting: the model performs excellently on the training data but poorly on new data. It has learned the training set specifically, including quirks and noise that don’t represent any real, generalizable relationship — imagine a curve so flexible it wiggles through every single training point exactly, chasing noise instead of signal. The model’s error is low on training data and high on everything else, which is the telltale gap.

A well-fit model sits between the two: training and test error are both reasonably low and reasonably close to each other, meaning the model has learned the real pattern without also memorizing the noise.

Comparison

	Underfitting	Good fit	Overfitting
Training error	High	Low	Very low
Test error	High	Low	High
Gap between the two	Small (both bad)	Small	Large
Typical cause	Model too simple, undertrained	Matched to the data’s real complexity	Model too complex, overtrained
Fix direction	Increase capacity or training	—	Reduce capacity or add regularization

Why overfitting happens

Overfitting is fundamentally a mismatch between model capacity and data volume. A model with enough free parameters can, in principle, fit any finite training set arbitrarily well, including its noise — the training data always includes measurement error, sampling quirks, and outliers that don’t reflect the true underlying pattern. Given enough flexibility and enough training time, a model will happily fit those quirks too, since nothing in the training process distinguishes signal from noise on its own; it’s simply minimizing error on the data it’s shown.

Deep neural networks are especially prone to this because they’re deliberately over-parameterized — they often have far more parameters than training examples, which is what gives them their expressive power, but also what makes them capable of memorizing training data outright if nothing constrains them not to.

Common fixes for overfitting

• More training data. The single most reliable fix — a larger, more representative dataset makes it harder for the model to memorize specifics, since the same noise doesn’t repeat across a bigger sample.
• Regularization. Techniques that penalize model complexity during training, discouraging the model from relying too heavily on any single feature or fitting noise too precisely.
• Dropout. A common technique in neural networks: randomly disabling a fraction of neurons during each training step, which prevents the network from becoming overly reliant on any specific pathway through it.
• Early stopping. Monitoring performance on a held-out validation set during training and stopping once validation performance starts getting worse even as training performance keeps improving — the point where the model has stopped learning the real pattern and started memorizing.
• Simpler models. Sometimes the direct fix is reducing model capacity outright — fewer layers, fewer parameters — so there’s less room to memorize noise in the first place.

Common fixes for underfitting

Underfitting’s fixes largely run in the opposite direction:

• More model capacity. A larger model, more layers, or more parameters gives the model the expressive power to represent the actual pattern.
• Train longer. Sometimes the model simply hasn’t converged yet; more training iterations let it keep improving.
• Better features. If the input representation doesn’t contain the information needed to predict the target, no amount of model capacity will fix it — this is where feature engineering, or for modern deep learning, better vector embeddings and representations, matter more than raw model size.
• Reduce regularization. If regularization is too aggressive, it can suppress the model’s ability to learn real patterns, not just noise — dialing it back can help if underfitting followed from an overcorrection.

Why the training/validation/test split matters

None of this is diagnosable from training performance alone, which is why the standard practice is to hold out a validation set the model never trains on, and often a separate test set held out even from decisions like when to stop training. Judging a model purely by its training accuracy is exactly how overfitting goes unnoticed — the number looks great and the model is quietly useless on anything new. This is also the underlying justification for techniques like cross-validation, which give a more robust estimate of generalization performance than a single train/test split.

The takeaway

Underfitting means the model hasn’t learned the real pattern, visible as poor performance across the board; overfitting means it’s learned the training data too specifically, visible as a gap between excellent training performance and poor test performance. The fix runs in opposite directions for each — more capacity and training for underfitting, more data and regularization for overfitting — and neither is diagnosable without a held-out validation set that stays untouched by training. Most of practical model tuning is finding the point between the two where the model has learned the signal without also learning the noise.