What is generalization in machine learning?

Generalization in machine learning is a trained model's ability to make accurate predictions on new, unseen data (i.e., data it was not exposed to during training). A model that generalizes well has learned the underlying patterns and relationships in its training data rather than memorizing specific examples. Without generalization, a model may perform well on the data it was trained on but fail when applied to real-world inputs. This article covers the definition of generalization, the failure modes of overfitting and underfitting, how different algorithms approach the problem, and the techniques practitioners use to improve generalization.

Key concepts

• Generalization: A measure of how well a trained model extends its learned patterns to new, unseen data, as opposed to memorizing the training examples it was exposed to
• Overfitting: A condition in which a model becomes too closely fitted to its training data, capturing noise and random variation instead of underlying patterns, causing poor performance on new data
• Underfitting: A condition in which a model is too simple to capture the structure of the data, resulting in high error on both training data and new data
• Algorithm-specific tendencies: Different ML algorithms carry different default risks of overfitting or underfitting based on their complexity and architecture
• Generalization techniques: Methods including regularization, cross-validation, data augmentation, feature engineering, and ensemble methods that help models perform more reliably on unseen data

Why generalization matters in machine learning

The practical purpose of generalization follows directly from how machine learning models are used. A model is trained on a finite, controlled dataset but deployed against data it has never seen. A spam classifier trained on a labeled email corpus must correctly classify new emails as they arrive. A recommendation model trained on historical purchases must suggest items to users who may have different histories or preferences than anyone in the training set.

In each case, the model's usefulness depends entirely on its ability to apply what it learned to inputs outside its training distribution. A model that cannot do this is not a useful model regardless of how well it performs during training. Generalization is the standard by which a model's real-world effectiveness is measured.

What is overfitting and underfitting in machine learning?

Generalization fails in two directions, depending on whether the model is too complex or too simple for the data it is learning from.

Overfitting

Overfitting occurs when a model learns the training data too precisely. Rather than identifying the patterns that distinguish signal from noise, an overfit model memorizes specific training examples, including their noise, outliers, and random variation. The result is a model that achieves high accuracy on the training set but performs poorly on new data, because the patterns it learned do not hold outside the training distribution.

Overfitting is more likely when a model has high capacity relative to the size of the training dataset. Deep decision trees, high-degree polynomial models, and large neural networks are all prone to overfitting when training data is limited, because the model can express a wide variety of functions that fit the training examples without capturing any generalizable pattern.

Underfitting

Underfitting occurs when a model is too simple to represent the structure of the data. An underfit model makes overly restrictive assumptions about the data's form, and this shows up in the training error: The model performs poorly even on examples it has already seen. Unlike overfitting, underfitting is typically visible immediately during training and does not require a separate evaluation set to detect.

Underfitting is associated with high bias. When a model's assumptions are too restrictive relative to the true complexity of the data, more training will not close the gap. The solution is generally a more expressive model rather than additional training iterations.

How do different machine learning algorithms affect generalization?

Different algorithms carry different tendencies toward overfitting or underfitting, based on their underlying structure and default complexity.

Decision trees can grow arbitrarily deep, fitting every training example precisely given sufficient depth. Without constraints on tree depth or minimum leaf size, a decision tree will overfit. Pruning and maximum depth limits are the standard approaches for controlling generalization in tree-based models.

Support vector machines are sensitive to the choice of kernel and regularization parameter. A high-degree polynomial or radial basis function kernel can produce highly complex decision boundaries that overfit when the data is not well-suited to that level of complexity. Adjusting the regularization parameter C controls the tradeoff between fitting the training data tightly and maintaining a simpler decision boundary.

Neural networks, particularly deep ones, have very high capacity. They can model complex patterns in training data but are prone to overfitting when the network architecture is larger than the training data can support. Dropout regularization, early stopping, and weight decay are the primary techniques for improving generalization in neural network models.

What techniques improve generalization in machine learning?

Practitioners use several methods to reduce overfitting and improve a model's ability to generalize to new data. These approaches work by limiting model complexity, expanding data coverage, or combining predictions from multiple models.

Regularization

Regularization adds a penalty term to the model's loss function that discourages large parameter values, pushing the model toward simpler representations that tend to generalize better. L1 regularization (lasso) encourages sparse solutions by driving some parameters to zero. L2 regularization (ridge) penalizes large parameter values more smoothly without producing sparsity. Both approaches reduce overfitting without requiring changes to the model architecture.

Cross-validation

Cross-validation estimates how well a model will generalize before it is deployed. The training data is divided into multiple subsets, and the model is trained on some subsets and evaluated on the remainder. This process is repeated across different subset combinations and results are averaged. Cross-validation provides a more stable estimate of generalization performance than a single train/test split and is widely used for model selection and hyperparameter tuning.

Data augmentation

Data augmentation expands the effective size of a training dataset by creating modified versions of existing examples. In image classification, common augmentation techniques include rotation, flipping, cropping, and adding noise. Augmentation exposes the model to a wider range of input variations during training, improving robustness to the kinds of variation it will encounter in new data.

Feature engineering

Feature selection and engineering affects how well a model can learn from available data. Selecting informative features and removing redundant or noisy ones reduces the risk that the model fits on irrelevant patterns. Dimensionality reduction compresses the feature space and can improve generalization by focusing the model on the most important axes of variation in the data.

Ensemble methods

Ensemble methods combine predictions from multiple models to reduce variance and improve generalization. Bagging trains multiple models on random subsets of the training data and averages their predictions. Boosting trains models sequentially, with each model correcting the errors of the previous one. Random forests, a bagging-based approach, are among the most widely used ensemble methods. By aggregating diverse models, ensembles tend to generalize better than any single model trained on the same data.

Summary

Generalization measures how well a machine learning model applies patterns learned during training to new, unseen examples. Overfitting and underfitting represent the two failure modes: models that are too complex fit noise in the training data, while models that are too simple cannot capture meaningful patterns. Techniques including regularization, cross-validation, data augmentation, feature engineering, and ensemble methods give practitioners tools for improving generalization across different algorithm types and data contexts.