Flat Minimum

A flat minimum is a zone in the error surface of a machine‑learning model where moving the parameters slightly does not cause the loss to rise much. In other words, instead of the loss forming a sharp, narrow dip, it settles into a broad, gently sloping basin. The key point is that the model’s performance remains stable across a range of nearby settings rather than being highly sensitive to an exact parameter choice.

Why this matters is that such stability tends to protect the model from over‑fitting the quirks of its training data. When the loss landscape around a solution is flat, small fluctuations—whether caused by noise in the data, slight changes in the training process, or modest alterations when deploying the model—are less likely to degrade performance. Consequently, models that land in flat minima often generalize better to new, unseen examples.

Flat minima show up whenever people train neural networks or other highly parameterised systems and examine the shape of their loss functions. Researchers who study learning dynamics, practitioners who tune regularisation techniques, and anyone interested in why some trained models succeed outside the training set all encounter this concept. It provides a mental picture for understanding how optimisation choices influence a model’s robustness beyond the data it was directly taught on.