Brier Score

The Brier score is a way of judging how well a forecast that gives probabilities matches what actually happens. For each prediction it looks at the difference between the probability that was assigned to an outcome and whether that outcome truly occurred – counting the outcome as one if it happened and zero if it did not – squares that difference, and then averages these squared errors over all of the predictions. Because the squaring gives larger weight to bigger mistakes, a lower Brier score signals a forecast that is both well‑calibrated (the probabilities reflect true frequencies) and sharp (it does not hedge too much).

People care about the Brier score because many decisions rely on probabilistic information rather than simple yes‑or‑no predictions. In weather forecasting, for instance, it can tell whether a 70 percent chance of rain is truly reflective of how often it rains when such a forecast is made. Medical risk models use it to compare different ways of estimating the chance of disease or complications, and machine‑learning practitioners employ it when they need to evaluate classifiers that output probability estimates instead of just class labels. In each of these settings, the Brier score provides a single interpretable number that summarizes both calibration and discrimination, making it easier to compare models and improve decision‑making.