Denoising Score Matching

Denoising score matching is a learning objective that asks a model to predict how the probability of data changes when a small amount of Gaussian noise is added. In practice the model receives a noisy version of an example and learns to output the direction in which the log‑probability density rises most steeply – the so‑called score. By training on many such corrupted pairs, the model acquires a representation of the underlying data distribution without ever seeing clean examples directly.

This approach matters because the score contains the essential geometry of a probability density: if you know how to move uphill in log‑density from any point, you can reconstruct or sample from the distribution by following those directions. Denoising score matching thus provides a tractable way to train models that can later be used for tasks like generating new data, estimating likelihoods, or performing inference under noise.

You will encounter denoising score matching most often in modern generative frameworks that build up samples through a gradual removal of noise, such as diffusion models and certain types of unsupervised density estimators. It also appears whenever researchers need to train energy‑based models without computing expensive normalizing constants, using the noisy data trick to sidestep those calculations.