How does empirical risk minimization (ERM) ensure that a predictive model generalizes well to unseen data?

Empirical risk minimization (ERM) works by picking the model that gives the smallest average error on the data we actually have, which is a stand‑in for the overall error we care about. Because the training data are drawn from the same process that produces future data, a model that fits the training set well is likely to fit new data well, provided the training set is large and varied. ERM also relies on limiting how complicated the model can be so that it doesn’t just memorize the training points. For example, if we fit a straight line to ten noisy measurements of a line, ERM will choose the slope that makes the line’s predictions closest to those ten points, and that line will usually predict nearby points accurately too. Thus, by minimizing error on a representative sample and keeping the model simple, ERM helps the model perform well on unseen data.