How does the sensitivity of a predictor impact its generalization ability in machine learning?

A predictor’s sensitivity measures how much its output changes when the input changes slightly; high sensitivity means the model reacts strongly to small variations in the data. When a predictor is overly sensitive, it tends to fit noise in the training set, so it performs well on that data but poorly on new data, hurting generalization. Conversely, a predictor with moderate sensitivity captures the underlying pattern without chasing random fluctuations, leading to better performance on unseen examples. For instance, a linear regression that over‑fits a few noisy points will predict wildly for new inputs, whereas a smoother model that ignores those outliers will give more stable predictions. Thus, controlling sensitivity—often through regularization—helps a model generalize by balancing fit and stability.