How does the introduction of a penalty term in regularization affect model complexity and the process of hyperparameter tuning in predictive models?

Adding a penalty term, such as L1 or L2, forces the optimization to keep coefficient values small, which discourages the model from fitting every minor fluctuation in the training data. This shrinkage reduces the model’s effective degrees of freedom, making it simpler and less likely to overfit, but it can also suppress useful signals. Because the penalty strength is a new hyperparameter, we must search over its values—often called λ or alpha—using cross‑validation or a validation set to balance bias and variance. For example, in ridge regression on a housing‑price dataset, setting λ to 0. 1 keeps most coefficients, while λ=10 drives many of them toward zero, changing the predictions noticeably.