How do residual connections in deep networks enhance training efficiency and mitigate the degradation problem?

Residual connections add the input of a layer to its output, creating a shortcut that lets the gradient flow directly backward and forward through the network. Because the shortcut bypasses many nonlinear transformations, the network can learn identity functions easily, preventing deeper layers from degrading performance. This direct path keeps the signal strength stable during back‑propagation, so the optimizer updates weights more reliably and training converges faster. As a result, very deep models no longer suffer from the “degradation problem” where adding layers makes accuracy worse. For example, a 50‑layer residual network can reach higher accuracy than a plain 50‑layer network, because each block can focus on learning residuals rather than relearning identity.