A data scientist is tasked with building a deep neural network to classify images of animals. They notice that as they increase the depth of the network, the accuracy of their model begins to degrade significantly. What is the most likely reason for this degradation, and what approach could they take to mitigate it?

Question

Seekh · Accepted Answer

The model’s accuracy drops because deeper networks suffer from the degradation problem, where adding more layers makes the optimization harder and gradients vanish or explode, so the network can’t learn useful representations. When gradients shrink, the deeper layers learn almost nothing, so the deeper network actually performs worse than a shallower one. A common fix is to use residual (skip) connections, as in ResNet, which let the gradient flow directly to earlier layers and preserve the identity mapping. By adding these shortcut paths, the network can train deeper without losing accuracy, as shown when a 50‑layer ResNet outperforms a plain 50‑layer CNN on ImageNet. Thus, replacing plain layers with residual blocks or using batch normalization and proper initialization can mitigate the degradation.

Learning Path

Choose the Best Answer

Understanding the Answer

Key Concepts

Deep Dive: Degradation Problem in Deep Networks

Definition

Topic Definition

Ready to Master More Topics?