In the context of training deep neural networks, the __________ problem refers to the difficulty in achieving convergence due to excessively small or large gradients during the optimization process.

Question

Seekh · Accepted Answer

The blank is filled by “vanishing/exploding gradients problem. ” This issue happens when back‑propagation pushes gradients through many layers, causing them to shrink toward zero or grow without bound. When gradients are too small, learning stalls because weight updates become negligible, and when they are too large, the updates overshoot and the network becomes unstable. For instance, training a 20‑layer network with a sigmoid activation can produce gradients of 10⁻¹⁰, making the weights barely change. The result is that the optimizer fails to converge to a good solution.

In the context of training deep neural networks, the __________ problem refers to the difficulty in achieving convergence due to excessively small or large gradients during the optimization process.

Learning Path

Choose the Best Answer

Understanding the Answer

Key Concepts

Deep Dive: Vanishing/Exploding Gradients Problem

Definition

Topic Definition

Ready to Master More Topics?