Learning Path
Question & Answer
Choose the Best Answer
Using normalized weight initialization
Increasing the learning rate
Reducing the number of layers
Applying dropout after every layer
Understanding the Answer
Let's break down why this is correct
When a neural network starts training, the small numbers from each layer can shrink or grow too much. Other options are incorrect because A higher learning rate can make the updates too large, which often makes the gradients explode instead of stabilizing; Removing layers does not solve the root of the problem; the network may still have many weights that can cause vanishing or exploding gradients.
Key Concepts
Vanishing/Exploding Gradients Problem
hard level question
understand
Deep Dive: Vanishing/Exploding Gradients Problem
Master the fundamentals
Definition
The vanishing/exploding gradients problem poses a challenge in training deep neural networks, hindering convergence during optimization. Techniques such as normalized initialization and intermediate normalization layers have been developed to mitigate this issue and enable the training of deep networks with improved convergence rates.
Topic Definition
The vanishing/exploding gradients problem poses a challenge in training deep neural networks, hindering convergence during optimization. Techniques such as normalized initialization and intermediate normalization layers have been developed to mitigate this issue and enable the training of deep networks with improved convergence rates.
Ready to Master More Topics?
Join thousands of students using Seekh's interactive learning platform to excel in their studies with personalized practice and detailed explanations.