Learning Path
Question & Answer
Choose the Best Answer
By ensuring all weights are initialized to zero, which simplifies calculations.
By employing random initialization with uniform distribution, and restricting gradients during backpropagation.
By only using shallow neural networks where gradients naturally stabilize.
By applying activation functions like ReLU without any modifications.
Understanding the Answer
Let's break down why this is correct
Randomly initializing weights with a uniform distribution keeps the size of signals the same as they move through layers. Other options are incorrect because Setting all weights to zero makes every neuron produce the same output; Using a shallow network does not guarantee that gradients will stay balanced.
Key Concepts
Vanishing/Exploding Gradients Problem
hard level question
understand
Deep Dive: Vanishing/Exploding Gradients Problem
Master the fundamentals
Definition
The vanishing/exploding gradients problem poses a challenge in training deep neural networks, hindering convergence during optimization. Techniques such as normalized initialization and intermediate normalization layers have been developed to mitigate this issue and enable the training of deep networks with improved convergence rates.
Topic Definition
The vanishing/exploding gradients problem poses a challenge in training deep neural networks, hindering convergence during optimization. Techniques such as normalized initialization and intermediate normalization layers have been developed to mitigate this issue and enable the training of deep networks with improved convergence rates.
Ready to Master More Topics?
Join thousands of students using Seekh's interactive learning platform to excel in their studies with personalized practice and detailed explanations.