Learning Path
Question & Answer
Choose the Best Answer
It ensures that all neurons start with the same weights, leading to uniform training.
It helps maintain the scale of the gradients throughout the layers, preventing them from becoming too small or too large.
It guarantees that the learning rate will always be optimal, which prevents gradient issues.
It automatically adjusts the architecture of the network to prevent gradient problems.
Understanding the Answer
Let's break down why this is correct
When weights are chosen with the right size, each layer passes gradients that stay about the same size. Other options are incorrect because People think giving every neuron the same weight will make training smooth, but it keeps all neurons identical; Weight initialization does not set the learning rate.
Key Concepts
Vanishing/Exploding Gradients Problem
medium level question
understand
Deep Dive: Vanishing/Exploding Gradients Problem
Master the fundamentals
Definition
The vanishing/exploding gradients problem poses a challenge in training deep neural networks, hindering convergence during optimization. Techniques such as normalized initialization and intermediate normalization layers have been developed to mitigate this issue and enable the training of deep networks with improved convergence rates.
Topic Definition
The vanishing/exploding gradients problem poses a challenge in training deep neural networks, hindering convergence during optimization. Techniques such as normalized initialization and intermediate normalization layers have been developed to mitigate this issue and enable the training of deep networks with improved convergence rates.
Ready to Master More Topics?
Join thousands of students using Seekh's interactive learning platform to excel in their studies with personalized practice and detailed explanations.