RNN Vanishing Gradient Problem Visualization
Watch how gradients exponentially decay as they backpropagate through time
Sequence Length: 20
Activation Function
tanh (typical)
sigmoid (worse)
ReLU (better)
Weight Scale: 0.50
Gradient Magnitude Over Time
Initial Gradient
1.0000
Final Gradient
0
Decay Ratio
0