Which problem occurs when the learning signal becomes too large, causing unstable updates?

Which problem occurs when the learning signal becomes too large, causing unstable updates?

• A. Underfitted Models
• B. Ridge Regression (L2 penalty)
• C. LASSO (L1 penalty)
• D. Exploding Gradients

Answer: (D)

When the learning signal becomes too large, the gradients used to update the parameters become huge, causing updates to overshoot the optimum. This leads to unstable training where the loss can spike, weights grow uncontrollably, and numerical values can blow up or NaNs may appear. This phenomenon is known as exploding gradients and it’s common in deep networks or recurrent networks where gradients are backpropagated through many steps. It’s different from underfitting, which happens when the model can’t capture the data patterns because the signal is too weak or capacity too low. Regularization methods like Ridge (L2) or LASSO (L1) constrain or shrink weights to improve generalization and stability, but they don’t describe the instability caused by overly large gradient updates.

When the learning signal becomes too large, the gradients used to update the parameters become huge, causing updates to overshoot the optimum. This leads to unstable training where the loss can spike, weights grow uncontrollably, and numerical values can blow up or NaNs may appear. This phenomenon is known as exploding gradients and it’s common in deep networks or recurrent networks where gradients are backpropagated through many steps. It’s different from underfitting, which happens when the model can’t capture the data patterns because the signal is too weak or capacity too low. Regularization methods like Ridge (L2) or LASSO (L1) constrain or shrink weights to improve generalization and stability, but they don’t describe the instability caused by overly large gradient updates.