Which method updates weights after seeing each individual, randomly selected data point?

Which method updates weights after seeing each individual, randomly selected data point?

• A. Batch Gradient Descent
• B. Dynamic Learning Rate
• C. Mini-Batch Gradient Descent
• D. Stochastic Gradient Descent

Answer: (D)

Updating weights after each individual data point relies on estimating the gradient from a single example. This makes learning fast and scalable to large datasets, since you don’t wait for a full pass over the data before updating. However, because one sample provides a noisy view of the true gradient, the updates are noisier and the path to convergence can be bumpy. This approach is stochastic gradient descent. In contrast, batch gradient descent uses the entire dataset to compute the gradient before any update, which is more stable but can be very slow on large datasets. Mini-batch gradient descent uses small groups of examples, offering a middle ground between stability and speed. Dynamic learning rate, meanwhile, changes the step size over time but does not specify how often you update.

Updating weights after each individual data point relies on estimating the gradient from a single example. This makes learning fast and scalable to large datasets, since you don’t wait for a full pass over the data before updating. However, because one sample provides a noisy view of the true gradient, the updates are noisier and the path to convergence can be bumpy. This approach is stochastic gradient descent. In contrast, batch gradient descent uses the entire dataset to compute the gradient before any update, which is more stable but can be very slow on large datasets. Mini-batch gradient descent uses small groups of examples, offering a middle ground between stability and speed. Dynamic learning rate, meanwhile, changes the step size over time but does not specify how often you update.