# mx.nd.sgd.update¶

## Description¶

Update function for Stochastic Gradient Descent (SGD) optimizer.

weight = weight - learning_rate * (gradient + wd * weight)

However, if gradient is of row_sparse storage type and lazy_update is True,


only the row slices whose indices appear in grad.indices are updated:

for row in gradient.indices:
weight[row] = weight[row] - learning_rate * (gradient[row] + wd * weight[row])


## Arguments¶

Argument

Description

weight

NDArray-or-Symbol.

Weight

grad

NDArray-or-Symbol.

lr

float, required.

Learning rate

wd

float, optional, default=0.

Weight decay augments the objective function with a regularization term that penalizes large weights. The penalty scales with the square of the magnitude of each weight.

rescale.grad

float, optional, default=1.

clip.gradient

float, optional, default=-1.

lazy.update
out The result mx.ndarray