Gradient Descent

Gradient descent is the core method for training most machine learning models. It computes the gradient of the loss function, the direction in which the loss increases fastest, and moves the parameters a small step in the opposite direction, repeating until the loss stops improving.

Why it matters

Neural networks have millions or billions of parameters, and gradient descent is how they are tuned. The step size is the learning rate: too large and training overshoots or diverges, too small and it crawls or stalls. Variants such as stochastic gradient descent, momentum and Adam compute gradients on mini-batches and adapt the step, making training practical on large datasets.

Example

loss = (w - 3)^2        gradient = 2(w - 3)
w = 0, learning_rate = 0.25

step 1: w = 0 - 0.25 * (-6)   = 1.5
step 2: w = 1.5 - 0.25 * (-3) = 2.25
step 3: w = 2.25 - 0.25*(-1.5) = 2.625   -> approaching 3

Common mistakes

• Using one learning rate without checking the loss curve.
• Assuming it finds the global minimum. On complex models it may settle in a local minimum or a flat region.
• Forgetting to scale inputs, which slows convergence.

Related

• Loss function
• Backpropagation
• Epoch
• Neural network

यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.