Backpropagation

Backpropagation is the method used to calculate the gradient of a neural network's loss with respect to every weight. It applies the chain rule of calculus, moving backward from the output layer to the input layer and reusing intermediate results so the whole calculation costs roughly the same as one forward pass.

Why it matters

Without backpropagation, training deep networks would be impractically slow, because you would have to estimate the effect of each parameter separately. It supplies the gradients, and an optimiser such as gradient descent then uses them to update the weights. Deep learning libraries compute these automatically, so engineers rarely write it by hand, but understanding it explains problems like vanishing and exploding gradients.

Example

Training step:
1. Forward pass: input -> layers -> prediction
2. Loss: compare prediction with target
3. Backward pass: compute d(loss)/d(weight) for every layer
4. Update: weight = weight - learning_rate * gradient

Common mistakes

• Using "backpropagation" and "gradient descent" as synonyms. One computes gradients, the other uses them.
• Ignoring vanishing gradients in very deep networks, which leave early layers barely trained.
• Thinking the brain learns this way. It is a mathematical technique, not a biological claim.

Related

• Gradient descent
• Loss function
• Neural network
• Deep learning

यह पेज सामान्य जानकारी है, जिसे हमारी संपादकीय टीम ने लिखा है और दस्तावेज़ीकरण बदलने पर अपडेट किया जाता है। सर्वर और प्रदाता अलग-अलग होते हैं, इसलिए बदलाव पहले सुरक्षित तरीके से जाँचें। देखें: संपादकीय नीति और अस्वीकरण.