Backpropagation
Computing gradients through a network by applying the chain rule backwards.
Overview
Backpropagation computes the derivative of a loss with respect to every weight by applying the chain rule from the output back to the input, reusing intermediate values. It solves the problem that differentiating a deep composition naively costs exponentially many evaluations. It exists as the algorithm that made training multi-layer networks tractable, and every framework is essentially an efficient implementation of it.
Technology Graph
How Backpropagation connects to the rest of the stack.
Uses and depends on
- Vanishing Gradient concept Algorithms Vanishing Gradient based on Backpropagation High ✓ source
Applied to
- Neural Network concept Algorithms Backpropagation used for Neural Network High ✓ source
- Pretraining technique Techniques Backpropagation used for Pretraining High ✓ source