Backpropagation
Computing gradients through a network by applying the chain rule backwards.
Aperçu
Backpropagation computes the derivative of a loss with respect to every weight by applying the chain rule from the output back to the input, reusing intermediate values. It solves the problem that differentiating a deep composition naively costs exponentially many evaluations. It exists as the algorithm that made training multi-layer networks tractable, and every framework is essentially an efficient implementation of it.
Graphe des technologies
Comment Backpropagation se rattache au reste de la pile.
Utilise et dépend de
- Vanishing Gradient concept Algorithms Vanishing Gradient based on Backpropagation High ✓ source
Appliqué à
- Neural Network concept Algorithms Backpropagation used for Neural Network High ✓ source
- Pretraining technique Techniques Backpropagation used for Pretraining High ✓ source