Gradient Descent
Updating weights in the direction that reduces the loss.
Überblick
Gradient descent adjusts each weight by a small step opposite the gradient, repeating until the loss stops falling. It solves the problem of finding a good configuration in a space far too large to search. It exists as the simplest member of a family of optimisers, and the practical work is in choosing the step size, the batch size and the schedule rather than in the update rule itself.
Technologiegraph
Wie Gradient Descent mit dem übrigen Stack zusammenhängt.
Nutzt und hängt ab von
- Learning Rate concept Algorithms Gradient Descent uses Learning Rate High ✓ source
Angewendet auf
- Fine-tuning technique Techniques Gradient Descent used for Fine-tuning High ✓ source
- Pretraining technique Techniques Gradient Descent used for Pretraining High ✓ source