Dropout
Randomly zeroing activations during training to reduce overfitting.
Overview
Dropout randomly disables a fraction of activations on each training step, so no unit can rely on another being present. It solves the problem that a network large enough to fit the training data will memorise it rather than learn from it. It exists as one of the cheapest regularisers available, and it is switched off at inference where the full network is used.
Technology Graph
How Dropout connects to the rest of the stack.
Applied to
- Deep Learning concept Algorithms Dropout used for Deep Learning High ✓ source
- Fine-tuning technique Techniques Dropout used for Fine-tuning High ✓ source
- Pretraining technique Techniques Dropout used for Pretraining High ✓ source
- Overfitting concept Algorithms Dropout solves Overfitting High ✓ source
Tools
Software that implements or integrates with Dropout.