Contrastive Learning
Training by pulling matching pairs together and pushing others apart.
Overview
Contrastive learning trains an encoder so that representations of matching pairs are close and representations of non-matching pairs are far apart. It solves the problem that labelled data is scarce, because a match can be defined by augmentation, co-occurrence or a paired modality rather than by a human. It exists as the objective behind most embedding models, including the image-text pairs CLIP is trained on.
Technology Graph
How Contrastive Learning connects to the rest of the stack.
Applied to
- CLIP model Multimodal Contrastive Learning used for CLIP High ✓ source
- Embedding concept Techniques Contrastive Learning used for Embedding High ✓ source
- LAION-5B dataset Datasets LAION-5B used for Contrastive Learning High ✓ source
- Sentence Transformers framework Frameworks Contrastive Learning used for Sentence Transformers High ✓ source