CLIP
Model that places images and text in one embedding space.
Overview
CLIP trains an image encoder and a text encoder so that matching pairs land near each other, giving a shared space where a text query can retrieve images. It solves the problem that supervised image classifiers are limited to their fixed label set, so recognising anything else needs a new dataset. It exists because zero-shot classification from a description turned out to work, and it is the component most multimodal systems are built on.
Technology Graph
How CLIP connects to the rest of the stack.
Uses and depends on
- LLaVA model Multimodal LLaVA based on CLIP High ✓ source
- Transformer architecture Algorithms CLIP based on Transformer High ✓ source
Applied to
- AI Search application Applications CLIP used for AI Search High ✓ source
- Contrastive Learning technique Algorithms Contrastive Learning used for CLIP High ✓ source
- LAION-5B dataset Datasets LAION-5B used for CLIP High ✓ source
- Multimodal AI application Multimodal CLIP used for Multimodal AI High ✓ source
- Zero-Shot concept Prompting Zero-Shot used for CLIP High ✓ source