CLIP
Model that places images and text in one embedding space.
Aperçu
CLIP trains an image encoder and a text encoder so that matching pairs land near each other, giving a shared space where a text query can retrieve images. It solves the problem that supervised image classifiers are limited to their fixed label set, so recognising anything else needs a new dataset. It exists because zero-shot classification from a description turned out to work, and it is the component most multimodal systems are built on.
Graphe des technologies
Comment CLIP se rattache au reste de la pile.
Utilise et dépend de
- LLaVA model Multimodal LLaVA based on CLIP High ✓ source
- Transformer architecture Algorithms CLIP based on Transformer High ✓ source
Appliqué à
- AI Search application Applications CLIP used for AI Search High ✓ source
- Contrastive Learning technique Algorithms Contrastive Learning used for CLIP High ✓ source
- LAION-5B dataset Datasets LAION-5B used for CLIP High ✓ source
- Multimodal AI application Multimodal CLIP used for Multimodal AI High ✓ source
- Zero-Shot concept Prompting Zero-Shot used for CLIP High ✓ source