CLIP
Model that places images and text in one embedding space.
Resumen
CLIP trains an image encoder and a text encoder so that matching pairs land near each other, giving a shared space where a text query can retrieve images. It solves the problem that supervised image classifiers are limited to their fixed label set, so recognising anything else needs a new dataset. It exists because zero-shot classification from a description turned out to work, and it is the component most multimodal systems are built on.
Grafo de tecnologías
Cómo se conecta CLIP con el resto del stack.
Usa y depende de
- LLaVA model Multimodal LLaVA based on CLIP High ✓ source
- Transformer architecture Algorithms CLIP based on Transformer High ✓ source
Aplicado a
- AI Search application Applications CLIP used for AI Search High ✓ source
- Contrastive Learning technique Algorithms Contrastive Learning used for CLIP High ✓ source
- LAION-5B dataset Datasets LAION-5B used for CLIP High ✓ source
- Multimodal AI application Multimodal CLIP used for Multimodal AI High ✓ source
- Zero-Shot concept Prompting Zero-Shot used for CLIP High ✓ source