LLaVA
Vision-language model built by connecting a vision encoder to a language model.
Aperçu
LLaVA pairs a frozen vision encoder with a language model and trains only the projection between them, so the model can discuss images. It solves the problem that training a multimodal model from scratch needs a scale most groups do not have. It exists as the demonstration that the expensive part can be reused and only the connection learned.
Graphe des technologies
Comment LLaVA se rattache au reste de la pile.
Utilise et dépend de
- Attention concept Algorithms LLaVA uses Attention High ✓ source
- CLIP model Multimodal LLaVA based on CLIP High ✓ source
- LLM concept Models LLaVA based on LLM High ✓ source
Appliqué à
- Multimodal AI application Multimodal LLaVA used for Multimodal AI High ✓ source
Outils
Logiciels qui implémentent LLaVA ou s'y intègrent.