LLaVA
Vision-language model built by connecting a vision encoder to a language model.
Visão geral
LLaVA pairs a frozen vision encoder with a language model and trains only the projection between them, so the model can discuss images. It solves the problem that training a multimodal model from scratch needs a scale most groups do not have. It exists as the demonstration that the expensive part can be reused and only the connection learned.
Grafo de tecnologias
Como LLaVA se liga ao resto do stack.
Usa e depende de
- Attention concept Algorithms LLaVA uses Attention High ✓ source
- CLIP model Multimodal LLaVA based on CLIP High ✓ source
- LLM concept Models LLaVA based on LLM High ✓ source
Aplicado a
- Multimodal AI application Multimodal LLaVA used for Multimodal AI High ✓ source
Ferramentas
Software que implementa ou se integra com LLaVA.