LLaVA
Vision-language model built by connecting a vision encoder to a language model.
Resumen
LLaVA pairs a frozen vision encoder with a language model and trains only the projection between them, so the model can discuss images. It solves the problem that training a multimodal model from scratch needs a scale most groups do not have. It exists as the demonstration that the expensive part can be reused and only the connection learned.
Grafo de tecnologías
Cómo se conecta LLaVA con el resto del stack.
Usa y depende de
- Attention concept Algorithms LLaVA uses Attention High ✓ source
- CLIP model Multimodal LLaVA based on CLIP High ✓ source
- LLM concept Models LLaVA based on LLM High ✓ source
Aplicado a
- Multimodal AI application Multimodal LLaVA used for Multimodal AI High ✓ source
Herramientas
Software que implementa o se integra con LLaVA.