LLaVA
Vision-language model built by connecting a vision encoder to a language model.
Overview
LLaVA pairs a frozen vision encoder with a language model and trains only the projection between them, so the model can discuss images. It solves the problem that training a multimodal model from scratch needs a scale most groups do not have. It exists as the demonstration that the expensive part can be reused and only the connection learned.
Technology Graph
How LLaVA connects to the rest of the stack.
Uses and depends on
- Attention concept Algorithms LLaVA uses Attention High ✓ source
- CLIP model Multimodal LLaVA based on CLIP High ✓ source
- LLM concept Models LLaVA based on LLM High ✓ source
Applied to
- Multimodal AI application Multimodal LLaVA used for Multimodal AI High ✓ source
Tools
Software that implements or integrates with LLaVA.