LLaVA
Vision-language model built by connecting a vision encoder to a language model.
Обзор
LLaVA pairs a frozen vision encoder with a language model and trains only the projection between them, so the model can discuss images. It solves the problem that training a multimodal model from scratch needs a scale most groups do not have. It exists as the demonstration that the expensive part can be reused and only the connection learned.
Граф технологий
Как LLaVA связана с остальным стеком.
Использует и зависит от
- Attention concept Algorithms LLaVA uses Attention High ✓ source
- CLIP model Multimodal LLaVA based on CLIP High ✓ source
- LLM concept Models LLaVA based on LLM High ✓ source
Применяется к
- Multimodal AI application Multimodal LLaVA used for Multimodal AI High ✓ source
Инструменты
ПО, которое реализует LLaVA или интегрируется с ним.