QLoRA
Fine-tuning a quantised model with LoRA adapters, so a large model fits on one GPU.
Обзор
QLoRA combines two savings: the base model is loaded in 4-bit precision, and only small LoRA adapters are trained. It solves the memory problem that otherwise makes fine-tuning a large model require a cluster, since the frozen weights dominate memory usage and quantisation shrinks them fourfold. It exists because it brought fine-tuning of 65B-parameter models onto a single commodity GPU, which is what made customisation available outside large labs.
Граф технологий
Как QLoRA связана с остальным стеком.
Использует и зависит от
- Quantization technique Techniques QLoRA uses Quantization High ✓ source
Применяется к
- PEFT library LLM Frameworks PEFT used for QLoRA High ✓ source
- Unsloth framework Frameworks Unsloth used for QLoRA High ✓ source
Evolution
- LoRA technique Techniques QLoRA extends LoRA High ✓ source