QLoRA
Fine-tuning a quantised model with LoRA adapters, so a large model fits on one GPU.
Aperçu
QLoRA combines two savings: the base model is loaded in 4-bit precision, and only small LoRA adapters are trained. It solves the memory problem that otherwise makes fine-tuning a large model require a cluster, since the frozen weights dominate memory usage and quantisation shrinks them fourfold. It exists because it brought fine-tuning of 65B-parameter models onto a single commodity GPU, which is what made customisation available outside large labs.
Graphe des technologies
Comment QLoRA se rattache au reste de la pile.
Utilise et dépend de
- Quantization technique Techniques QLoRA uses Quantization High ✓ source
Appliqué à
- PEFT library LLM Frameworks PEFT used for QLoRA High ✓ source
- Unsloth framework Frameworks Unsloth used for QLoRA High ✓ source
Evolution
- LoRA technique Techniques QLoRA extends LoRA High ✓ source