QLoRA
Fine-tuning a quantised model with LoRA adapters, so a large model fits on one GPU.
Overview
QLoRA combines two savings: the base model is loaded in 4-bit precision, and only small LoRA adapters are trained. It solves the memory problem that otherwise makes fine-tuning a large model require a cluster, since the frozen weights dominate memory usage and quantisation shrinks them fourfold. It exists because it brought fine-tuning of 65B-parameter models onto a single commodity GPU, which is what made customisation available outside large labs.
Technology Graph
How QLoRA connects to the rest of the stack.
Uses and depends on
- Quantization technique Techniques QLoRA uses Quantization High ✓ source
Applied to
- PEFT library LLM Frameworks PEFT used for QLoRA High ✓ source
- Unsloth framework Frameworks Unsloth used for QLoRA High ✓ source
Evolution
- LoRA technique Techniques QLoRA extends LoRA High ✓ source