Unsloth
Fine-tuning implementation with hand-written kernels for speed and low memory.
Resumen
Unsloth rewrites the compute-heavy parts of fine-tuning with custom kernels, reporting substantially lower memory use and faster training than the standard implementation. It solves the cost problem for people fine-tuning on a single consumer GPU, where memory is the binding constraint rather than time. It exists because the generic implementation does more work than the task requires, and hand-specialising it recovers a large margin.
Grafo de tecnologías
Cómo se conecta Unsloth con el resto del stack.
Aplicado a
- LoRA technique Techniques Unsloth used for LoRA High ✓ source
- QLoRA technique Techniques Unsloth used for QLoRA High ✓ source
- Quantization technique Techniques Unsloth used for Quantization High ✓ source