Fine-tuning
Continue training a pre-trained model on task-specific data to adapt its behaviour.
Overview
Fine-tuning continues training an already pre-trained model on a smaller, task-specific dataset so that its weights shift toward the behaviour you want. It solves the problem that a general model may not follow a house style, a domain vocabulary or a fixed output format. It exists because pre-training is expensive but adaptation is not: starting from a pre-trained checkpoint means a modest dataset is enough. It is the main alternative to retrieval when the goal is behaviour rather than fresh facts.
Technology Graph
How Fine-tuning connects to the rest of the stack.
Uses and depends on
- Instruction Tuning technique Techniques Instruction Tuning based on Fine-tuning High ✓ source
- Supervised Fine-Tuning technique Techniques Supervised Fine-Tuning based on Fine-tuning High ✓ source
Applied to
- Axolotl framework Frameworks Axolotl used for Fine-tuning High ✓ source
- DeepSpeed framework Frameworks DeepSpeed used for Fine-tuning High ✓ source
- Dropout technique Techniques Dropout used for Fine-tuning High ✓ source
- Gradient Descent technique Algorithms Gradient Descent used for Fine-tuning High ✓ source
- Learning Rate concept Algorithms Learning Rate used for Fine-tuning High ✓ source
- LoRA technique Techniques LoRA used for Fine-tuning High ✓ source
- MLflow tool MLOps MLflow used for Fine-tuning High ✓ source
- PEFT library LLM Frameworks PEFT used for Fine-tuning High ✓ source
- RLHF technique Techniques RLHF used for Fine-tuning High ✓ source
- Regularization concept Algorithms Regularization used for Fine-tuning High ✓ source
- Reinforcement Learning concept Algorithms Reinforcement Learning used for Fine-tuning High ✓ source
- Synthetic Data technique Techniques Synthetic Data used for Fine-tuning High ✓ source
- TRL library LLM Frameworks TRL used for Fine-tuning High ✓ source
- Weights & Biases tool MLOps Weights & Biases used for Fine-tuning High ✓ source
Classification
- Overfitting concept Algorithms Overfitting part of Fine-tuning High ✓ source
Alternatives
Comparisons
Side-by-side breakdowns with the alternatives above.