Supervised Fine-Tuning
Fine-tuning a pretrained model on demonstrations of the target behaviour.
Overview
Supervised fine-tuning trains a pretrained model on input and output pairs that demonstrate the behaviour wanted, before any preference optimisation. It solves the problem that a model trained to continue text has no notion of answering, so the first thing any assistant pipeline needs is examples of answers. It exists as the stage that both produces a usable model on its own and supplies the outputs a preference model is then trained on.
Technology Graph
How Supervised Fine-Tuning connects to the rest of the stack.
Uses and depends on
- Fine-tuning technique Techniques Supervised Fine-Tuning based on Fine-tuning High ✓ source
Classification
- RLHF technique Techniques Supervised Fine-Tuning part of RLHF High ✓ source