Supervised Fine-Tuning
Fine-tuning a pretrained model on demonstrations of the target behaviour.
Überblick
Supervised fine-tuning trains a pretrained model on input and output pairs that demonstrate the behaviour wanted, before any preference optimisation. It solves the problem that a model trained to continue text has no notion of answering, so the first thing any assistant pipeline needs is examples of answers. It exists as the stage that both produces a usable model on its own and supplies the outputs a preference model is then trained on.
Technologiegraph
Wie Supervised Fine-Tuning mit dem übrigen Stack zusammenhängt.
Nutzt und hängt ab von
- Fine-tuning technique Techniques Supervised Fine-Tuning based on Fine-tuning High ✓ source
Classification
- RLHF technique Techniques Supervised Fine-Tuning part of RLHF High ✓ source