Supervised Fine-Tuning
Fine-tuning a pretrained model on demonstrations of the target behaviour.
Aperçu
Supervised fine-tuning trains a pretrained model on input and output pairs that demonstrate the behaviour wanted, before any preference optimisation. It solves the problem that a model trained to continue text has no notion of answering, so the first thing any assistant pipeline needs is examples of answers. It exists as the stage that both produces a usable model on its own and supplies the outputs a preference model is then trained on.
Graphe des technologies
Comment Supervised Fine-Tuning se rattache au reste de la pile.
Utilise et dépend de
- Fine-tuning technique Techniques Supervised Fine-Tuning based on Fine-tuning High ✓ source
Classification
- RLHF technique Techniques Supervised Fine-Tuning part of RLHF High ✓ source