Supervised Fine-Tuning
Fine-tuning a pretrained model on demonstrations of the target behaviour.
Обзор
Supervised fine-tuning trains a pretrained model on input and output pairs that demonstrate the behaviour wanted, before any preference optimisation. It solves the problem that a model trained to continue text has no notion of answering, so the first thing any assistant pipeline needs is examples of answers. It exists as the stage that both produces a usable model on its own and supplies the outputs a preference model is then trained on.
Граф технологий
Как Supervised Fine-Tuning связана с остальным стеком.
Использует и зависит от
- Fine-tuning technique Techniques Supervised Fine-Tuning based on Fine-tuning High ✓ source
Classification
- RLHF technique Techniques Supervised Fine-Tuning part of RLHF High ✓ source