Supervised Fine-Tuning
Fine-tuning a pretrained model on demonstrations of the target behaviour.
Resumen
Supervised fine-tuning trains a pretrained model on input and output pairs that demonstrate the behaviour wanted, before any preference optimisation. It solves the problem that a model trained to continue text has no notion of answering, so the first thing any assistant pipeline needs is examples of answers. It exists as the stage that both produces a usable model on its own and supplies the outputs a preference model is then trained on.
Grafo de tecnologías
Cómo se conecta Supervised Fine-Tuning con el resto del stack.
Usa y depende de
- Fine-tuning technique Techniques Supervised Fine-Tuning based on Fine-tuning High ✓ source
Classification
- RLHF technique Techniques Supervised Fine-Tuning part of RLHF High ✓ source