TRL
Training library for fine-tuning and aligning models on preference data.
Visão geral
TRL provides the training loops for supervised fine-tuning, reward modelling and the preference optimisation methods that follow. It solves the problem that aligning a model is a pipeline of stages with different data shapes, and getting the stages to compose is most of the work. It exists because the methods are published faster than they are implemented, so a library that tracks them shortens the distance from paper to a trained model.
Grafo de tecnologias
Como TRL se liga ao resto do stack.
Aplicado a
- DPO technique Techniques TRL used for DPO High ✓ source
- Fine-tuning technique Techniques TRL used for Fine-tuning High ✓ source
- RLHF technique Techniques TRL used for RLHF High ✓ source