TRL
Training library for fine-tuning and aligning models on preference data.
Aperçu
TRL provides the training loops for supervised fine-tuning, reward modelling and the preference optimisation methods that follow. It solves the problem that aligning a model is a pipeline of stages with different data shapes, and getting the stages to compose is most of the work. It exists because the methods are published faster than they are implemented, so a library that tracks them shortens the distance from paper to a trained model.
Graphe des technologies
Comment TRL se rattache au reste de la pile.
Appliqué à
- DPO technique Techniques TRL used for DPO High ✓ source
- Fine-tuning technique Techniques TRL used for Fine-tuning High ✓ source
- RLHF technique Techniques TRL used for RLHF High ✓ source