TRL
Training library for fine-tuning and aligning models on preference data.
Resumen
TRL provides the training loops for supervised fine-tuning, reward modelling and the preference optimisation methods that follow. It solves the problem that aligning a model is a pipeline of stages with different data shapes, and getting the stages to compose is most of the work. It exists because the methods are published faster than they are implemented, so a library that tracks them shortens the distance from paper to a trained model.
Grafo de tecnologías
Cómo se conecta TRL con el resto del stack.
Aplicado a
- DPO technique Techniques TRL used for DPO High ✓ source
- Fine-tuning technique Techniques TRL used for Fine-tuning High ✓ source
- RLHF technique Techniques TRL used for RLHF High ✓ source