TRL
Training library for fine-tuning and aligning models on preference data.
Обзор
TRL provides the training loops for supervised fine-tuning, reward modelling and the preference optimisation methods that follow. It solves the problem that aligning a model is a pipeline of stages with different data shapes, and getting the stages to compose is most of the work. It exists because the methods are published faster than they are implemented, so a library that tracks them shortens the distance from paper to a trained model.
Граф технологий
Как TRL связана с остальным стеком.
Применяется к
- DPO technique Techniques TRL used for DPO High ✓ source
- Fine-tuning technique Techniques TRL used for Fine-tuning High ✓ source
- RLHF technique Techniques TRL used for RLHF High ✓ source