LLM-as-a-Judge
Using a capable model to score another model's output.
Aperçu
LLM-as-a-judge asks a strong model to rate or compare outputs, usually with a rubric and a reference. It solves the problem that open-ended generation has no correct answer to compare against, so the most important capability was the least measurable. It exists because the correlation with human preference turned out to be usable, and its known weaknesses - position bias, verbosity bias, self-preference - are why the rubric matters.
Graphe des technologies
Comment LLM-as-a-Judge se rattache au reste de la pile.
Utilisé par
- MT-Bench benchmark Benchmarks LLM-as-a-Judge used by MT-Bench High ✓ source
- RLAIF technique Safety & Alignment LLM-as-a-Judge used by RLAIF High ✓ source