LLM-as-a-Judge
Using a capable model to score another model's output.
Visão geral
LLM-as-a-judge asks a strong model to rate or compare outputs, usually with a rubric and a reference. It solves the problem that open-ended generation has no correct answer to compare against, so the most important capability was the least measurable. It exists because the correlation with human preference turned out to be usable, and its known weaknesses - position bias, verbosity bias, self-preference - are why the rubric matters.
Grafo de tecnologias
Como LLM-as-a-Judge se liga ao resto do stack.
Usado por
- MT-Bench benchmark Benchmarks LLM-as-a-Judge used by MT-Bench High ✓ source
- RLAIF technique Safety & Alignment LLM-as-a-Judge used by RLAIF High ✓ source