LLM-as-a-Judge
Using a capable model to score another model's output.
Resumen
LLM-as-a-judge asks a strong model to rate or compare outputs, usually with a rubric and a reference. It solves the problem that open-ended generation has no correct answer to compare against, so the most important capability was the least measurable. It exists because the correlation with human preference turned out to be usable, and its known weaknesses - position bias, verbosity bias, self-preference - are why the rubric matters.
Grafo de tecnologías
Cómo se conecta LLM-as-a-Judge con el resto del stack.
Usado por
- MT-Bench benchmark Benchmarks LLM-as-a-Judge used by MT-Bench High ✓ source
- RLAIF technique Safety & Alignment LLM-as-a-Judge used by RLAIF High ✓ source