LLM-as-a-Judge
Using a capable model to score another model's output.
Обзор
LLM-as-a-judge asks a strong model to rate or compare outputs, usually with a rubric and a reference. It solves the problem that open-ended generation has no correct answer to compare against, so the most important capability was the least measurable. It exists because the correlation with human preference turned out to be usable, and its known weaknesses - position bias, verbosity bias, self-preference - are why the rubric matters.
Граф технологий
Как LLM-as-a-Judge связана с остальным стеком.
Используется в
- MT-Bench benchmark Benchmarks LLM-as-a-Judge used by MT-Bench High ✓ source
- RLAIF technique Safety & Alignment LLM-as-a-Judge used by RLAIF High ✓ source