MT-Bench
Multi-turn conversation quality judged by a model acting as judge.
Overview
MT-Bench scores a model on multi-turn conversations across several categories, using a strong model as the judge rather than comparing against reference answers. It solves the problem that open-ended dialogue has no single correct output to score against, which left the most important capability unmeasured. It exists because using a model as a judge turned out to correlate well enough with human preference to be useful, and it is also the clearest example of a benchmark that depends on the model it is judging.
Technology Graph
How MT-Bench connects to the rest of the stack.
Uses and depends on
- LLM-as-a-Judge technique Evaluation MT-Bench uses LLM-as-a-Judge High ✓ source