MMLU
Multiple-choice benchmark across 57 academic subjects.
Resumen
MMLU measures a model's knowledge across fifty-seven subjects, from mathematics to law, using multiple-choice questions at roughly undergraduate difficulty. It solves the problem that a single narrow benchmark rewards narrow training: a model tuned for one task can score well while knowing little else. It exists as the most widely quoted general-knowledge figure for a model, which is exactly why its saturation became a story of its own - most current models score high enough that the benchmark no longer separates them.
Grafo de tecnologías
Cómo se conecta MMLU con el resto del stack.
Aplicado a
- LLM concept Models MMLU used for LLM High ✓ source
- LM Evaluation Harness tool Evaluation LM Evaluation Harness used for MMLU High ✓ source