基准测试
公开结论背后的具体测试，每一项衡量什么、又不衡量什么。用来分辨真实的能力提升与一个好看的成绩。
11 technologies
- MMLU benchmark Multiple-choice benchmark across 57 academic subjects.
- LMArena benchmark Public arena where anonymous model pairs are ranked by human votes.
- MTEB benchmark Benchmark suite ranking embedding models across many tasks.
- SWE-bench benchmark Real GitHub issues that a model must resolve by editing a repository.
- BEIR benchmark Heterogeneous retrieval benchmark for zero-shot evaluation.
- HumanEval benchmark Python programming problems scored by running the generated code.
- GSM8K benchmark Grade-school mathematics word problems requiring multi-step reasoning.
- MT-Bench benchmark Multi-turn conversation quality judged by a model acting as judge.
- GPQA benchmark Graduate-level science questions written to resist search and recall.
- TruthfulQA benchmark Questions where a plausible-sounding answer is false.
- HellaSwag benchmark Commonsense sentence completion that is trivial for people and hard for models.