HellaSwag
Commonsense sentence completion that is trivial for people and hard for models.
Обзор
HellaSwag asks a model to pick the most plausible continuation of a described situation, with the wrong options generated adversarially so they are superficially similar. It solves the problem that earlier completion benchmarks became trivial once models were trained on enough text. It exists as one of the standard components of an aggregate language-model score.
Граф технологий
Как HellaSwag связана с остальным стеком.
Применяется к
- LLM concept Models HellaSwag used for LLM High ✓ source
- LM Evaluation Harness tool Evaluation LM Evaluation Harness used for HellaSwag High ✓ source