HellaSwag
Commonsense sentence completion that is trivial for people and hard for models.
Aperçu
HellaSwag asks a model to pick the most plausible continuation of a described situation, with the wrong options generated adversarially so they are superficially similar. It solves the problem that earlier completion benchmarks became trivial once models were trained on enough text. It exists as one of the standard components of an aggregate language-model score.
Graphe des technologies
Comment HellaSwag se rattache au reste de la pile.
Appliqué à
- LLM concept Models HellaSwag used for LLM High ✓ source
- LM Evaluation Harness tool Evaluation LM Evaluation Harness used for HellaSwag High ✓ source