HellaSwag
Commonsense sentence completion that is trivial for people and hard for models.
Visão geral
HellaSwag asks a model to pick the most plausible continuation of a described situation, with the wrong options generated adversarially so they are superficially similar. It solves the problem that earlier completion benchmarks became trivial once models were trained on enough text. It exists as one of the standard components of an aggregate language-model score.
Grafo de tecnologias
Como HellaSwag se liga ao resto do stack.
Aplicado a
- LLM concept Models HellaSwag used for LLM High ✓ source
- LM Evaluation Harness tool Evaluation LM Evaluation Harness used for HellaSwag High ✓ source