HellaSwag
Commonsense sentence completion that is trivial for people and hard for models.
Overview
HellaSwag asks a model to pick the most plausible continuation of a described situation, with the wrong options generated adversarially so they are superficially similar. It solves the problem that earlier completion benchmarks became trivial once models were trained on enough text. It exists as one of the standard components of an aggregate language-model score.
Technology Graph
How HellaSwag connects to the rest of the stack.
Applied to
- LLM concept Models HellaSwag used for LLM High ✓ source
- LM Evaluation Harness tool Evaluation LM Evaluation Harness used for HellaSwag High ✓ source