HellaSwag
Commonsense sentence completion that is trivial for people and hard for models.
Resumen
HellaSwag asks a model to pick the most plausible continuation of a described situation, with the wrong options generated adversarially so they are superficially similar. It solves the problem that earlier completion benchmarks became trivial once models were trained on enough text. It exists as one of the standard components of an aggregate language-model score.
Grafo de tecnologías
Cómo se conecta HellaSwag con el resto del stack.
Aplicado a
- LLM concept Models HellaSwag used for LLM High ✓ source
- LM Evaluation Harness tool Evaluation LM Evaluation Harness used for HellaSwag High ✓ source