SWE-bench
Real GitHub issues that a model must resolve by editing a repository.
Обзор
SWE-bench takes real issues from open-source repositories and requires a model to produce a patch that makes the project's own tests pass. It solves the problem that isolated programming exercises do not resemble real work, where the task is to find the relevant code in a large repository first. It exists because execution against a real test suite is the hardest-to-game signal for coding ability.
Граф технологий
Как SWE-bench связана с остальным стеком.
Применяется к
- AI Agent application AI Agents SWE-bench used for AI Agent High ✓ source
- Code Generation application Applications SWE-bench used for Code Generation High ✓ source