GPQA
Graduate-level science questions written to resist search and recall.
Overview
GPQA is a set of graduate-level science questions, written and validated by domain experts, designed so that a skilled non-expert with unrestricted web access cannot reliably answer them. It solves the problem that most benchmarks can be passed by recalling training data. It exists because the questions were verified to be difficult in that specific way, which is what makes a high score informative.
Technology Graph
How GPQA connects to the rest of the stack.