blog.bluedot.org/p/agi-timelines
1 correction found
Humanity’s Last Exam (HLE) contains 2,500 questions contributed by over 1,000 subject-matter experts
Official HLE sources say the benchmark’s 2,500 questions came from nearly 1,000 contributors, not over 1,000.
Full reasoning
The official HLE project page and the Nature paper describing HLE both state that the dataset was built with questions from nearly 1,000 subject-expert contributors, not over 1,000.
- The HLE website says it is "a global collaborative effort, with questions from nearly 1,000 subject expert contributors affiliated with over 500 institutions across 50 countries."
- The peer-reviewed Nature paper uses the same wording: "questions from nearly 1,000 subject expert contributors affiliated with more than 500 institutions across 50 countries."
So the article overstates the number of contributors. The "2,500 questions" part is correct, but the contributor count in the post does not match the benchmark’s own published description.
2 sources
- Humanity's Last Exam
Humanity's Last Exam (HLE) is a global collaborative effort, with questions from nearly 1,000 subject expert contributors affiliated with over 500 institutions across 50 countries.
- A benchmark of expert-level academic questions to assess AI capabilities | Nature
HLE is a global collaborative effort, with questions from nearly 1,000 subject expert contributors affiliated with more than 500 institutions across 50 countries.