For years, scientists have measured artificial intelligence using tests originally designed for humans.
But as AI systems improved, many of these exams became too easy. Programs began scoring near-perfect marks on benchmarks that once seemed extremely difficult, making it harder for researchers to tell how smart the systems really were.
To solve this problem, nearly 1,000 experts from around the world created a new and much tougher assessment called “Humanity’s Last Exam.”
The goal was not to trick AI, but to clearly reveal the limits of what current systems can do.
The project, described in the journal Nature, includes 2,500 questions covering a vast range of topics, from mathematics and science to history, languages and specialized academic fields.
The questions were written and reviewed by specialists to ensure they required deep knowledge rather than simple pattern recognition.
Each problem has one clear, correct answer that cannot be easily found through a quick internet search. Some tasks involve translating rare ancient texts, identifying tiny biological structures or analyzing subtle details in historical languages.
These are areas where human expertise still plays a major role.
To make the test especially challenging, the team checked each question against leading AI models. If a system could answer it correctly, that question was removed. The result is an exam deliberately placed just beyond the reach of today’s technology.
Early results show that even the most advanced AI systems struggle. Some widely known models answered only a small percentage of the questions correctly. Even newer, more powerful systems reached only about half of the answers at best. This suggests that despite impressive progress, AI still lacks the depth of understanding and context that humans use when solving complex problems.
Researchers say creating a tougher benchmark is important because without accurate tests, people may misunderstand what AI can truly do. High scores on outdated exams might give the impression that machines are approaching human intelligence, when in reality they are simply good at specific tasks.
Despite its dramatic name, Humanity’s Last Exam is not meant to signal the end of human importance. Instead, it highlights how much knowledge remains uniquely human. Understanding where AI struggles can help scientists build safer and more reliable systems while reminding society that human skills and expertise are still essential.
The exam is designed to remain useful for years to come. Only part of it has been released publicly, while most questions remain hidden to prevent AI systems from memorizing answers. This ensures the test continues to measure genuine reasoning ability rather than recall.
The project also demonstrates the power of global collaboration. Experts from many disciplines worked together to create a test that no single field could design alone. Ironically, the effort shows that while AI is advancing rapidly, human cooperation and diverse knowledge remain its greatest benchmark—and its greatest advantage.
Source: Texas A&M University
