Analytics firm SemiAnalysis has released the GDPx benchmark, testing AI agents on 206 complex tasks across realistic professional software environments. The initial results show that no AI agent scores above 12 percent, highlighting significant limitations in current automated knowledge work capabilities.
Claude Opus 5.5 leads the leaderboard with a 12 percent score, closely followed by Google's Gemini 4 Carbon at 10.9 percent and GPT-6 Astra at 9.6 percent. The benchmark evaluates performance over several hours across eleven domains including chip design, quant trading, and cybersecurity, utilizing strict evaluation criteria where even minor spreadsheet errors invalidate outcomes.
Safety filters and varying provider software conditions heavily influence model performance and rankings across different specialized test domains. While closed-task secrecy prevents AI labs from optimizing their models directly for the test questions, critics note that small task counts per domain make the results volatile.
This benchmark matters because it exposes the wide gap between theoretical AI capabilities and reliable execution in real-world professional software environments.