Apple researchers show how popular AI models ‘collapse’ at complex problems

A new research paper by a group of people at Apple has said that artificial intelligence (AI) ‘reasoning’ is not all that it is cracked up to be. Through an analysis of some of the most popular large reasoning models in the market, the paper showed that their accuracy faces a “complete collapse” beyond a certain complexity threshold.

- 

The researchers put to the test models like OpenAl 03-mini (medium and high configurations), DeepSeek-R1, DeepSeek-R1-Qwen-32B, and Claude-3.7Sonnet (thinking). Their findings showed that the Al industry may be grossly overstating these models’ capabilities. They also benchmarked these large reasoning models (LRMs) with large language models (LLMs) with no reasoning capabilities, and found that in some cases, the latter outperformed the former.

“In simpler problems, reasoning models often identify correct solutions early but inefficiently continue exploring incorrect alternatives — an ‘overthinking’ phenomenon. At moderate complexity, correct solutions emerge only after extensive exploration of incorrect paths. Beyond a certain complexity threshold, models completely fail to find correct solutions,” the paper said, adding that this “indicates LRMs possess limited self-correction capabilities that, while valuable, reveal fundamental inefficiencies and clear scaling limitations”.