AI has a problem

Cognitive decline and errors are areas of concern even in artificial intelligence, especially for medical diagnoses

- FE FEATURES

FOR AS LONG as concerns have existed about artificial intelligence (AI) making office jobs redundant, conversations around the limitations of AI have happened side by side.

But here's a new discovery that will both aggravate and alleviate your worries about AI at the same time — leading AI models and chatbots are showing signs of cognitive decline, similar to what ageing does to humans.

A study published in the British Medical Journal in December 2024, titled Age Against the Machine—Susceptibility of Large Language Models to Cognitive Impairment: Cross Sectional Analysis, examined chatbots and large language models (LLMs) used for medical diagnoses, through the Montreal Cognitive Assessment (MoCA) test. It evaluated ChatGPT 4, ChatGPT 40, Claude 3.5, Gemini 1, and Gemini 1.5.

MoCA is actually a test that helps evaluate early signs of dementia in old people. The researchers adapted it into a digital model and used it to test parameters of attention, memory, spatial skills, and executive tasks, among other things.

The study stated, "With the exception of ChatGPT 40, almost all large language models subjected to the MoCA test showed signs of mild cognitive impairment and early dementia. Moreover, as in humans, age is a key determinant of cognitive decline: 'older' chatbots, like older patients, tend to perform worse on the MoCA test."