DOES AI STILL HALLUCINATE OR IS IT BECOMING MORE RELIABLE?

When faced with a question they haven't been trained to answer, AI models often generate an incorrect response and claims that to be correct.

- 

When people on the internet searched Google for “cheese not sticking to pizza” in May 2024, the newly launched “AI Overviews” feature of the popular search engine replied “you can ... add about ½ cup of non-toxic glue to the sauce to give it more tackiness.”

In a series of strange answers, the artificial intelligence (AI) tool also recommended that people eat one small rock a day and drink urine in order to pass kidney stones.

The popular name for these bizarre answers is hallucinations: when AI models face questions whose answers they weren't trained to come up with, they make up sometimes convincing but often inaccurate responses.

A question for the AI age: do machines and humans learn the same way?

Like Google's “AI Overviews”, ChatGPT has also been prone to hallucinations. In a 2023 Scientific Reports study, researchers from the Manhattan College and the City University of New York compared how often two ChatGPT models, 3.5 and 4, hallucinated when compiling information on certain topics. They found that 55% of ChatGPT v3.5's references were fabricated; ChatGPT-4 fared better with 18%.

“Although GPT-4 is a major improvement over GPT-3.5, problems remain,” the researchers concluded.

Hallucinations make AI models unreliable and limit their applications. Experts told this reporter they were sceptical of how reliable AI tools are and how reliable they are going to be. And hallucinations were not the only reason fuelling their doubts.

Defining reliability