Do Not Be Fooled By AI Hallucinations

All AI models, especially large language models (LLMs), are prone to hallucinating—that is, they sometimes give wrong or fictitious responses that appear correct. Experts recommend providing unambiguous prompts and cross-checking responses before making decisions based on them.

- Janani G. Vikram

If you regularly use AI models, you might have encountered some instances where they confidently deliver incorrect, nonsensical, or fabricated answers—much like an egoistic person who does not like to admit that they do not know something! For example, when asked about a spiritual magazine, it may associate it with a popular guru or cult with which it is not even remotely connected. Or, when asked to list the top 10 models of a product, it might include models that have been discontinued. Even worse, it might simply fabricate the information when asked for a legal case precedent or a patient’s medical records!

Sometimes, the answer is so obviously nonsensical that the user is unlikely to be misled. But at other times, the chatbot sounds so confident that the user might proceed on the assumption that the answer is correct—and ultimately land in trouble because of that. This is precisely what happened when a legal team presented case precedents suggested by ChatGPT, which turned out to be fictitious! They had to contend with public disapproval and hefty fines imposed by the court.

In another incident, Air Canada had to partially compensate a customer who was misled by the AI chatbot’s fabricated information about the company’s bereavement travel policies. There have been several other examples of generative AI models providing slanderous information about celebrities, inexistent bank transactions, imaginary patient records, and incorrect company policies.

Such cases where AI models present factually incorrect, illogical, or fabricated responses—often in a very confident tone—are known as AI hallucinations.