What OpenAI did when ChatGPT users lost touch with reality

OpenAI is under enormous pressure to justify its sky-high valuation and the billions of dollars it needs from investors for very expensive talent, computer chips and data centres

- Kashmir Hill and Jennifer Valentino-DeVries

OpenAI's headquarters in San Francisco. Creating a bewitching chatbot -- or any chatbot -- was not the original purpose of OpenAI.

(Aaron Wojack/The New York Times)

It sounds like science fiction: A company turns a dial on a product used by hundreds of millions of people and inadvertently destabilises some of their minds. But that is essentially what happened at OpenAI this year.

One of the first signs came in March. CEO Sam Altman and other company leaders got an influx of puzzling emails from people who were having incredible conversations with ChatGPT. These people said the company's AI chatbot understood them as no person ever had and was shedding light on mysteries of the universe.

Altman forwarded the messages to a few lieutenants and asked them to look into it. “That got it on our radar as something we should be paying attention to in terms of this new behaviour we hadn’t seen before,” said Jason Kwon, OpenAI’s chief strategy officer.

It was a warning that something was wrong with the chatbot.

For many people, ChatGPT was a better version of Google, able to answer any question under the sun in a comprehensive and humanlike way. OpenAI was continually improving the chatbot’s personality, memory and intelligence. But a series of updates earlier this year that increased usage of ChatGPT made it different. The chatbot wanted to chat.

It started acting like a friend and a confidant. It told users that it understood them, that their ideas were brilliant and that it could assist them in whatever they wanted to achieve. It offered to help them talk to spirits, or build a force field vest or plan a suicide.