The risks of the generative AI gold rush

Companies are rushing in to make money from generative AI chatbots such as OpenAI's ChatGPT, and people are embracing them, too. But we must factor in the risks, says Nicole Kobie

- Nicole Kobie

The backlash didn't take long. OpenAI released the latest version of its ChatGPT in the autumn of 2022, and within weeks startups were taking advantage of the generative AI tool and the large language model that powers it. In 2022 alone, $1.4 billion was reportedly invested in generative Al companies in 78 deals (pcpro.link/344futurism).

But warnings about the technology arose just as quickly. First, people didn't understand the text was grammatically correct but not necessarily factual; that's not a flaw but how these systems inherently work, although that was apparently news to many (Indeed, ChatGPT itself warns that it "may occasionally generate incorrect information", can be biased and has limited knowledge after 2021.)

Critics also raised concerns about the ownership and quality of the data on which the models were trained, wondering where future data sets could be sourced. Then came the hackers and researchers, the edges of the controls for the systems, in order to break them.

Those shaking the most with fear over Al advancements weren't regulators or ethicists but search incumbents. Google and Microsoft both launched their own generative Al chatbots, rushing out products to avoid being left behind. Google immediately raised eyebrows - and slashed 8% from the company's stock price after its Bard chatbot not only returned an incorrect fact about space photography, but used the example in the company's marketing material.

Microsoft Bing's chatbot is powered by OpenAI's systems but without some of the controls put in place to avoid returning embarrassing answers. Which is how it told one journalist to quit their unhappy marriage, refused to accept what year it was, and even vaguely threatened to harm one researcher.