The destructive potential of sentient AI

Worries about a superintelligent and sentient artificial intelligence (AI) destroying humanity is not new.

- PROSENJIT DATTA

Science fiction novels and movies have explored the scenario many times. Even some highly respected AI researchers whose breakthroughs have led to the current era of Large Language Models (LLMs) and Generative AI have warned that the technology could soon get too powerful and pose a threat for humans or humanity. In early 2023, hundreds of AI scientists, including Geoffrey Hinton and Yoshua Bengio, whose research led to the current generation of AI, signed a letter that said: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

If Anyone Builds It, Everyone Dies by Eliezer Yudkowsky and Nate Soares is not simply rehashing tropes about superintelligent AI posing a threat to humanity though their book stands out because of the way it examines what the current generation of AI is, what it isn’t yet, and what it can be. The authors, incidentally, were among those who signed the letter along with Drs Hinton and Bengio. However, as they point out in their introduction, they thought the sentence was a severe understatement.