Survival of the Fittest

The evolutionary case that humanity is already on its way to giving up its own dominance

- By Dan Hendrycks

A broad coalition of AI experts recently released a brief public statement warning of "the risk of extinction from AI." There are many different ways in which Als might become serious dangers to humanity, and the exact nature of the risks is still debated, but imagine a CEO who acquires an AI assistant. They begin by giving it simple, low-level assignments, like drafting emails and suggesting purchases. As the AI improves over time, it progressively becomes much better at these things than their employees. So the AI gets "promoted." Rather than drafting emails, it now has full control of the inbox. Rather than suggesting purchases, it's eventually allowed to access bank accounts and buy things automatically.

At first, the CEO carefully monitors the work, but as months go by without error, the AI receives less oversight and more autonomy in the name of efficiency. It occurs to the CEO that since the AI is so good at these tasks, it should take on a wider range of more open-ended goals: "design the next model in a product line," "plan a new marketing campaign," or "exploit security flaws in a competitor's computer systems." The CEO observes how businesses with more restricted use of Als are falling behind, and is further incentivized to hand over more power to the AI with less oversight. Companies that resist these trends don't stand a chance. Eventually, even the CEO's role is largely nominal. The economy is run by autonomous AI corporations, and humanity realizes too late that we've lost control.

These same competitive dynamics will apply not just to companies but also to nations. As the autonomy of Als increases, so will their control over the key decisions that influence society. If this happens, our future will be highly dependent on the nature of these AI agents.