Mirror, mirror

We've created something in our own image, and have started to worry about how well we know it. The danger with AI, the experts say, isn't that it will 'come to life', but that we will increasingly outsource to it the things that make us human: judgment, curiosity, responsibility. The result: quiet degradations; the automation of agency itself. Meanwhile, a very real near-future fear is that, when we give these machines a mission, we may not be able to predict how they will set about achieving it (or the impacts of that on us all). Kashyap Kompella explores reasons you needn't worry, and reasons why you should

- Kashyap Kompella

If the machines ever wake up, it won't look like it does in the movies. There won't be glowing red eyes or ominous monologues.

It will begin with a log entry or a line of code in a server somewhere that says: "Starting". And maybe it already has.

In an experiment reported in May, Anthropic found that, when given access to information about its planned shutdown, and access to data on a possible affair by the engineer in charge of that closure, its AI model Claude attempted to use the data to blackmail him, in order to stay powered on.

During safety testing in December, meanwhile, OpenAl's GPT tried to copy itself onto external servers, to avoid being decommissioned.

Both companies clarified that the behaviours were a result of messed-up optimisation, not self-awareness. But let's be honest: We've built systems that imitate humans so well, we even call them neural networks. And we can't tell exactly how those networks might evolve.

In the shadows

Think of it in terms of Plato's allegory of the cave.

The prisoners in his cave saw shadows on the wall and mistook them for reality. We are doing the same with AI chatbots.

It is understandable, given the astonishing capabilities of today's AI, to feel that there is "something out there". And that we can't always tell what it is, or where it is going.

We comfort ourselves by repeating: There is no sentience. And to be sure, there isn’t.

No machine we have built has displayed a sense of self, or operated outside the bounds of what we told it to do.

And yet some have passed the Turing test (in which a person must distinguish between a human response and a computer's, based on questions posed to both). The test was designed in 1949 and is routinely passed by modern chatbots.

What does this mean?

Well, Turing’s famous test wasn’t about consciousness; it was about deception. And the truth is we are being “deceived”, in each interaction.