WHAT IS AN NPU?

The latest specialist chips put AI at your fingertips. Darien Graham-Smith finds out what's new in neural processing

- Darien Graham-Smith

We’re all familiar with the CPU and the GPU – they sit at the heart of almost every PC made in the past 40 years (albeit the GPU has sometimes been built into the CPU). Lately, though, they’ve been joined by a new friend: the neural processing unit, or NPU.

NPUs aren’t a brand-new idea. The first implementations appeared, with little fanfare, in mobile chipsets in 2017. But in the past few years, as AI workloads have exploded onto the scene, they’ve become more and more important. Today it could be reasonably argued that the rise of this new type of processor is the most significant development in systems architecture in 50 years.

The reason is that, while the NPU is simpler and more specialised than either the CPU or GPU, it opens up a whole new dimension of computing capabilities, enabling the sort of complex on-device AI processing that a regular CPU would struggle with.

This doesn’t mean you can run a complete ChatGPT or Midjourney engine on your personal laptop. But it does mean that almost any application can now take advantage of the sort of AI processing functions that power those platforms – and it provides a standard hardware model for growing and improving these capabilities in the future. When you next buy any sort of consumer electronic device, from a high-end laptop to a smart TV or a home security gadget, there’s a good chance it will include an NPU.

What does an NPU do?

NPUs are designed specifically for AI operations, which in practice means working with data structures called tensors. Entire books have been written about exactly what tensors are and what can be done with them, but the simple way to think about a tensor is as a matrix of values that can have any number of dimensions.