Supercharging AI

Nvidia has utterly dominated the market for AI chips by repurposing semiconductors originally designed for video games. Now GROQ, a tiny eight-year-old startup, is taking on one of the world's most valuable companies with a purpose-built chip designed for AI from scratch.

- Richard Nieva

Jonathan Ross' first inkling that something was wrong came back in February while he was speaking to a host of Norwegian Parliament members and tech execs in Oslo. Ross, the 42-year-old CEO of AI chip startup Groq, was in the middle of a demo he hoped would revitalize the languishing company: an AI chatbot that could answer questions almost instantaneously, faster than a human can read. But it was lagging slightly. It unnerved Ross, who was pitching a Groqpowered European data center that would showcase the specialized chips responsible for those superfast answers. "I just kept checking the numbers," he recalls. "People didn't know why I was so distracted."

The culprit was an influx of new users. A day before Ross's Oslo meeting, a viral tweet from a tech founder raving about "a lightning-fast AI answer engine" sent tons of new traffic to the online demo, buckling the company's servers. It was a problem, but a good one to have.

When he cofounded Groq eight years ago, Ross' idea was to design AI chips explicitly for what's known in the industry as "inference": the part of artificial intelligence that mimics human reasoning by applying what it has learned to new situations. It's what enables your smartphone to identify your dog as a corgi in a photo it's never seen before, or an image generator to imagine Pope Francis in a Balenciaga coat. It's quite different than Al's other computational suck: training the massive models to begin with.