Supercharging AI

Armed with a newly raised 640 million, GROQ thinks it can challenge one of the world’s most valuable companies with a purpose-built chip designed for Al from scratch.

- Richard Nieva

Jonathan Ross's first inkling that something was wrong came back in February while he was speaking to a host of Norwegian Parliament members and tech execs in Oslo. Ross, the 42-year-old CEO of AI chip startup Groq, was in the middle of a demo he hoped would vitalize the languishing company: an AI chatbot that could answer questions almost instantaneously, faster than a human can read. But, for some reason, it was lagging slightly. It unnerved Ross, who was pitching a Groq-powered European data center that would showcase the specialized chips responsible for those super-fast answers. "I just kept checking the numbers," he recalls.

"People didn't know why I was so distracted." The culprit was an influx of new users. A day before Ross's Oslo meeting, a viral tweet from an enthusiastic developer raving about "a lightning-fast AI answer engine" sent tons of new traffic to the online demo, buckling the company's servers. It was a problem, but a good one to have.

When he founded Groq eight years ago, Ross’s idea was to design AI chips explicitly for what’s known in the industry as “inference”: the part of artificial intelligence that mimics human reasoning by applying what it’s learned to new situations. It’s what enables your smartphone to identify your dog as a Corgi in a photo it’s never seen before, or an image generator to imagine Pope Francis in a Balenciaga coat. It’s quite different than AI’s other computational suck: training the massive models to begin with.

But until OpenAI released ChatGPT in late 2022, touching off a global AI frenzy, the demand for superfast inference was limited, and the company was limping along. “Groq nearly died many times,” Ross says from inside the startup’s semiconductor lab in San Jose, California, recalling one low point in 2019 where the startup was a month away from running out of money. “We started Groq maybe a little bit early.”