How Google finally leapfrogged rivals with new Gemini rollout

Call it America’s next top model.

- Katherine Blunt

Google CEO Sundar Pichai has worked to overhaul the company's AI development strategy.

(AFP)

With the release of its third version this week, Google’s Gemini large language model surged past ChatGPT and other competitors to become the most capable AI chatbot, as determined by consensus industry-benchmark tests.

The results represent public validation for Google employees who, for months, have been conducting their own, personal tests of the model—asking it for jokes, trying to stump it with math problems—and coming away convinced they had something that would finally tilt the LLM field in the company’s favor.

For one of her “vibe checks,” Tulsee Doshi, Gemini’s senior director of product management, asked the model to write in Gujarati, a language that is spoken widely in India but isn’t especially prevalent on the internet. The results were far better than what she had gotten from earlier models.

“[call it signs of life, right?” she said. “People were coming back and saying, ‘I feel it, I think we've hit on something.”

Aaron Levie, chief executive of the cloud content management company Box, got early access to Gemini 3 late last week, several days ahead of the launch. The company ran its own evaluations of the model over the weekend to see how well it could analyze large sets of complex documents.

“At first we kind of had to squint and be like, ‘OK, did we do something wrong in our eval? because the jump was so big,” he said. “But every time we tested it, it came out double-digit points ahead.”

The launch of Gemini 3 has handed Google an elusive victory: The company, for the first time in years, has pulled well ahead in the race to develop artificial intelligence.