Cutting-edge AI Was Thought to Get Cheaper. It's Costlier Now

As artificial intelligence got smarter, it was supposed to become too cheap to meter. It's proving to be anything but.

- Christopher Mims

Developers who buy AI by the barrel, for apps that do things like make software or analyze documents, are discovering their bills are higher than expected—and growing.

What's driving up costs? The latest AI models are doing more "thinking," especially when used for deep research, AI agents and coding. So while the price of a unit of AI, known as a token, continues to drop, the number of tokens needed to accomplish many tasks is skyrocketing.

It's the opposite of what many analysts and experts predicted even a few months ago. That has set off a new debate in the tech world about who the AI winners and losers will be.

"The arms race for who can make the smartest thing has resulted in a race for who can make the most expensive thing," says Theo Browne, chief executive of T3 Chat.

Browne should know. His service allows people to access dozens of different AI models in one place. He can calculate, across thousands of user queries, his relative costs for the various models.

Penny wise, pound-foolish

Remember, AI training and AI inference are different. Training those huge models continues to demand ever more costly processing, delivered by those AI supercomputers you've probably heard about. But getting answers out of existing models—inference—should be getting cheaper fast.

Sure enough, the cost of inference is going down by a factor of 10 every year, says Ben Cottier, a former AI engineer who is now a researcher at Epoch AI, a not-for-profit research organization that has received funding from OpenAI in the past.