Artificial intelligence appears to be more expensive than anyone expected, and CFOs at major US companies are now faced with a brutal new trade: tokens or people.
This was the image two AI business executives at the center of the building described on CNBC this week. Their account of what happened inside the Fortune 500 paints a stark picture of the threat posed by the rising costs of AI trading. It’s a risk the market hasn’t seen yet as it reaches record highs and creates new billion dollar companies like Micron.
The number one topic for all businesses right now is overspending AI budgets, Arvind Jain, CEO of AI company Glean, told CNBC.
“Companies tell us that their AI budgets run out in one or two months, but these are annual budgets,” he said.
That’s because the cost of AI isn’t coming down as much as consumers expected. Instead, it has increased. Each new model release from frontier labs costs twice as much per token as the one it replaces, which puts business AI in what Jain calls “an untenable path right now.”
“This is the first time I remember that technology is as expensive as people, and you make that comparison: choose technology or people,” he said. “We’ve never had that conversation historically, because technology is part of the total cost of any viable business.”
That growing AI budget, he says, is increasingly being replaced by future population growth.
Arvind Jain, CEO of Glean, on the SaaS Monster stage during the first day of Web Summit 2022 at the Altice Arena in Lisbon, Portugal, on Nov. 2, 2022.
Harry Murphy | Sports File | Getty Images
Matan Grinberg, CEO of Factory AI, who leads the engineering work on all AI models at the frontier, explained the change as a problem of resource allocation that exists now in leadership teams.
“Companies are saying, hey, if we can increase one thing, is it the number of employees we have, or the amount of money AI spends per employee?” Grinberg said.
Grinberg said the companies went through three different phases in about a year. The first involved boards want their executives to do something about AI. Then came so-called tokenmaxxing, or using AI in any way necessary without cost. In the third phase, leadership teams review their needs when it comes to premium models.
“Do we need to use Opus-level intelligence for every single task?” Grinberg said. “You just don’t need to.”
Paying more than you get back
The root of the squeeze is that the technology is working but not yet paying off.
“The way AI works today, it’s very powerful, but it’s not efficient,” Jain said. “AI-driven value right now is following the costs businesses are making.”
A big part of the problem is inefficiency in choosing models. About 95% of enterprise AI implementations still operate on borderline expensive models, even for tasks that can be handled by cheaper alternatives, Jain said.
There is an easy fix: move the easy task to the cheaper category. Jain said that is low hanging fruit.
“You have 10x the savings you can get if you use the right route up front,” he said.
That’s also the voice behind Factory AI, which automatically sends each task to the model that best suits it. The trick, Grinberg says, is realizing how rare the job is that you need to top the line. He matched the gap between the very young models and two seasoned academics.
“Opus 4.7 versus Opus 4.8 is like the difference between a 13-year professor and a 15-year professor,” Grinberg said. “For the average person, it’s really hard to tell the difference.”
All AI trading is based on a bet that historical demand will remain, with buyers indifferent to costs. But a view from inside the Fortune 500 suggests that demand may be more sensitive to prices than the trade assumes.
Read more about what it means to calculate the price of AI in the comparison of OpenAI and Anthropic, who built their business models on high prices.
WATCH: CNBC’s full interview with Altimeter’s Pauline Yang
