AI infrastructure providers are increasingly talking about applications that rely on agents and data to continually improve and drive returns. Why? Enterprises, currently the thin part of the demand barbell, are key to justifying the capital expenditures in the years ahead.
Neocloud providers, notably CoreWeave and Nebius, outlined strong second quarter revenue growth and swelling backlogs. These companies are also raising debt to plow into building infrastructure. Today, these companies have two types of customers–frontier AI labs and AI-native customers–driving the results. CoreWeave is also starting to see large enterprises like Caterpillar leverage AI throughout operations.
Subscribe to the Constellation Insights newsletter
The working theory in the AI economy is that infrastructure is different this time. Infrastructure is just as important as the applications and software that typically created the most value. Now infrastructure and AI applications are blended together.
In other words, infrastructure spending is an ongoing purchase. That’s why everything related to hardware is inflated. Even Seagate CEO William Mosley noted that enterprises are building AI applications that will continually tap into a data flywheel that leverages storage.
CoreWeave CEO Michael Intrator said enterprise adoption is broadening. Intrator said:
“AI is moving from experimentation into core operations and the organizations that act decisively are creating an advantage. Deployment is no longer the finish line. As AI moves into production, the way applications are built is changing, and the leaders will be those who learn and iterate the fastest.
For the last several years, many organizations treated a model like a deliverable, train it, deploy it and move on. Enterprises no longer operate that way. Training, inference, evaluation and improvement now form a single continuous loop. Models and agents in production generate real-world data. That data informs evaluation, driving new experiments, which improve the model or application before being redeployed into production.
The loop repeats and capability compounds over time. That shift fundamentally changes both the demand curve and the economics of AI. Compute is no longer a onetime requirement concentrated at the beginning of a model’s life. It becomes an ongoing requirement that grows with every application in production and every cycle of improvement.”
Nebius Chief Business Officer Roman Chernin had a similar riff. The company is focused on an open AI ecosystem where customers can take models and customize them for their use cases. Chernin said customers are looking to extract value from specialized knowledge and data to create cost-effective and value-creating AI systems.
This approach is more about a continual flywheel. “We also see more customers bolstering their own models, creating additional demand for inference and grounding throughout the development cycle, not only in production. Reinforcement learning rollouts, evaluations, synthetic data generations and ground and training workflows all require significant inference capacity and reliable access to external information,” said Chernin.
Constellation Research’s Esteban Kolsky, who in his newsletter outlined how open weight models are critical to enterprise control, noted that the customers outlined by the AI neoclouds are a small subset over the broader enterprise customer base.
Andy Jassy, Amazon CEO, said the adoption curve at the moment is “very barbelled.” “The AI labs are consuming gobs and gobs of compute,” he said. “On the other end of the barbell are the enterprises getting real value from AI. We’re in the relative early stages of AI demand.”
Jassy’s bet is that the demand in the middle part of the customer barbell is going to pick up dramatically. The question is when.
Kolsky reckoned that your average enterprise would adopt AI at pace in the next two to four years or so. That timeline could slip or be sped up. But development cycles are compressed. Previous shifts typically took 7 to 10 years to play out.
DigitalOcean Holdings CEO Padmanabhan Srinivasan said the company has more than 6,000 customers leveraging its AI inference engine through its second quarter. Open models are 75% of token volume on DigitalOcean’s platform.
“We believe that the market is shifting towards valuemaxxing, the right model at the right cost for every task, measured in business outcomes per dollar. This shift is a tailwind for us as we believe that value creation opportunities will expand from just whoever built the model to include whoever serves it the best,” said Srinivasan. “Most AI native customers first need inference with the right model at the right performance and the right price for every task. From there, inference graduates into agentic workflows, which use and generate data that requires databases, storage, knowledge bases and observability. That generated data becomes raw material for learning, improving and customizing the models. Agent run times and learning drive demand for compute.”
While the chorus from AI infrastructure players is that it’s different this time, there are multiple unknowns.
- We don’t know how fast enterprise AI adoption will broaden and whether it’ll help save the debt day. AI infrastructure is being driven by companies that don’t generate cash from core businesses and are funded by debt in many cases.
- Open models change the equation and make the most sense for enterprises. Will enterprises procure their own compute?
- SaaS vendors may wind up being the biggest consumers of AI infrastructure because they will be the primary distributor of AI apps.
- It’s also unclear how much of AI infrastructure demand is driven by FOMO and pulled forward.
One thing is certain: Without broad enterprise adoption of AI-based systems the AI infrastructure ecosystem is likely to correct.
How do enterprise leaders play this AI flux? For starters, consider open models. Also see what is already included from your SaaS vendors. Architectures need to ensure you’re not boxed in. And finally should any vendors mention tokens or tokenmaxxing run like hell.
Leave a Reply