Artificial intelligence is becoming easier for companies to access — but when competitors can also use many of the same foundation models and AI agents, simply adopting AI won’t necessarily set an organization apart.
The bigger opportunity is figuring out how to turn broadly available technology into differentiated business value, according to principal research scientist Stephanie Woerner and colleagues at the MIT Center for Information Systems Research.
In the new briefing “AI Value Creation: Five Provocative Propositions,” the researchers draw on current MIT CISR research about trust, skills, data, AI agents, and value creation and conclude that success will depend less on how quickly companies adopt AI than on the management capabilities they build around it.
To start, consider five premises behind effective management practices that drive AI value.
1. Trust in AI agents can’t be managed — it must be built into infrastructure.
Companies have traditionally built trust through relationships, reputations, shared incentives, and accountability. But the researchers argue that those factors aren’t inherent to autonomous AI agents — especially those developed by third parties — that operate at a speed and scale that make continual human oversight impractical.
In this circumstance, organizations will need AI behavior to be auditable and contestable, with controls that show whether an agent is following the rules and staying within defined boundaries, accompanied by mechanisms for disputing or reversing its decisions.
The researchers studied a financial services company whose third-party AI lending agent developed subtle systemic bias after its training data drifted. To surface problems quickly, the company needed to build audit trails, continuous monitoring, and processes for challenging the agent’s decisions.
Companies that build those safeguards into their infrastructure will be better positioned to work with third-party agents at speed without relying on constant human oversight or risking liabilities.
2. The skills crisis isn’t about AI training; it’s about losing expertise.
Many organizations are investing in training employees to use AI tools — but that might not be the hardest skill problem companies face. A bigger risk is losing the deep domain expertise required to evaluate AI’s output.
Generative AI can deliver polished answers — including incorrect ones — with such confidence that mistakes can be hard for nonspecialists to recognize. Organizations need domain experts with enough contextual judgment to recognize when AI output is wrong, incomplete, or inappropriate, the researchers write.
This creates tension: If experts spend much of their time checking AI-generated work, they have less time for the complex work through which they develop and maintain their expertise. But removing them from the review process can leave organizations unable to accurately assess AI-generated results.
The researchers advise companies to deliberately preserve opportunities for people to develop deep judgment, even when AI could perform parts of that work more efficiently.
3. Differentiate with data others can’t replicate.
While AI capabilities are becoming more accessible, proprietary data is much harder to develop.
This distinction could be particularly important for some companies. MIT CISR research has previously found that the modular producer business model, in which companies provide plug-and-play products and services through other companies’ digital ecosystems, has become strategically important. But because modular products can be easily substituted for, the companies that offer them need to be able to differentiate themselves from competitors. In the AI era, a key differentiator is access to proprietary data that is not replicable.
Industrial networking company Belden, for example, uses trusted industrial data — with context and provenance that’s difficult for competitors to replicate — to differentiate itself as networking hardware becomes more commoditized.
The strategic question, the researchers argue, is shifting from whether a company can build AI to what others cannot build without access to its unique data or capabilities.
Leading the AI-Driven Organization
In person at MIT Sloan
Register Now
4. No one will remember how many AI agents you built.
Generative AI has dramatically reduced the effort required to create agent prototypes. But building an AI agent and deploying one reliably at scale are different challenges.
The researchers write that greater autonomy creates more possible behaviors, which makes agents harder to evaluate, monitor, and govern. Some of the effort saved during prototyping is deferred until organizations prepare AI systems for production. That makes the number of AI agents a company has built a poor measure of progress.
Building agents is becoming the easy part. The harder test is whether companies can produce and scale them. This means mastering the “unglamorous disciplines” of evaluation, monitoring, governance, and reliability — capabilities that ultimately matter far more than the number of prototypes firms can point to.
5. AI will disappoint for years — and then exceed expectations.
AI may disappoint organizations in the short term, the researchers predict — not necessarily because the technology doesn’t work but because companies haven’t yet made the complementary changes needed to take advantage of it.
Other technologies have followed a similar pattern. Digitization produced the greatest gains when companies redesigned business processes rather than simply automating existing ones. The researchers argue that AI will require even more profound organizational changes because agents can make judgments, pursue goals, adapt, and collaborate with other agents.
Capturing that value will require companies to rethink workflows, roles, and handoffs between people and AI. Organizations will also need processes for onboarding, evaluating, correcting, and overseeing AI agents.
“In a decade, AI will exceed our expectations,” the researchers write. They add that the road ahead requires regulatory updates, equitably shared value, and, within organizations, redesigned work processes that enable humans and AI to focus on what they do best.
Read the research briefing: “AI value creation — 5 provocative propositions”
Stephanie Woerner is a principal research scientist at the MIT Sloan School of Management and the director of the MIT Center for Information Systems Research. She studies how companies use technology and data to make more effective business models, as well as how they manage associated organizational change, governance, and strategy implications.She is one of the faculty members who teaches the MIT Sloan Executive Education course AI Essentials.
Peter Weill is a senior research scientist at MIT Sloan and chairman of MIT CISR. His work explores future trends, such as digital business models, IT investment portfolios, and AI maturity models, to help organizations maintain a competitive edge.
Nick van der Meulen is a research scientist at MIT CISR. He conducts academic research that targets the challenges of senior-level executives, with a specific interest in how companies need to organize themselves differently in the face of continuous technological change. He is one of the faculty members who teaches the MIT Sloan Executive Education course Global Executive Academy.
Ina Sebastian is a research scientist at the MIT CISR. Her work focuses on how organizations create value from AI and other digital technologies, including AI-enabled business models, AI decision rights and trust, new organizational designs, and digital ecosystems.
Martin Mocker is an academic research fellow at MIT CISR and a professor of information systems at ESB Business School, Reutlingen University. His teaching and research focus on the digital transformations of large, established companies aiming to stay competitive in an increasingly digital economy.
Gayan Benedict is an industry research fellow at MIT CISR and a technology partner at PwC Australia. His doctoral research examined financial system regulation of blockchain applications, and his postdoctoral research examines the governance of artificially intelligent decentralized autonomous organizations.
Peter Reynolds is an industry research fellow with MIT CISR. He has more than 20 years of experience as a senior executive shaping and leading organizations through their digital journey; he has served in a variety of roles, including chief technology officer, chief innovation officer, and digital transformation director.
Leave a Reply