‘A company should be able to use a model without giving up the knowledge that makes it unique’: Microsoft CEO Satya Nadella says enterprises shouldn’t be sharing so much data with AI providers
The Microsoft chief warned that corporate data could be at risk thanks to AI models
Companies using AI should be wary about losing control of their data – and the risk of AI developers making use of that knowledge.
That’s according to Microsoft CEO Satya Nadella, who in a recent blog post warned many AI users are "paying twice" thanks to what he calls the "reverse information paradox”.
This is a process through which AI providers, particularly frontier labs, get access to proprietary data about their own customers. Nadella said the trend represents a huge risk for enterprises, and called for protections akin to patents.
The critique is intriguing as Microsoft itself is an AI company, offering plenty of AI-driven products to customers and cramming AI features into its software. Beyond that, Microsoft was an early funder of OpenAI, though that relationship has weakened of late.
Nadella pointed to economist Kenneth Arrow's information paradox, in which the seller of data or knowledge risks giving it away in order to prove it's worth selling. He noted that AI has created the reverse problem, in which the buyer gives away knowledge in order to use the product purchased.
"You essentially pay for intelligence twice, once with money, and again with something even more valuable: the proprietary knowledge you must reveal to make that intelligence useful," he noted. "The better you want the model to perform, the more of that knowledge you have to feed it!"
"Over time, the information asymmetry becomes increasingly skewed. The seller learns more and more about you as you use what you purchased, while you learn very little about what the seller is learning in return."
Sign up today and you will receive a free copy of our Future Focus 2026 report - the leading resource for IT decision-maker insight on priorities and investment areas in AI, security and more.
The ‘AI exhaust’
This concern isn't just about uploading corporate data to systems, or accidentally including proprietary data in a prompt, though those are both concerns. Instead, Nadella said every interaction with AI inherently includes information that could be useful.
"Models learn from 'exhaust,' the prompts people write, the tools agents use, and especially the corrections people make when the model is wrong," he wrote.
"Every correction is distilled into institutional know-how. It’s the kind of knowledge a competitor could never buy, and the kind that leaks almost imperceptibly: trace by trace, correction by correction, eval by eval."
Ironically, Nadella said it was necessary for the "great innovation" of allowing model developers to use public data – some of them face lawsuits for hoovering up copyrighted data to train their systems.
When it comes to business data, providers should share anything learned with customers. Very few actually do this, however.
"If learning flows in only one direction, economic value converges toward the owners of the learning infrastructure rather than the creators of the knowledge itself," he said. "Therefore, it’s imperative that we distribute the learning infrastructure to every firm so that they can control their own learning loop."
Nadella's tips on how to protect your company from AI
Nadella said the existing setup requires a protection system on par with patents, but also advised companies to set a "trust boundary" by considering a few key aspects – some of which may be easier by simply switching to open source AI.
He suggested companies should control their own "evals", or evaluations of the AI system, because that reveals what looks "good" to the organization, and keep ownership of decisions, feedbacks, and other outputs for their own use.
Beyond that, companies should build capability with AI via their own learning environments where models can learn using real workflows without exposing corporate data.
Improving choice can be achieved by decoupling the orchestration layer from any single model, enabling the ability to switch to another model if one is withdrawn. This can also help organizations cut costs by choosing the right model based on their individual needs.
Lastly, he said to "compound" those four ideas to "create your own continuous learning loop".
"In other words, a company should be able to use a model without giving up the knowledge that makes it unique," he added.
FOLLOW US ON SOCIAL MEDIA
Follow ITPro on Google News and add us as a preferred source to keep tabs on all our latest news, analysis, views, and reviews.
You can also follow ITPro on LinkedIn, X, Facebook, and BlueSky.
Freelance journalist Nicole Kobie first started writing for ITPro in 2007, with bylines in New Scientist, Wired, PC Pro and many more.
Nicole the author of a book about the history of technology, The Long History of the Future.
-
Microsoft says small tweaks to model weights could create AI ‘sleeper agents’News Microsoft Research shows technique could work to create a backdoor in AI systems, but says it hasn't yet been spotted in use in the wild
-
‘AI’s price tag is starting to shape its path through the enterprise’: 89% of enterprises have scaled back AI deployments over token costs – and they’re still struggling to track spendingNews Volatile token costs across 2026 have caused major headaches for IT leaders
-
Microsoft wants to tackle the biggest issue for UK AI adoption: confidenceNews UK and Ireland CEO Darren Hardman says it's our responsibility to get AI into every corner of the UK
-
"Right now, your washing machine is subject to more safety rules than AI": Microsoft's Brad Smith wants more AI kill switchesNews Brad Smith has called for more regulation, saying the UK is well-positioned to lead
-
Huawei is positioning itself as a viable alternative to big US tech companies. Can it pull it off?Opinion With Nvidia and AMD excluded from the Chinese AI market, Huawei is building a powerful reputation at home and abroad
-
“Enterprises are becoming much more rigorous about the economics of AI”: Snowflake wants to help you cut AI costs by choosing the right model for the right taskNews New dynamic model routing capabilities aim to help customers box clever when it comes to their AI model choice
-
What is AWS Trainium?In-depth Learn about the AI accelerator chips known as AWS Trainium and understand how, where, and why they are used within AI infrastructure...
-
Poor business context is scuppering enterprise AI adoption – here’s why that mattersNews Research from Alteryx has found that more than half of organizations can't effectively operationalize the business knowledge AI needs