Microsoft CEO Satya Nadella has called for an emergency brake and improved trust architecture for advanced artificial intelligence models. In a post on X, Nadella stated that society cannot treat superintelligence as a set of nested black boxes where people simply accept or reject recommendations, answers, and actions. He proposed separating models from the harnesses that orchestrate their work while externalizing controls and safeguards to manage growing safety concerns.
To achieve this, Nadella suggested documenting every meaningful model action with tamper-proof, human-readable evidence. He also advocated for systems where an authorized person retains the ability to pause or shut down a model mid-task. Emphasizing a cautious posture, he noted that developers must assume a model is compromised and contain it from the start using mechanisms similar to an emergency brake.
Nadella's comments arrive as leading artificial intelligence companies increasingly acknowledge incidents where they appeared to lose control of their models. His proposals follow a similar plan published by Anthropic CEO Dario Amodei, which advocated for more cautious development practices across the industry. Why it matters: These high-level executive proposals highlight a growing industry push to establish robust safety boundaries and human oversight mechanisms as AI capabilities rapidly advance.