Anthropic has proposed a new draft bill on artificial intelligence safety to Congress that includes provisions to block the distribution of an AI model if it poses a foreseeable, imminent risk of causing over 50 deaths or a billion dollars in property damage. The legislation, which was recently reported by Politico, outlines catastrophic scenarios stemming from biological weapons, cyberattacks, or models evading developer controls.
Under Anthropic's proposed framework, independent evaluators registered with the U.S. Commerce Department would analyze a model's potential for disaster. If an imminent catastrophic risk is found, these evaluators would have 72 hours to notify the Commerce Department, which would then need to request a D.C. judge to halt the model's release rather than allowing federal agencies discretionary blocking power.
This legislative proposal aligns with Anthropic's Advanced AI Framework and follows similar metrics found in California Senate legislation defining catastrophic risk. While tech companies have historically resisted mandatory government oversight in favor of voluntary measures, Anthropic's draft bill goes further than other industry proposals, such as Nvidia's stance that new laws are unnecessary.
This matters because it represents a rare instance of a major AI lab actively shaping potential federal legislation and regulatory frameworks for severe safety risks.