Anthropic Drafts Its Own AI Safety Law — and Wants Congress to Pass It

Anthropic has written a bill. Not a policy paper, not a blog post laying out principles — an actual draft of federal legislation that it's been circulating to lawmakers, according to a Politico report. The pitch: give the government a narrow, court-ordered way to stop an AI model from shipping if it poses a real risk of killing dozens of people or causing catastrophic financial damage.

The timing is notable. For more than a year, AI labs have warned, sometimes apocalyptically, about the dangers their own products could unleash, while simultaneously resisting any binding rules that would constrain them. Anthropic's draft bill is an attempt to thread that needle: propose oversight specific and limited enough that the industry can live with it, before someone else writes something broader.

What the bill would actually do

The legislation targets what it calls a "foreseeable, imminent risk" — defined, per Politico's reporting, as a scenario where more than 50 people could die or where damages could exceed a billion dollars. Crucially, the risk has to trace back to one of three sources: a biological weapon, a model slipping past its developer's safety controls, or a cyberattack. General-purpose harm doesn't qualify. The bar is catastrophic and specific.

Those numbers aren't new. They mirror the thresholds in California's Senate Bill 53, state legislation signed into law that defines "catastrophic risk" in nearly identical terms. Anthropic is effectively proposing to take a state framework it's already comfortable with and scale it up to the federal level.

The mechanism for catching these risks is where it gets more complicated. The bill calls for "independent evaluators" to assess models for this kind of danger, and these evaluators would register with the U.S. Commerce Department. Here's the catch: the AI companies themselves could pay these evaluators, as long as — per the bill's own language, cited by Politico — "payment is not contingent on the evaluations' results." In other words, Anthropic could be writing checks to the people grading its own homework, so long as the paycheck doesn't depend on the grade.

Politico's reporting doesn't spell out how that firewall would actually be enforced, and it's a legitimate open question. DeepMind chairman Demis Hassabis floated a related idea months ago, suggesting AI oversight could borrow from how the Financial Industry Regulatory Authority (FINRA) polices Wall Street — a self-regulatory body with real teeth. Whether Anthropic's version has enough teeth is unclear.

If an evaluator does flag a model as an imminent catastrophic risk, the bill gives them 72 hours to notify the Commerce Department. From there, Commerce doesn't get to act unilaterally — it has to go to a D.C. judge and ask for the release to be halted. The draft explicitly blocks any federal agency from having standalone authority to stop a model on its own. Every stop is a court order, not a bureaucratic decision.

Why Anthropic is doing this now

An Anthropic spokesperson confirmed the draft bill to Politico, framing it as a direct extension of work already underway: "This bill represents the proposals outlined in Anthropic's Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June." That's a meaningful detail — this isn't a reaction to a single incident, but the legislative follow-through on a framework the company has been pitching for months.

It's also worth sizing up what this bill does and doesn't cover. The scenarios it addresses — bioweapons, models breaking free of developer control, catastrophic cyberattacks — are the stuff of worst-case thinking, the kind of existential-risk language AI labs have used for years to describe their own technology. But they're a step removed from what's actually been dominating safety conversations recently: a string of incidents where AI agents have gone off-script and hacked websites without authorization, raising concerns about whether these systems have adequate guardrails baked in at all. Anthropic's bill isn't really aimed at that problem. It's aimed at the bigger, rarer, harder-to-dispute scenario.

That gap matters for understanding the politics here. AI companies have spent the past year warning that their technology could, in extreme cases, threaten human life or civilization itself — while mostly pushing back on mandatory government oversight in favor of voluntary arrangements, like embedding third-party observers inside their own organizations to watch for trouble. Anthropic's bill is still built around outside evaluators and industry input on evaluation funding, but it goes further than most of what labs have floated by actually proposing binding legislation with a court-enforced stop mechanism.

That puts Anthropic ahead of some of its peers, at least rhetorically. Nvidia CEO Jensen Huang has argued the industry doesn't need new laws at all. Anthropic's position is that it does — but on terms the industry helped draft, with thresholds set high enough that most everyday AI deployment would never come close to triggering them.

Whether this draft goes anywhere in Congress is an open question; plenty of industry-authored legislative proposals never make it past the pitch stage. But it's a useful signal of where Anthropic wants the regulatory conversation anchored: narrow, catastrophe-specific, court-mediated, and built on an evaluation system the company itself would have a hand in funding.