Anthropic's AI Safety Bill: Block Models That Kill 50+

Anthropic has drafted its own AI safety bill and is shopping it around Congress, according to Politico. The legislation would let regulators block the release of a model under a narrow set of conditions: a "foreseeable, imminent risk" that more than 50 people could die, or that the damage could exceed a billion dollars in property losses. The risk has to trace back to one of three sources, a biological weapon, a model slipping its developer's controls, or a cyberattack.

Those numbers aren't new. California's Senate Bill 53, signed into law recently, uses the same 50-deaths and billion-dollar thresholds to define what it calls "catastrophic risk." Anthropic's proposal leans on that precedent while adding a new wrinkle: independent evaluators, registered with the Commerce Department, who would assess whether a model clears the bar. The bill would allow AI labs to pay those evaluators directly, with one condition attached, that compensation can't be tied to the outcome of the evaluation.

The move puts Anthropic in the position of writing its own rulebook before Washington does, at a moment when federal AI legislation remains stalled and state-level efforts like California's are filling the gap.