one year on
California legislature passes controversial frontier AI safety bill SB 1047, sends to Governor Newsom
The bill targets the largest AI models, including those that cost at least $100 million and use 10^26 FLOPS during training, requiring safety protocols and an emergency stop capability that shuts down the entire AI model, as Silicon Valley splits over potential impact on innovation.
California’s legislature passes SB 1047, sending the AI safety bill to Governor Gavin Newsom’s desk. The Safe and Secure Innovation for Frontier Artificial Intelligence Models Act requires developers of the largest AI models—those costing over $100 million to train—to implement safety protocols, an emergency shutdown capability, and annual third-party audits, with enforcement by a new Board of Frontier Models. Violations could trigger injunctions or fines up to $30 million.
Proponents, including Senator Scott Wiener, argue the bill preempts catastrophic harms like AI-enabled weapons or cyberattacks. Anthropic offered cautious support after amendments, and Elon Musk backed the bill. But opposition is fierce: OpenAI and a16z warn it stifles innovation, while Stanford’s Fei-Fei Li and Andrew Ng say it endangers startups and open source. Nancy Pelosi and the Chamber of Commerce also oppose.
The debate now narrows to Newsom’s signature. The governor has not signaled his stance, and the bill would not go into effect immediately, as the Board of Frontier Models is set to be formed in 2026.
Senator Wiener said the bill aims to get ahead of potential AI harms, criticizing federal inaction on tech regulation.
Center for AI Safety director called the bill in the long-term interest of industry, saying a major safety incident would be the biggest roadblock to advancement.
a16z's chief legal officer said the bill would burden startups with arbitrary thresholds, creating a chilling effect.
Stanford professor and AI pioneer said the bill will harm the budding AI ecosystem in a Fortune column.
Stanford researcher called the bill an assault on open source during a Y Combinator event.
Meta's chief AI scientist said the bill would hurt research efforts and is based on an illusion of existential risk pushed by delusional think-tanks.
Omniscience CEO said the bill confuses that all models have hazardous capabilities as defined, and bad actors should be punished, not AI labs.
One year later — open only if you can handle spoilers
Newsom vetoed SB 1047 on September 29, 2024, citing its narrow focus on the largest models and lack of flexibility. The bill’s failure led to a renewed push for federal AI regulation, though Congress has not passed comprehensive legislation as of mid-2026.