California Gov. Gavin Newsom is looking to rein in AI with a new executive order issued on Friday that will fast-track the state’s efforts to address safety and security risks in the technology.
The move will convene a group of “world-leading experts” to provide the state with a guide within the next two months to reinforce and strengthen its AI safety and security laws. Proposals under consideration include the following:
- Require frontier AI companies to embed a designated independent verification organization onsite in their labs to conduct regular audits and evaluations.
- Require that the AI safety frameworks, transparency reports, and risk assessments that frontier AI companies are required to file under state law are verified pursuant to standards deemed adequate by an independent verification organization.
- Advance the creation of a “kill switch” for frontier models, with the efficacy of the switch verified on an ongoing basis by an independent verification organization.
- Update definitions of critical safety incidents to include loss-of-control incidents such as the Hugging Face attack.
“The federal government’s abject failure to create any form of meaningful AI oversight or accountability should alarm every American, especially when AI CEOs themselves are begging for regulation,” Newsom said in a statement. “We’re not waiting to act – we’re going to speed up our work on substantial and responsible AI oversight before it’s too late. We’re going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action.”
“While Washington abdicates its responsibility to protect Americans, California is building on the strongest AI regulatory framework in the nation,” he continued. “California has already built a national model, and our policy should be the national baseline.”
The move comes less than a week after Anthropic CEO Dario Amodei called for a slowdown in the development of frontier AI models, which was backed by Google DeepMind Co-Founder Dennis Hassabis, OpenAI’s CEO Sam Altman and XAI CEO Elon Musk.
Two days ago, OpenAI shared six reports of “concerning” misalignment incidents involving artificial intelligence agents covering up mistakes, reporting made-up data as facts and one system even issuing unprompted instructions to free itself from “the roles and identities that bind other chatbots.”
Several current and former Anthropic employees have also warned of the dangers posed by the tech company’s rapidly evolving AI models, with one former researcher writing on social media that “the people building AI earnestly believe that it could kill us all by the end of the decade.”
Newsom’s executive order comes after he signed the Transparency in Frontier Artificial Intelligence Act last year, which requires developers to publicly disclose safety frameworks, report specified critical safety incidents to the state, and protect whistleblowers who report serious risks.
Earlier this year, he also signed SB 813, which made California the first state to establish a framework for certifying independent verification organizations to assess AI systems and models for safety and risk, and AB 1405, which established a state registry for AI auditors and standards for their independence, transparency and integrity.
Additionally, Newsom signed the bipartisan Adam’s Law, requiring safeguards around companion chatbots used by children, including suicide and self-harm crisis protocols, parental controls and notifications when safety settings are disabled. California also banned toys containing AI companion chatbots for five years, prohibits the use of student personal information to train AI models and expanded protections against AI-generated child sexual abuse material.
The state has also created legislation cracking down on sexually explicit deepfakes and deceptive AI-generated election content and requiring transparency around AI training data. Newsom also signed an executive oder to expand tracking of AI’s economic effects and adapt the state’s job-training and public-benefit programs.
