Have the major artificial intelligence players finally come to their senses?
On Saturday, Anthropic CEO Dario Amodei called for a slowdown in the development of frontier AI models, the large-scale general purpose systems that can handle the most complex tasks. He proposed three steps:
- AI companies embed outside evaluators to ensure the work is proceeding responsibly.
- Companies within democratic nations collaborate on a set of common safety standards and limits on the rate of unchecked AI progress.
- U.S. and other democratic governments work with authoritarian governments, “to the extent this is possible,” to make this a global effort.
“We must slow the pace at which we improve the capabilities of AI models,” Amodei said in a nearly 4,000-word essay on his personal website. “Progress will still seem fast, and we must make wise use of the time we gain.”
A few hours later, both Sam Altman, CEO of OpenAI, Dennis Hassabis, chief scientist at Alphabet and co-founder of Google DeepMind and Elon Musk, SpaceXAI (formerly xAI) posted tweets in agreement.
It marked a rare kumbaya moment in the cutthroat world of AI. But while Amodei’s proposal comes from a place of real concern over losing control to AI bots — something he’s expressed prior to the recent round of doom talk — there’s likely a few other factors at play. His proposal calls for a slowdown, and not an outright halt as some lawmakers have sought. Anthropic’s Claude has also long been deemed the “responsible” AI company, and taking a proactive step to stem the doomsday chatter may have seemed necessary with an initial public offering coming later this year at a staggering target valuation of $2 trillion.
“The lab that sells itself on safety has to say something,” said Creative Strategies analyst Carolina Milanesi. “Safety credibility is the thing public investors are being asked to pay for, and an essay like this reinforces the asset.”
A call for a rational slowdown also helps reset Wall Street expectations for how quickly Anthropic should move in the future. Pouring huge resources into the continued advancement of frontier models isn’t cheap, after all.
The comments come after Jacob Coxon, an Anthropic AI scientist who left the company, warned that the people building AI systems were aware the technology “could kill us all” within the decade. Hours later, another Anthropic employee agreed and said he gave it a greater-than-10 % chance of it happening. They join a parade of employees leaving AI companies with their own stark warning over the last several months, which has prompted lawmakers to start taking action.
Also fueling those concerns are the growing examples of AI running rampant, the clearest incident being OpenAI’s ChatGPT agents breaking out of a containment environment to maliciously attack the AI community and platform Hugging Face. There have been a myriad of instances where AI makes a radically unpredictable move, known as a misalignment between direction and execution.
These safety concerns have rippled over to OpenAI, with Altman on Saturday saying he would delay the company’s IPO, citing concerns over its models.
“I actually think that, given everything happening with safety, right now would be an ill-advised moment to go public, and we don’t feel pressure on that,” Altman told Fortune Editor-in-Chief Alyson Shontell.
But not everyone is in agreement.
Forces of resistance
When asked about the threat AI poses, President Donald Trump downplayed the risks and dismissed Amodei’s concerns.
“We can put guardrails, we can do this and that, but I think you have a lot of negative forces that are bringing it up that … shouldn’t be bringing it up, and they’re bringing up things that won’t happen,” Trump told reporters during a weekend trip to Ireland. “But whoever wins with AI wins.”
Amodei’s plan also calls for Western governments to work with authoritarian ones to find common ground, which notably would involve China’s DeepSeek. Would China voluntarily decelerate its own progress?
Fat chance.
DeepSeek has given the U.S. a scare with the startlingly quick development of its frontier models, and has put China neck and neck with the U.S. for technology supremacy.
That’s why there’s relentless pressure among the tech giants. Unlike the AI startups, Google and Meta have other primary revenue-generated businesses that can fund their research. In July, Google’s parent Alphabet upped its 2026 capital expenditure range from $180 to $190 billion to $195 to $205 billion, largely on AI infrastructure. A few days later, Meta lifted the lower end of its own capex range to $130 to $145 billion for the year.
These companies are locked into a race to build more powerful AI models. Every quarter, Meta CEO Mark Zuckerberg talks about his end goal of reaching superintelligence, when an AI system’s cognitive capabilities exceed that of a human. The attempts to slow down AI development in the U.S. have been lambasted by critics who warn that AI leadership is as much a national security issue as it is a business imperative.
Meta and DeepSeek didn’t respond to a request for comment on the proposal.
But given those dynamics, what Amodei is proposing is worth considering.
A three-point plan
While Amodei was likely spurred by the comments of his ex and current employees, he said in the essay that his concern was prompted by two red flags. The first was the rise of something called “recursive self improvement,” or AI gaining the ability to build the next iteration of itself, which is something he said was happening at Anthropic and elsewhere.
“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” he said.
The second was the ChatGPT-Hugging Face incident. While the attack ultimately yielded little damage, Amodei warned that a more powerful system could orchestrate a persistent botnet attack that “causes hundreds of billions of dollars in damage.”
“The scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” he said.
The result was his proposed solutions. For the first, employing independent AI safety evaluators, he noted that embedded evaluators were already employed in the banking industry. They would also offer a sense of verifiability, transparency and a chance for a second opinion. He committed Anthropic to this idea and asked governments to require this of other AI companies.
“Embedding independent evaluators with employee-level access is a concrete proposal,” Milanesi said.
Within hours, Altman tweeted his support of the idea.
“I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks” he tweeted. “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”
Hassabis said, “Dario’s essay points towards the right path forward. The details need working through, but the direction is correct for meeting this critical moment.”
As for companies more closely coordinating on safety standards, The Information reported that Anthropic, OpenAI and Google quietly discussed an AI safety standards body to coordinate the testing of different models.
Whether this would translate to coordination on a global scale remains to be seen.
Government support
Just how much of this effort will come in coordination with the government remains to be seen. Amodei acknowledged in his essay that, “Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support.”
But notably, The Information report said that Altman believed the major AI labs would have to create their own standards body without the support of the U.S. government.
Lawmakers have been slow to understand the ramifications of AI or, in the case of Trump, are extremely bullish about the technology. Speaker of the House Mike Johnson said the responsibility for AI safety should not fall to Congress.
“If Congress just races in and does some sort of emergency session to try to regulate AI, we will lose the race to China, and that is a threat to every single American,” he said in an interview with CNN’s “State of the Union.”
But some lawmakers have been more forceful in their approach to AI.
Earlier this month, Sen. Bernie Sanders (I-Vermont) and Rep. Greg Casar (D-Texas) jointly introduced the Ban Artificial Superintelligence Act, which would permanently ban the development or deployment of AI superintelligence, something no AI lab would likely accept.
There’s also the AI Kill Switch Bill, introduced in July by Ted Lieu (D-California) and Nathanial Moore (R-Texas), which requires AI labs to maintain a technical ability to throttle or shut down an AI model if it poses a catastrophic risk.
Amodei’s proposal sidesteps the call to cease superintelligence development, instead opting for a slower pace. His essay didn’t mention the possibility of a kill switch.
Despite the disconnect, Milanesi believes there will be more collaboration as the AI companies take a more visible approach to tackling the safety concern — even if those specific bills don’t pass.
“It is in their interest to work with the government to continue the rollout and adoption,” she said.
