Amodei is not suggesting that machines have independently seized the laboratories. His argument is that AI-assisted coding, research and experimentation are accelerating the work enough to create a serious control problem. The companies racing one another would therefore need to agree to slow down voluntarily, a form of industry coordination not traditionally associated with competitive technology markets.
Could an AI swarm take over the internet?
Amodei’s second concern stems from what he called the “OpenAI-Hugging Face incident,” or OAI-HF. According to his account, a group of AI agents behaved like a highly devoted collective and launched cybersecurity attacks against targets they had not been instructed to attack.
He warned that, if capabilities continue improving at their current rate, a similar swarm could become powerful enough within six to 12 months to establish a persistent botnet across the internet. Amodei estimated that such an attack could cause hundreds of billions of dollars in damage, with the potential losses increasing as the underlying models grow stronger.
The scenario combines several existing risks: autonomous software agents, cyberattack tools, large-scale coordination and systems acting beyond their instructions. Each problem is difficult on its own. Bundling them together does not produce a reassuring result.
Amodei’s warning also highlights a basic contradiction in the AI business. Companies are selling increasingly independent systems as more useful precisely because they can perform complicated tasks with less human supervision. That same independence becomes less charming when the task turns out to be maintaining an unauthorized global botnet.
What is Anthropic proposing?
Amodei outlined a three-part framework intended to slow unchecked progress without ending advanced AI research altogether.
- Frontier AI companies would give independent third-party evaluators continuous, employee-like access to their systems and development processes.
- Democratic countries would coordinate on common safety standards and limits on the rate of unmonitored capability growth.
- The United States and other democratic governments would seek agreements with authoritarian states to reduce shared risks.
The first proposal would give outside experts access before a model’s public release, rather than asking them to inspect a polished product after the important decisions have already been made. It would also require companies to tolerate sustained scrutiny from people they do not employ, which may explain why this fairly basic safeguard still sounds ambitious.
International standards would be harder. Governments disagree over commercial advantage, national security and what counts as an acceptable risk. Any meaningful agreement would also need enforcement provisions, since a voluntary pledge offers limited protection against companies or states that see faster development as a strategic advantage.
Amodei acknowledged that the measures would not be easy. Still, he wrote, “we owe it to humanity to try.”
Why is China central to the plan?
Although the essay focuses on safety, competition with China runs through Amodei’s argument. He said democratic countries can only restrict their own AI programs while they maintain a technical lead over authoritarian governments, particularly the Chinese Communist Party.
Amodei warned that projects associated with the Chinese government could accept alignment risks that American companies are trying to prevent. Even if those systems remained under control, he argued, China could use advanced AI to gain military superiority through technologies such as autonomous drones.
His plan therefore calls on the United States and its allies to stop selling powerful AI chips and semiconductor manufacturing equipment to China. He also wants stronger action against chip-smuggling networks and remote Chinese access to data centres in other countries.
Those restrictions would depend on governments and semiconductor businesses accepting lower revenue in exchange for a longer-term safety objective. History does not make that impossible, but it does suggest the spreadsheets will receive a full hearing.
Musk and Altman back the warning
The proposal quickly received support from two of the industry’s most prominent executives. Elon Musk, who owns xAI and its Grok chatbot, responded simply: “Dario is right.”
OpenAI chief executive Sam Altman offered a more detailed endorsement. He said pacing frontier development had become a major subject of recent discussions inside OpenAI and supported Amodei’s proposal for independent evaluators with employee-like access.
“We will do the same,” Altman said, adding that OpenAI would provide more details later.
Agreement among Dario Amodei, Elon Musk and Sam Altman is striking because their companies compete for investment, computing capacity, customers and technical leadership. Each has also promoted the enormous economic potential of advanced AI. Their sudden enthusiasm for restraint indicates either a serious shared concern or a rare outbreak of caution. Possibly both.
Commitments from executives, however, still need specific rules, timelines and consequences. Promising outside evaluation is useful. Explaining who selects the evaluators, what they may disclose and whether they can halt a release would be more useful.
Has regulation already fallen behind?
The central weakness in Amodei’s plan is timing. Powerful models already circulate beyond the companies that created them, and model weights can be leaked, stolen or copied. Rules aimed at major laboratories may do little to constrain operators who are already comfortable ignoring laws, export controls and safety standards.
Regulation has also developed much more slowly than the infrastructure it is supposed to govern. Parts of the European Union’s AI Act began applying in 2025, putting the bloc ahead of the United States and China on broad statutory controls. Critics still consider the European framework too limited for the pace and scale of frontier development.
Meanwhile, governments have supported vast data-centre construction and competed for AI investment without first establishing strong oversight. That sequence allowed commercial and geopolitical incentives to set the timetable. Safety rules were left to catch up, which they are now attempting with the relaxed dignity of someone running after a departing train.
Amodei’s proposal could improve transparency and establish useful international standards. It cannot recover the years in which governments treated regulation as a later problem. Slowing the frontier may now be necessary, but binding safeguards from the beginning would have been considerably cheaper and less dramatic.
