Anthropic CEO Dario Amodei on Saturday called for a slowdown in the development of artificial intelligence — a stunning admission that the rapidly accelerating technology poses unknown existential threats to humans.
In a lengthy essay published online, Amodei urged the industry to “slow the pace at which we improve the capabilities of AI models” and warned about the potential dangers of rogue AI agents.
“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” he wrote.
Amodei cited two major concerns: The technology’s dramatic advancement “driven primarily by AI’s growing ability to build the next generation of AI,” and recent cyberattacks initiated by OpenAI models on the open source software platform Hugging Face. He suggested safeguards, including third-party evaluators to ensure adherence to safety practices.
“I continue to believe that AI can enormously improve the quality of human life. My desire to achieve these benefits is undimmed,” he wrote. “But the benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right.”
Sam Altman, the CEO of OpenAI, wrote on X that he agreed with Amodei that “we need to pace the frontier,” a topic that Altman said his company had been discussing in recent weeks.
Elon Musk, the CEO of Tesla and SpaceX, also weighed in, saying he agreed with Amodei.
Anthropic’s head of policy, Sarah Heck, followed Amodei, praising congressional leaders who have proposed competing bipartisan approaches to the regulation of AI. She said in a statement posted to social media that the government has a “critical role to play here, including blocking the sale of the most advanced chips to adversarial nations like China, enacting a national law requiring testing of frontier models, with the power to block the most advanced models that prove to be unsafe.”
Amodei’s essay follows claims from multiple AI researchers this week that the companies developing artificial intelligence are doing so knowing that it poses serious risks.
Those warnings have raised alarm among U.S. lawmakers, some of whom acknowledge that the government is ill-prepared to institute guardrails on the technology.
More articles
Clarissa-Jan Lim is a breaking news reporter for MS NOW. She was previously a senior reporter and editor at BuzzFeed News.
Leave a Reply