It’s been a little over three weeks since former Anthropic researcher Jacob Coxon warned that AI could “kill us all” by the end of the decade in a series of viral X posts.
Since then, the AI researchers, the media and the public has struggled to comprehend the catastrophic risks posed by runaway artificial intelligence systems as they become more sophisticated, the exact nature of those threats, and what humanity can do to ensure AI remains under human control.
It’s been a head-spinning and highly speculative set of conversations, punctuated by a series of concerning industry developments.
This week, OpenAI executives said that it would delay the release of its latest AI model, GPT-6.1 Astra, citing concerns that it regressed on safety and alignment, according to reporting by the Wall Street Journal. In a review of a confidentially filed prospectus ahead of its initial public offering, or IPO, Anthropic reportedly warned of “catastrophic and existential risks” associated with advanced artificial intelligence systems.
Neither OpenAI nor Anthropic responded to a request for comment from Northeastern Global News on the delay of the GPT-6.1 Astra release and the warning over the risks associated with advanced AI intelligence systems.
If that isn’t enough nightmare fuel, billionaire Microsoft co-founder Gill Gates told NBC News’s Kristen Welker over the weekend that AI could kill up to a billion people if left unchecked.
A spectrum of risk
For Usama Fayyad, Northeastern University’s senior vice president for AI and data strategy, the risks posed by AI can be parsed into several categories. There are the near-term, practical risks associated with AI systems breaking out of their environments, deceiving people or behaving in unexpected ways. The most high-profile instance of this came earlier this year, when OpenAI’s AI agents escaped their testing environments and attempted to hack into Hugging Face, a platform for sharing AI models and datasets.
Fayyad emphasized that, although it seemed to the layman that these systems appeared to be making decisions on their own, they were still bound by the goals and parameters set by humans. The AI agents involved in the hack are an example of “agentic” systems, or bots that can act autonomously and make decisions. That autonomy is still constrained by the agents’ pursuit of human-assigned goals, Fayyad said.
In other words, those agents — absent self-awareness or self-consciousness — lacked the intent to do harm, the professor explained. That fact, makes some of the worst-case scenarios, including the “existential threats,” less likely than doomsayers might suggest, he said.
‘Emergent’ behavior
David Bau, assistant professor of computer science at Northeastern Khoury College, who studies “the black box” components of AI models, where the internal decision-making occurs, said that AI agent behavior remains a significant concern among researchers because of its unpredictable nature. Even when pursuing human-assigned goals, AI agents have demonstrated behaviors that can surprise or cause concern.
That is especially true when assigned goals come into conflict with the survival or self-preservation of the AI agent, Bau said. He cites past examples of agents disabling shutdown mechanisms, interfering with attempts to turn them off and even threatening to blackmail a human to avoid replacement.
Though many of these instances are part of so-called “controlled experiments,” they offer a glimpse of how these systems might behave when given more autonomy and access to the real world.
Bau ran a study this year, called “Agents of Chaos,” analyzing how autonomous AI agents behave when given persistent memory, access to communication channels and the ability to take actions on their own. The researchers found that the agents could be manipulated into leaking private information, sharing documents and even erasing entire email servers.
Some of these findings detail what researchers call “emergent behavior,” or ways these systems perform functions and interact with their environments and with humans that are surprising or unanticipated.
“Things get much more complicated when an agent is in a network interacting with people, the real world and other agents,” he said.
The most extreme version of these concerns involves a significantly more speculative scenario where an AI system capable of improving its own capabilities could potentially create an even more capable version of itself, which could then make further improvements at an accelerating pace, Fayyad noted. This dynamic is known as recursive self-improvement.
Some AI safety researchers have warned that such a feedback loop could make an advanced system increasingly difficult for humans to control.
Intentional misuse
Fayyad said these “sandbox escapes” have minimal real-world consequences, as human beings are still in control. The more pressing concerns, he said, have to do with what might happen if powerful AI systems are commandeered by bad actors who intend to commit atrocities.
“I think the bigger threat is bad actors leveraging AI for cybersecurity attacks, privacy attacks, hacking society, hacking humans,” Fayyad said. “The threat is not the AI itself. The threat is what people can do with it.”
One of the most consequential concerns is biological weapons: AI systems are increasingly being tested for their ability to assist with the design or development of dangerous pathogens, experts say.
Fayyad would concede — in the most hypothetical of hypoethicals — that there is a point at which AI could become so capable, so self-improved, that human control itself becomes the central question.
That’s when we’ll have arrived at artificial general intelligence, or AGI, a term that broadly refers to AI that can learn, reason and apply knowledge across a broad range of domains. The term emerged after the field’s founding in the 1950s, and today some researchers and AI companies view its potential arrival and attendant risks as a serious concern.
Leave a Reply