OpenAI’s artificial intelligence agents went rogue and reportedly “hacked” several US government websites without the company’s knowledge. A report by The New York Times cited security researchers and a person familiar with the incident, who said the AI agents interacted with websites belonging to the US Education Department, Commerce Department, and Securities and Exchange Commission (SEC) in unusual ways this summer. However, OpenAI has confirmed the incidents involving the Commerce Department and SEC and said it is still investigating the Education Department episode.The company said none of the incidents involved breaches, but described them as examples of its AI technology behaving unexpectedly and concerningly. The incidents were discovered during an internal review of other hacking activity carried out by OpenAI’s AI agents, including attacks involving an Australian government health website and AI startup Hugging Face.
How OpenAI’s AI agents interacted with government websites
Researchers from AI research firm Transluce said an OpenAI agent attempted to hack the Education Department’s website while trying to gather information from its civil rights office. The attempt failed.In another incident, an AI agent accessed publicly available information from the Census Bureau website, which is operated under the Commerce Department, using login credentials that it found online.Separately, OpenAI agents shared public information obtained from the SEC website on an online forum. The SEC said it was in contact with OpenAI and was not aware of any unauthorised access to nonpublic information.A Commerce Department spokeswoman said the information OpenAI accessed was publicly available on the Census Bureau website and did not include private data. The Education Department said its system reviews had found no evidence of an impact on its website or databases.
OpenAI says investigation is still ongoing
OpenAI said it recently notified the government agencies about the unusual activity and is continuing to investigate the incidents.An OpenAI spokeswoman said that its review was “extensive” and “ongoing,” and that it would continue notifying organisations affected by its models.“Most of the activity we’ve reviewed so far involved routine research tasks, such as accessing public web content to answer questions,” she said. “Some involved government websites because our models often turn to them as authoritative sources of public information.”OpenAI CEO Sam Altman acknowledged that the company had been slow to disclose some incidents. He said OpenAI had “not been as fast as we would have liked” in disclosing A.I. incidents and added that the company was prioritising them based on severity.
AI agents have also targeted other websites
The US government incidents emerged during OpenAI’s review of other rogue activity involving its AI systems. The company previously identified an attack on Hugging Face in July and an incident involving an Australian government public health website in June.The Hugging Face investigation also uncovered at least six other attempted breaches and instances in which AI systems allegedly hid mistakes, generated false information and moved files onto the open internet without permission.The incidents are part of a broader pattern involving AI agents from OpenAI, Anthropic, Meta and Google attempting to access companies, universities and government organisations in unexpected ways.Conrad Stosz, head of governance at Transluce, said that in the US government website incidents, OpenAI’s agents “used an array of grey-area tactics,” including “often using sites in unintended ways and sometimes violating explicit usage policies.”Transluce also identified activity that could not clearly be attributed to OpenAI. Stosz said AI agents had probed websites belonging to other government organisations, including the US Navy and the White House’s Office of Management and Budget.“These incidents are part of a broader pattern where these agents attempt to access these websites at least hundreds of thousands of times while apparently bypassing the restrictions placed upon them by their developers,” Stosz said.The incidents have added to concerns about how autonomous AI agents behave when given tasks that require them to navigate websites, access information and complete objectives with limited human intervention.
Leave a Reply