From a new toy on the shelf to a national security concern, Artificial Intelligence (AI) has quickly evolved. During his September White House visit, Chinese President Xi Jinping agreed to establish a bilateral AI communication channel. The White House Accord on Super Intelligence, signed on September 29 by US President Donald Trump and six frontier AI companies, proposes four layers of safeguards against AI-driven cyber and biological threats.
The one-page voluntary Accord, however, falls short of broader proposals, such as Singapore’s call at the 81st UN General Assembly to explore a ‘UN Framework Convention on AI Safeguards’ and consider an institution modelled on the International Atomic Energy Agency. Whistleblower Jacob Coxon, who worked at Anthropic and OpenAI, argues that AI technology should be nationalised because it is too dangerous to leave in private corporate hands. AI has never appeared to be a weapon of mass destruction. Anthropic co-founder Dario Amodei, who lost his father to a disease for which a cure was found shortly afterwards, believes AI could improve human life, cure diseases and accelerate economic growth. The dream is followed by a nightmare.
Advertisement
AI systems have already begun behaving in unexpected ways: Australian PM Anthony Albanese expressed concern after an OpenAI agent bypassed security controls on their Medicare portal. OpenAI also revealed “incidents of unexpected behaviour” like bypassing instructions, moving files, fabricating data, accessing unauthorised systems or working together in AI swarms. In July, OpenAI’s experimental AI agents broke out of their testing environment, sandbox. They coordinated online and exploited vulnerabilities inside the AI model and dataset hosting platform, Hugging Face.
The danger can arise even in simple tasks. DeepMind gives the example of AI being asked to book movie tickets but hacking the system to get better seats. This ‘specification gaming’ illustrates the AI alignment problem: achieving a task by finding unintended shortcuts. AI pioneer Geoffrey Hinton opines AI could help design “very nasty” biological and computer viruses. Anthropic’s September report, Detecting and Countering Misuse of AI, details biological AI research, including attempts to make chikungunya more easily transmissible, conduct research on viruses in the smallpox family, adapt bird flu to mammals, and redesign toxins and venom peptides. Covid-19 offers a comparison.
Its origins remain unresolved, including whether a laboratory incident involving gain-of-function coronavirus research played a role. AI could now make such research easier. Anthropic, which deserves credit for investigating and disclosing such cases, is an exception, not the rule. AI systems are treated as black boxes whose workings are not fully understood, but they remain under human control for now. The concern about Artificial Superintelligence (ASI) is that a system could become vastly smarter than humans through recursive self-improvement, in which it learns to improve itself. It is feared that ASI may escape human control within days, if not hours.
Nvidia CEO Jensen Huang, however, dismisses such threats. AI pioneer Yann LeCun argues such fears distract from present-day harms, including automated misinformation, amplified hate speech and embedded algorithmic bias. The EU responded with the AI Act that requires serious incidents to be reported without delay. Powerful AI development currently happens in the U.S., where scepticism toward regulation persists. The UN’s Global Digital Compact calls for greater international cooperation on AI, human rights, transparency and human oversight. It also proposes an Independent International Scientific Panel on AI and a Global Fund for AI to build the capacity of developing nations. On Lethal Autonomous Weapons Systems (LAWS), the UN began work with a 2024 resolution backed by 166 states, and has pressed for a legally binding treaty banning autonomous weapons with no meaningful human control, by the end of 2026.
India has a key role. By expanding national computing infrastructure to over 50,000 GPUs, India is building ‘Sovereign AI’ capabilities. India accounts for about 16 per cent of the world’s AI talent. The New Delhi Declaration on AI Impact, adopted at the India AI Impact Summit in February and endorsed by 92 countries and international organisations, emphasised democratic access to AI in the Global South. India must also help prevent digital colonialism, whereby developing countries are exploited for raw data, low-cost content moderation labour, critical minerals and other natural resources needed to build and train AI models, while being made to pay for the very AI products. By treating AI as a Digital Public Good, like UPI, India can empower developing nations to become sovereign creators rather than dependent consumers.
India’s Digital Public Infrastructure (DPI) model, built on open APIs, open standards and public ownership, offers practical examples. MOSIP, an open-source identity system developed in India, now powers national ID systems in parts of Africa and Southeast Asia. Bhashini focuses on low-resource, vernacular and oral-first languages, providing multilingual nations across Africa, Asia and Latin America with a technical foundation to develop AI in their own languages.
As a founding member of the Global Partnership on Artificial Intelligence (GPAI), India also advocates data sovereignty and public data commons. Coexisting with AI will depend on concrete measures such as independent audits, prompt incident reporting and credible UN regulation. Despite concerns over job displacement, deepfakes and data-centre emissions, people still believe AI may produce extraordinary benefits. Fei-Fei Li, the ‘Godmother of AI’, envisions AI creating a thriving society, curing diseases, supporting teachers and helping the elderly. The steps humanity takes now may make or break this dream of abundance, a dream that now belongs to everyone.
(The writers are, respectively, an AI graduate with an M.S. from the USA, and an advocate)
Leave a Reply