About two years ago, education entrepreneur Boris But had what he calls a “penny-drop moment” – that, like countless others in China, his company needed to embrace AI before it was made obsolete by it.
His clients, well-off parents looking to send their children to study overseas, had begun arriving in his office reeling off information and opinions gleaned not from rival firms, but from chatbots.
In response, he started developing his own AI study tool, concluding: “If you can’t beat them, join them.”
The pace and quality of AI development in China is second only to the United States, with the two superpowers locked in an intensifying rivalry as the stakes rise with each new model’s capabilities.
Analysts generally agree that frontier, or cutting-edge, US models are still some way ahead of their Chinese counterparts.
But over the last few weeks the mood music from Silicon Valley has been decidedly sombre, as insider after insider has warned of the potential catastrophes that could result from “loss of control” of the most advanced AIs.
And last Saturday Anthropic chief executive Dario Amodei pitched a “worldwide pacing of the frontier” – something he said would “require cooperation with China”.
“China is definitely not receptive” to a slowdown on US firms’ terms, though, said Lian Jye Su from technology research group Omdia.
For the government, AI is a “core sovereignty capability”, and Beijing will not want to jeopardise the progress made in closing the gap with US frontier models.
“It sees these calls [for a slowdown] as a bid to lock in US advantage,” said Su.
Amodei made this explicit, mentioning China specifically when arguing it was key “to keep democracies’ AI lead over autocracies as large as possible”.
He advocated for measures including continued export controls to China on advanced chips and machinery – the existence of which some argue already amounts to a forced unilateral “pacing”.
China’s foreign ministry responded to Amodei’s piece by saying “fearmongering, confrontation and vicious competition will only disrupt the process of global AI governance”.
However, several analysts told the Guardian it would be a mistake to assume this suggested a rejection of the broader concerns raised by the essay.
This week China released the third iteration of its AI safety governance framework – a document that functions as a “mapping of the AI risk landscape and roughly what one should do about it”, said Gabriel Wagner, a researcher at Concordia, a Beijing-based AI safety consultancy.
Included in the updates are the risks posed by agents, the cybersecurity threats from frontier models and the issue of recursive self-improvement – an as-yet hypothetical scenario where AI becomes advanced enough to improve itself.
“In many ways, I think there is more common ground than people might assume between what Amodei is saying and what the Chinese government is saying,” said Wagner.
That contrasts with US president Donald Trump, who this week called fears AI could wipe out humanity a “hoax”.
Trump is set to meet counterpart Xi Jinping in Washington next week, with AI expected to make the agenda, though analysts are sceptical a major breakthrough will happen.
The ruling Communist party, finely attuned to risks to its political control, has been proactive in ensuring the new technology has developed within boundaries set by the state.
Centralised authority over critical infrastructure and commercial activity means China is generally better placed than the United States “to address any AI disaster”, said Omdia’s Su.
However, Chinese companies do less voluntary monitoring and evaluation work than US ones, said Leia Wang from the Carnegie Endowment for International Peace (CEIP).
Neither government seems prepared for a loss-of-control incident like those outlined over the last few weeks, she said.
China’s regulation, though, is “iterative … they are quick to adapt and release new measures”, she added.
China has had an algorithm registry since 2022, but warnings on agents released this year suggest a switch in focus “from controlling what AI says to controlling what it does”, per Concordia AI.
Beijing also issued national regulations this year aimed at curbing emotional dependency on companion AIs.
At the same time, AI is being actively encouraged across the economy to promote growth, enshrining this goal in its latest Five Year Plan.
Local governments are offering funding and low rents to companies to help achieve this – entrepreneur But was recently invited to pitch his company, Hampshire Education, to a small city three hours from Shanghai set on establishing a reputation as a tech hub.
Factories across the country’s huge manufacturing base are being encouraged to convert their processes into the AI era.
“Safety and controllability are certainly major concerns, and they are actively discussed,” said a spokesperson for Black Lake Technologies, an industrial agent platform that says it has worked with 40,000 factories globally.
“But in manufacturing, these concerns tend to be much more concrete and operational … what companies care most about is whether the AI is reliable.”
Daily life in China is already highly digitised, with traditionally less discourse around data privacy helping with widespread adoption.
For example, digital health service A-fu, which features AI chatbot versions of prominent doctors trained on real clinical studies, recently hit 150 million users.
“Most people treat AI as an already-useful everyday tool, not an abstract threat, so the practical attitude is to live with it,” said Ni Tao, founding editor of tech website yangtzeer.com.
“There is real anxiety about jobs and misinformation, but it coexists with a strong belief that China must master AI rather than slow down.”
Leave a Reply