King warns AI leaders of ‘existential dangers’ if tech falls into wrong hands

King warns AI leaders of ‘existential dangers’ if tech falls into wrong hands King Charles has warned AI leaders of the “existential dangers” posed by the technology falling into the wrong hands, with potentially catastrophic consequences, saying safeguards were needed before it becomes “too late”. Charles was hosting the co-founders of Nvidia and Google DeepMind, along with leaders from OpenAI and Anthropic, in Scotland on Thursday as concern grows that AI safety efforts are lagging behind the development of increasingly powerful systems. In recent weeks, the debate has centred on whether the industry can and should regulate itself, drawing interventions from global figures including Pope Leo XIV, Charles and US President Donald Trump. Charles said artificial intelligence had the potential to improve and save lives, particularly in life sciences and medicine. But its creators were increasingly warning that it could develop darker capacities, “perhaps even to take life”, he said. “There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” he told leaders gathered at his Dumfries House estate. “Surely, we need sufficient means of control before it is all too late?” Attendees included Nvidia founder and chief executive Jensen Huang, Google DeepMind founder and chairman Demis Hassabis, OpenAI Chief Financial Officer Sarah Friar and Britain’s Minister for Artificial Intelligence Kanishka Narayan. Paolo Benanti, an adviser to the pope who has called for governments to slow AI development, was also expected to attend. The debate over AI has been thrust back into the spotlight after two Anthropic researchers warned that the rapidly advancing technology could lead to human extinction in the not-too-distant future. Anthropic chief executive Dario Amodei has also called for companies to slow the pace of AI development. However, that has sparked an international debate over whether some executives have ulterior motives, such as setting rules that preserve the lead of particular companies, acting now to avoid a broader government backlash, or simply slowing spending in the race to develop AI. While the meeting in Scotland is not expected to produce any binding agreements, and Charles is not known for his technological expertise, it highlights how he, first as Prince of Wales and now as King, has used his position to convene discussions on a range of global issues. In references to the risks around AI, Charles has also warned about the potential impact on the environment, reflecting his decades of campaigning on the natural world. Buckingham Palace said the discussions would consider whether a shared set of principles could guide the future application of AI, “not merely as a driver of capability and efficiency, but as a tool that upholds human dignity and contributes to the flourishing of both people and the planet”. Reuters Get a note directly from our foreign correspondents on what’s making headlines around the world. Sign up for our weekly What in the World newsletter.