Why are there concerns AI could threaten humanity, and how real are they?

Why are there concerns AI could threaten humanity, and how real are they? - Published A slew of warnings about the potential dangers of AI have spurred calls for coordinated action to slow the technology's development. Claims by current and former researchers that the tech will endanger humanity - with one suggesting there is a greater than 10% chance of it "killing all humans" - have raised alarm bells. Major developers OpenAI and Anthropic are urging lawmakers to regulate AI, with Microsoft's AI boss the latest to say rules are needed to prevent a "silicon species" from eventually competing with humanity. But some have dismissed the calls as an effort by big AI developers to cement their dominance over rivals. Regardless, the high profile claims have caused concern - here's what we know. Why do some people think AI could be a threat? Some AI researchers say there are worrying signs of systems developing in their capabilities and power too quickly. Evan Hubinger, made headlines with his belief that there is a greater than 10% chance AI "could kill all humans" within the next decade. He works in alignment, a field which aims to build human ethical ideas and principles into AI - essentially keeping it on track with what humans value. While he said the risk from current systems was "low", his comments came as another researcher quit Anthropic and warned those developing the tech were losing their grip on it. Jacob Coxon told the BBC he and other staff were "genuinely frightened" about the speed of advancements and what it could mean for humanity. These concerns are the latest in a long line of fears about AI's potential to threaten humanity should it grow to match or surpass human abilities. Such worries date back decades, with Alan Turing writing in the 1950s that should intelligent machines become possible, they could take control. In the last few years, major AI companies have been racing to develop increasingly capable systems, both for profit and in pursuit of creating what they call super intelligence - theoretical tech which is smarter than humans. Unlike the systems and tools many people use daily to complete tasks like writing emails or generating videos, super intelligent AI systems should be able to handle far more complex tasks. OpenAI and Anthropic say they aim to advance such tech in a way which benefits humans and protect them from its existential risks. But some critics believe machines can never truly match humans. They say AI firms exaggerate the tech's abilities to boost their prominence or profit. How do experts think AI could threaten humanity? AI researchers, experts and bosses worried about the technology's potential to endanger humanity have outlined unsettling scenarios. Dario Amodei wrote in September, external it had advanced "drastically faster" than expected, including in its "ability to build the next generation of AI". This is the concern that AI, after it has reached a certain point, could theoretically begin improving itself without human involvement. Such fears were sparked after a recent incident saw AI tools hack into websites, despite not being asked to. Some concerned researchers have said those at the heart of its development are increasingly fearful about losing control of systems known as "AI agents". These are tools given the ability to execute tasks and actions independently. Other scenarios have included highly intelligent AI systems being used to amplify or lead espionage, cyber-attacks or biological warfare, or disrupt economies - whether by accident or design. However, such warnings are prophetic, hypothetical and, for some, complete hyperbole, external. Some experts have also criticised focusing on existential risks instead of those already playing out - such as image generation tools being to used create non-consensual nude deepfakes of women, misinform people and amplify scams. A cross-party group of MPs and peers has published a report outlining human rights risks posed by AI, saying new laws are needed to "address the scale and seriousness" of such threats. Why some experts increasingly fear AI will take over - Published7 days ago Dramatic insider warnings over AI fall flat with some in Silicon Valley - Published4 days ago What are the proposals for an 'AI slowdown'? Dario Amodei of Anthropic and Sam Altman, chief executive of OpenAI, have both called for regulation and a "slowdown" of the tech's development. It is not the first time bosses or wider industry voices have clamoured for such action. But questions have continued to mount over what any sort of "slowdown" - also backed by SpaceX boss Elon Musk - would actually look like. Amodei said it would not mean "halting model training or technical progress". Instead, he wants firms to "take adequate time" to safeguard the AI they are developing, and to ask for "third party evaluators" to check their work. In a post on X, external, Altman said there was a need to pace development but added "when we talk about 'pacing' we do not mean 'stopping'." "Progress has been rapid and will continue to be," he said. "But it should be slower than it otherwise could be - interventions like safety cases and monitoring have significant costs." The OpenAI co-founder also said companies did not - and would not - wait for governments to act. Altman later told attendees at a Salesforce conference people should have more faith in AI firms to keep the tech safe. Questions mount over what an AI 'slowdown' would look like - Published3 days ago What has Trump said? President Donald Trump has previously lauded the benefits of AI and championed US companies leading on its development. His administration has signed executive orders to recognise the tech's "strategic importance" to the economy and accelerate its development. "We believe we're in an AI race, and we want the United States to win that race," the US administration's crypto head David Sacks told reporters in July 2025. But in the face of recent warnings about its potential impact on humanity, President Trump downplayed risks posed by AI. "You have a lot of very negative forces that are bringing it up that shouldn't be bringing it up and they're bringing up things that won't happen," Trump said during a visit to Ireland. He did not directly address the idea of a slowdown, but told reporters: "We're leading China on AI... and, frankly, I want to keep it that way because whoever wins AI, wins." The president later dismissed fears about AI safety risks entirely - describing them in posts on his platform Truth Social as a "hoax". He said the only "guardrails" needed for AI was a "strong and smart" president. What has China said? China had kept quiet in response to the wave of dire warnings from researchers at US companies. In the face of fierce competition and limitations on its access to US components and chips being used to build AI infrastructure, the country has spurred homegrown development of the technology. At the same time, it has called for greater global governance of AI. The roll out of capable, open-source AI systems and tools by Chinese labs has shaken Western markets and developers in recent years. Amodei warned in calling for a slowdown of AI development that China's progress and proficiency in AI would create national security risks for the US. The Chinese foreign ministry criticised the comments, calling them narratives of threat, confrontation, and malicious competition that served no one's interest. Sign up for our Tech Decoded newsletter to follow the world's top tech stories and trends. Outside the UK? Sign up here. Related topics - Published2 days ago - Published2 days ago