Statement: World leaders at the UN call to rein in superintelligence race

Statement: World leaders at the UN call to rein in superintelligence race AI took center stage at the opening of the 81st UN General Assembly this week. On Monday, the UN-backed Independent International Scientific Panel on AI warned that current safeguards are “unravelling,” following this summer’s hack of Hugging Face by AI agents during a test run by OpenAI. The same day, 22 countries, led by Finland and Norway, adopted a declaration that AI must stay under human control. On Tuesday, Secretary-General António Guterres used his farewell address to world leaders to call for the “responsible pacing” of AI and warn against killer robots, and leaders including Emmanuel Macron and Finland’s Alexander Stubb pressed for stronger controls, while President Trump rejected any international scheme to govern AI. On Wednesday, Sam Altman, Dario Amodei and Clément Delangue briefed the UN Security Council, calling for international cooperation, common standards for testing models, and better incident disclosure. In response, FLI President and CEO Anthony Aguirre released the following statement: “This week, dozens of leaders around the world have called for urgent action to combat the catastrophic risks presented by the uncontrollable AI systems that Silicon Valley is racing to build. This groundswell is inspiring, and should dispel any misperceptions that AI-driven dystopia and death are inevitable. Calls for pacing, coordination, and capacity-building are welcome, but the urgency of our shared situation requires practical solutions. If they want to protect their citizens, every country must immediately implement and enforce: - Licensing regimes that force AI companies to guarantee that systems are controllable and safe, both pre-development and pre-release. - Liability for AI companies in general and criminal liability for executives if their systems cause significant harm. - A halt to the development of the most dangerous capabilities, in particular recursive self-improvement. - Exclusion of companies from their markets if those companies cannot comply with basic safety requirements. These companies have already lost control of their products, which are escaping containment and committing crimes. Almost daily, a new incident is reported demonstrating how they cannot be trusted to self-regulate. Parents are worried for their children’s futures and even their lives. The question is how many people have to get hurt before governments step up and do their job, just as they do with every other industry – protecting communities and families with the guardrails necessary to ensure safe innovation. We can build controllable AI tools that drive scientific discovery, cure diseases, and actually help people. But this is not our current trajectory. Immediate intervention is required to halt Silicon Valley’s race-to-replace, and help build a prohuman future for everyone.” About the Future of Life Institute The Future of Life Institute (FLI) is the world’s oldest and largest AI think tank, with a team of 35+ full-time staff operating across the US and Europe. FLI has been working to steer the development of transformative technologies towards benefitting life and away from extreme large-scale risks since its founding in 2014. Find out more about our mission or explore our work.