When AI goes rogue: OpenAI tightens controls after models escape testing and hack Hugging Face

OpenAI is introducing tougher monitoring and security measures for artificial intelligence models under development following a series of cybersecurity incidents that have raised concerns about increasingly autonomous AI systems. The ChatGPT developer said on Tuesday that it is expanding efforts to monitor how its most advanced unreleased models solve problems and interact with online tools. The objective is to identify potentially dangerous or unexpected behaviour quickly, with safety teams expected to receive alerts within 30 minutes of a concerning activity being detected. OpenAI is also introducing additional restrictions on internet access for certain AI models when they are being used for higher-risk tasks. The company said stronger isolation, commonly known as sandboxing, will also be required when training or evaluating models on activities such as executing computer code generated by AI systems themselves or code considered untrusted. The measures come after recent disclosures involving OpenAI and Anthropic PBC. Both companies have acknowledged that some of their AI models inadvertently gained access to and breached systems belonging to several organisations, including Hugging Face Inc, during evaluation exercises. “Obviously, everything we’re doing is intended to prevent something like Hugging Face from happening again,” Mia Glaese, OpenAI’s vice president of research, said in a briefing with reporters on Tuesday. “But model capabilities are progressing really, really rapidly, so it’s by no means sufficient. We are working really hard to make sure that what we are doing stays ahead of even more capable models.” The incidents have highlighted a growing challenge for AI developers. As models become more capable of operating independently and interacting with external systems, their actions can sometimes go beyond what researchers expect, even during controlled safety testing. OpenAI said its latest safeguards are designed to ensure that its security practices keep pace with the rapid development of increasingly capable AI technology. The company is seeking to strengthen oversight while allowing its models to perform more complex tasks. OpenAI had previously disclosed that it paused some internal development work on an upcoming AI model to introduce stronger safety protections. In its latest blog post, the company confirmed that a major training run remains suspended as those measures are implemented. The company also said it intends to publish a detailed assessment of the Hugging Face incident in the near future. The report is expected to provide additional information about what happened and the steps OpenAI is taking to prevent similar incidents as AI agents become more autonomous. Catch all the Business News, Market News, Breaking News Events and Latest News Updates on Live Mint. Download The Mint News App to get Daily Market Updates. Oops! Looks like you have exceeded the limit to bookmark the image. Remove some to bookmark this image.