AI-generated intelligence nearly triggered US operation against Chinese ship: Reports

A false intelligence report produced with the help of artificial intelligence (AI) reportedly came close to triggering a US military operation against a Chinese vessel in West Asia during the war with Iran. The intelligence report, circulated across the US military this spring, claimed that the Chinese ship was carrying components linked to a nuclear weapons programme. The assessment immediately raised alarm within the US military and prompted preparations to intercept the vessel, CNN reported. Four sources familiar with the episode told CNN that the US military moved towards an operation against the Chinese vessel. Two sources said armed US military personnel were preparing to board the ship, while two others said military aircraft were already in the air. However, shortly before the planned operation, officials examined the intelligence more closely and discovered that the report had been prepared with the assistance of an AI system. According to the sources, a chatbot used by an analyst at US Special Operations Command had inaccurately identified the material being transported by the ship. CNN said it was unable to determine what the cargo actually was. One source described the intelligence report as ‘entirely false’. The same source said the incident ‘almost started a war’, warning that any US military operation against a Chinese vessel could have risked escalating into an armed conflict between the two countries. According to CNN, the analyst had queried a chatbot about intelligence reporting concerning the ship’s manifest. The original reporting had come from US Special Operations Command Pacific, based in Hawaii. It remained unclear whether the chatbot was a commercially available system or a product developed by the US government. The chatbot reportedly combined open-source intelligence with secret signals intelligence contained in government holdings before reaching the erroneous conclusion about the cargo. The analyst then used AI for a second time to package the findings into a standard intelligence report. That report was subsequently disseminated through military channels, CNN reported, citing sources. The episode comes as the US military and intelligence community accelerate the use of AI across a wide range of functions, including analysing large volumes of intelligence, selecting targets for strikes, moving military assets, and managing budgeting, logistics and supply chains. The incident has highlighted concerns about the use of AI-generated information in military targeting, particularly when decisions are being made during an ongoing conflict in West Asia. US officials have argued that rapidly integrating AI is necessary to help the military make battlefield decisions faster and avoid falling behind adversaries that could potentially use similar technologies. In January, US Secretary of War Pete Hegseth released the Pentagon’s ‘Artificial Intelligence Acceleration Strategy’, which seeks to speed up the military’s adoption of AI. ‘We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI’, Hegseth said in a speech announcing the strategy. The strategy also calls for expanding access to AI across the US Department of War. A memo announcing the initiative said its objective was to ‘democratise AI experimentation and transformation across the Department by putting America’s world-leading AI models directly in the hands of our three million civilian and military personnel, at all classification levels’. However, multiple US officials familiar with the situation told CNN that AI adoption remains decentralised, with different parts of the government using different systems, orders and safety standards. According to the report, there is no single set of standards governing how information generated by these systems is verified. The range of AI systems being deployed across the military and intelligence community also means their reliability and functionality can vary. The incident has raised concerns among intelligence officials about the possibility of AI-generated errors being incorporated into formal intelligence products and subsequently relied upon by military decision-makers. One source familiar with current military policies told CNN: ‘AI in targeting is definitely something that is ramping up, and there is no real guidance for how having a human in the loop will prevent civilian casualties or fratricide’. CNN reported that the hallucination involved in the Chinese ship incident was not necessarily an isolated occurrence within the intelligence community since AI tools began becoming more widely used across the US government, citing one of its sources. Several sources also said some older intelligence officials who otherwise support the use of AI are concerned that the technology is creating pressure on analysts to produce and distribute intelligence more quickly, potentially increasing the likelihood of mistakes. Young analysts, according to the sources, are more accustomed to using AI tools and may be more likely to trust their output without sufficient scrutiny. ‘AI allows you to get to a bad idea faster’, one source told CNN. The Chinese ship episode points at the potential consequences of inaccurate AI-generated intelligence being incorporated into established military reporting systems, particularly when such information is used during an active conflict and could influence operational decisions. (with ANI inputs) Stay informed on all the latest news, real-time breaking news updates, and follow all the important headlines in india news and world news on Zee News.