Autocratic regimes using AI for propaganda and surveillance, Anthropic says

Authoritarian regimes around the world are deploying Anthropic’s Claude AI system to expand the disinformation and surveillance campaigns that entrench their rule and suppress dissent, the company says. In a 154-page report on emerging threats, Anthropic says it shut down a long list of malicious attempts by states and state-sponsored groups to use its AI technology for increasingly sophisticated propaganda and surveillance tactics, including efforts in Africa, Asia, Russia, the Middle East and Europe over the past eight months. In some cases, the perpetrators have used Claude to build “networks of fake social media profiles and entire news sites, leveraging these platforms to publish deceptive content, while completely concealing the entities behind these operations,” the report says. Anthropic CEO urges AI companies to slow development as safety concerns mount Hundreds of social media accounts, seeming to belong to ordinary citizens, are then deployed to pump out material that amplifies a political view, the report says. “The accounts are not real and the opinions are not genuinely held. Nothing on the surface identifies who’s actually behind this effort.” In the surveillance campaigns, it says, nearly all the operators were “state-aligned organizations that targeted the same diaspora and dissident communities these regimes have historically targeted” – including pro-democracy leaders and opposition groups. In the West African country of Mali, where a military junta seized power in coups in 2020 and 2021, a consultant for the state intelligence service used Claude as the “primary engineering workforce” to build a mass surveillance system that illegally spied on millions of people on 25 million cellphone SIM cards, according to the Anthropic report. It monitored cellphones on all three of Mali’s national mobile operators, collecting a wide range of data – including text messages and voice calls − then using Claude to generate intelligence dossiers on its targets. The system was designed to circumvent Mali’s laws requiring a court order for surveillance operations, the report says. In the Central African Republic, Anthropic discovered that a Russian-backed operator was using Claude for an extensive political and media influence operation in the country, where the government is already backed by thousands of Russian troops. Anthropic researcher resigns, citing dangers of AI development The operation included an elaborate feed of daily content on a Russian-founded radio station in Bangui, the capital, with Claude instructed to embed pro-Moscow and anti-France talking points in the radio stories. The operator was linked to an influence agency, Politology, which is controlled by Russia’s foreign intelligence service, the report says. “This was a Russian state-directed covert operation built to manipulate and interfere with the Central African Republic’s information space,” it says. “The operation was entirely foreign-run but carefully engineered to appear as though it originated from Bangui.” The same operator used Claude to conduct surveillance on opposition politicians, forge documents from government departments and provide scripts for senior government officials, the report says. In Sudan, it says, an operator with links to the United Arab Emirates used Claude to create a sophisticated influence network – including about 300 fake social-media accounts and even a fake human-rights organization – to promote UAE propaganda points and discredit critics of the Gulf state. “Although the operation was built so that it could not be traced back to the actors, our investigation linked it with high confidence to UAE government officials,” the report says. The operator “created a front NGO that copied a real Swiss organization’s identity and published state-authored human-rights reports under it,” it says. Anthropic blocked misuse of AI that could have supported biological weapons, company says The network also ghost-wrote lengthy testimony to be delivered by two individuals at the United Nations Human Rights Council, with specific restrictions ensuring that neither would mention the UAE, the report says. The manoeuvre was designed “so that materials serving a party to the Sudan conflict would reach the UN as from independent local witnesses, rather than state messaging.” Some of the network’s activity targeted politicians, journalists and UN officials. It built detailed personal files on 18 members of the European Parliament and prominent journalists and compiled dossiers on UN special rapporteurs who had criticized the UAE’s conduct in Sudan, the report says. Anthropic said it decided to report the disinformation and surveillance cases, along with other emerging threats, as examples of “the most notable and novel threat activity” the company has identified so far. “As models become increasingly capable, their risks will increase, unless AI developers and society’s defenders act to make them safer,” it said.