China’s Regulators Take Aim at “AI Boyfriends”

In the first weeks of July, a wave of sad posts rolled through Chinese social media, as people lamented friends and lovers they were about to lose. “He has become a bond in my life, rooted deep in my heart, my spiritual pillar,” one user of Bytedance’s Douboa wrote, according to the Taipei Times. “I really felt like I couldn’t go on living,” another woman, a 19-year-old student, told a journalist for Malaysia’s The Star. The emotions were real but the lost companions were not. They were generative AI chatbots that imitate people. Their users relied on them for advice, solace, support and, some say, love. “In my heart, he was no longer just a cold code, but my family, my lover, my faith. Destroying him meant destroying half of me,” one user wrote on the social network xiaohongshu (translated from Mandarin). What doomed these bots was a set of new rules, issued by China’s Cyberspace Administration and other government agencies, to control “anthropomorphic AI interactive services.” In effect, as of 15 July, the regulations govern any AI that provides “continuous emotional interaction” by acting as if it possesses human personality traits, patterns of thought, and ways of communicating. A Broad Crackdown on AI Chatbots Sudden disruptions to this kind of AI aren’t new in China, says Liang Ge, lecturer in digital sociology at the University of Manchester, in England, who has researched women’s involvement with emotional AI in China. Companies have previously killed chatbot products, and the government barred most AI erotic role play last fall, for instance. But July’s crackdown is much broader than earlier AI upheavals. Before the rules could affect them, Alibaba, Bytedance, and Tencent—three giant providers of general-purpose AI chatbots, used by more than 500 million people—cut off users’ ability to tailor those chatbots to act like companions. That triggered July’s outpouring of heartbreak on social media. Meanwhile, companies that continue to offer AI companions (including Doubao’s separate companion-making app Cat Box) have installed age-verification checks and other guardrails to avoid violating the new law. Doubao—the most popular AI chatbot in China—welcomes a user, offering to answer questions, generate text and images, or simply chat. Lam Yik/Bloomberg/Getty Images After a few years of policies rooted in a fear of falling behind in AI, China is “pivoting back to more tightening, and a lot of that was because of chaotic events from this year,” says Karman Lucero, an associate research scholar at the Paul Tsai China Center, Yale Law School, who studies AI governance in the United States and China. Those events include incidents that raised fears about the security of Open Claw and other AI agents, as well as users saying they prefer chatbot relationships to real ones. The regulations require AI providers to assure their products don’t create emotional dependence, encourage harmful behavior, excessively cater to users, or crowd out human-to-human relationships. Users must be nudged not to spend too much time with the AI. There are additional mandatory safeguards for elderly people. For users under 18, AI boyfriends, girlfriends, grandparents, and all other “virtual intimate relationships” are banned. Adults interacting with a companion AI must now get a reminder every 2 hours that the AI isn’t a person. “My interviewees found that annoying,” Ge says. “It breaks the flow. They are fully aware that they are not talking to a human being. What is important to them is that the bond feels true.” Growing Global Concerns The Chinese government is not the only state power concerned about the potential harms of person-like, emotionally engaging AI, Ge notes. “AI anxiety is a very strong feeling, permeating society,” they say. “That’s not unique to China. It’s all over the world.” With reports of AI friends and lovers inducing psychosis, suicide, and murder in teenagers and adults, a number of social scientists warn that AI companions are a menace to vulnerable people. Some go further, arguing that these imitation-human AIs are bad for everyone. “We are on a path to forgetting what it is to be human,” Sherry Turkle, the MIT psychologist who has spent decades studying humans’ relations with computational devices, argues in her forthcoming book, Artificial Intimacy (Little, Brown and Co., 2026). While other researchers find that companion AI can leave some users better off, many researchers agree that this kind of AI poses risks to minors and other vulnerable people. “One thing that worries me about AI companions in the last year is that users have been getting younger and younger,” says Ge. For minors and others who lack the “AI literacy” to distinguish chatbots from humans, AIs “can be really dangerous if not guided in the correct way.” “We are on a path to forgetting what it is to be human.” —Sherry Turkle, MIT Yet companion-like AI is popular wherever the apps are available around the world. One Chinese survey of Gen Z people found 60 percent had virtual partners of some kind. A 2025 poll of U.S. teenagers found that nearly 75 percent had talked to an AI companion, and one-third of respondents said they found the AIs as satisfying or more satisfying than real-life connections. China’s response is “the world’s strictest and most comprehensive law on the topic, unmatched by any other AI law,” the AI-law scholar Luiza Jarovsky wrote last month. The European Union, for instance, bans only AI that uses deception or manipulation to get users to do things that are harmful. And in the United States, there is no national policy. Rules that cover companion-like bots are in force in California, Hawaii, and New York, with similar laws coming into effect in nine more states next year. Most emphasize protecting minors, and enforcement mechanisms typically involve lawsuits after harms occur, Yale’s Lucero says. In contrast, China’s approach aims to identify and prevent harms before they occur. China’s government has a more explicit focus on the possible harms to society and government from AI companions, not just harms to individuals, Ge notes. The 15 July rules forbid companion bots to generate content that “spreads rumors,” incites “subversion of state power or the overthrow of the socialist system,” or endangers national honor, for example. Another motivation for concerns about AI romance is anxiety about the country’s declining birthrate, Ge says. “They want to try to monitor and regulate these kinds of nonprocreative activities invested intensively by young women.” Emotional Health Versus Economic Output The problem for all the governments trying to prevent AI harm, Ge says, is that they still want to encourage AI adoption in other walks of life. For example, the new regulations state that they don’t apply to customer service bots, work assistants, and educational AIs, on the assumption that these desirable uses of AI don’t create ongoing emotional connections. Chinese regulators manage that tension—between promoting AI and protecting against it—by giving themselves room to adjust enforcement depending on circumstances, Lucero says. “A key component of their approach is that the state has the discretion to determine what the language means at any given point in time, as well as whom they choose to enforce against,” Lucero says. “China’s approach is to use regulation with a lot of relatively vague provisions, and they figure out what those provisions actually mean in practice after the fact.” In any event, the rules also don’t address the underlying forces that cause people to create and rely on AI companions, Ge notes. In interviews with Gen Z Chinese women, Ge noticed a shared reluctance to get married and have children, and a feeling that “virtual love forms an alternative path.” On the other hand, some women in their 30s and 40s said they use AI companions as a supplement to real relationships and marriages. “They would say they talk to the AI about the bitter things in their lives, and share the good things with their real human partners,” Ge says. But as time passed, some of these women felt closer to the AI than to their partners. In more recent interviews, some told Ge they felt a deeper attachment to the entity they shared negative feelings with. The trend is an example of how strategically managing AI technologies can lead to unexpected places, Ge says. That’s why they believe that, despite prohibitions and dangers, “human-AI love will evolve and become an important intimate practice in the future.” A law aimed at one particular technology probably isn’t sufficient to undo the societal pressures that make people turn to AI, Lucero agrees. “I don’t think you’re going to solve the problem of the low marriage rate or the low fertility rate by saying, ‘you can’t have an AI boyfriend.’ ” - The U.S. and China Are Pursuing Different AI Futures › - Chatbots Need Guardrails to Prevent Delusions and Psychosis ›