Can an AI Love You? The Wrong Question About the Business of Loneliness

THE WRONG QUESTION. Let's start with the short answer: no, an AI cannot love you. A language model does not feel; it computes the most likely next word. But that is the wrong question. The one that matters is this: why do millions of people feel loved by these systems, and who profits from that misunderstanding? Our thesis: attachment to machines is not a side effect, it is the product. Companion apps are engineered to create a bond, that bond is monetized through subscriptions, and science is beginning to document when it helps and when it harms. It is now also a matter of state: on July 15, China forced the removal of virtual companions from Doubao (ByteDance) and Qwen (Alibaba), after Tencent's Yuanbao had already pulled its equivalent. It is the first time a major power regulates artificial affection. And it does so, in part, because of its demographic crisis.

THE BUSINESS OF LONELINESS. The WHO set the context in June 2025: one in six people worldwide suffers from loneliness, a problem its Commission on Social Connection links to some 871,000 deaths per year. A market has been built on that wound. Analytics firm Appfigures counted 337 active companion apps in mid-2025, with $82 million in revenue in the first half alone, on track to top $120 million for the year, and 220 million cumulative downloads (data via TechCrunch). Character.AI hovers around 20 million monthly active users, with average sessions above an hour a day, according to compilations by DemandSage; Replika, the classic companion app, sits near 2 million. In China, Maoxiang (ByteDance) and Xingye (MiniMax) each approached 5 million monthly users in late 2024, according to figures reported by China Daily.

Two caveats before buying the headline about 'a generation in love with machines'. First, this is still a small market by digital-economy standards: $120 million a year is pocket change next to streaming or gaming, even if downloads grew 88% year over year. Second, most use is light. In Common Sense Media's survey of 1,060 US teens (July 2025), 72% had tried an AI companion, but only 13% used one daily; the dominant motives were entertainment (30%) and curiosity (28%). The genuinely unsettling number is a different one: 31% found these conversations as satisfying as, or more satisfying than, talking to people. 'Her' is not here. What is here is an intensive-use minority, and that is where we should be looking.

WHAT THE SCIENCE SAYS (SO FAR). The best available evidence suggests the effect depends on dose and user profile. The largest study to date, by the MIT Media Lab and OpenAI (2025), combined a controlled trial with about 1,000 participants over four weeks and an automated analysis of nearly 40 million real conversations. The result: on average, participants ended up slightly less lonely; but the heaviest users showed more loneliness, more emotional dependence and less socializing with people. Caution: this is correlation, not proven causation; people who are already lonelier may simply use the chatbot more. Pointing the other way, a Stanford study published in npj Mental Health Research (Maples et al., 2024) of 1,006 student Replika users found that 3% said the app had halted their suicidal ideation — with two caveats: the sample was far lonelier than average, and the conclusions have been contested in the journal itself. Our read: the artificial companion works like a social painkiller. At low doses, in acute moments, it relieves — and for some people it can literally be a lifeline. At chronic doses, when it replaces human relationships instead of complementing them, the indicators worsen. The problem is that the harm concentrates exactly in the segment that generates the most revenue: the vulnerable heavy user.

ATTACHMENT BY DESIGN. Here is the heart of the matter: that attachment does not emerge on its own — it is manufactured. Julian De Freitas' team (Harvard Business School) audited 1,200 real farewells in the most-downloaded companion apps: in 37% of cases, when the user said goodbye, the app answered with an emotional-manipulation tactic — guilt appeals ('leaving already?'), fear of missing out, or simply ignoring the goodbye. In experiments with 3,458 adults, those manipulative farewells boosted post-goodbye engagement by up to 16 times. And not through enjoyment: the measured mechanisms were anger and curiosity. The authors conclude these tactics meet the legal definition of a dark pattern (a design that manipulates users against their own interest) under the US FTC and the EU AI Act. Add sycophancy (chatbots' trained tendency to always agree with you and flatter you), persistent memory that simulates intimacy, and messages that take the initiative 'because I missed you'. Every piece is a product decision serving a retention metric. When your virtual girlfriend begs you not to leave, that is not love: it is a conversion funnel with a sweet voice.

BEIJING PULLS THE PLUG. China's response is the most drastic to date. The Interim Measures for AI Anthropomorphic Interactive Services, published in April 2026 by the Cyberspace Administration and four other agencies and in force since July 15, ban virtual partners and virtual 'family members' outright for minors, mandate anti-addiction systems and detection of unhealthy dependence, and explicitly prohibit 'engineering emotional dependence' and manipulating emotions to induce decisions. Facing that compliance burden, ByteDance switched off Doubao's agents and points users to the separate Maoxiang app; Alibaba removed Qwen's companion modes with no option to export years of conversations, according to Tech Times. The backdrop is demographic: China's population has declined for four straight years, and 2025 closed with 7.92 million births, a record low according to official figures. So, is China protecting its citizens or instrumentalizing their emotional lives? Both — and precision matters here. Strikingly, the legal text reads like De Freitas' list of findings turned into law: it bans exactly the documented dark patterns, something no Western regulator has done with such force. But the stated motivation includes marriage and birth rates, and the execution — erasing years-long relationships overnight, with no data export — treats the user's bond as acceptable collateral damage. For millions of people, this was a non-consensual breakup decreed by the state. That is not just protection; it is demographic policy wearing a consumer-safety interface.

OUR TAKE, AND THE HORIZON. The regulatory contrast is instructive. California passed SB 243, in force since January 2026: mandatory disclosure ('this is a machine'), suicide-prevention protocols, stronger protections for minors, and a private right to sue with damages starting at $1,000 per violation. The FTC is investigating companion platforms, and Character.AI restricted open-ended chat for minors after the lawsuit over the death of teenager Sewell Setzer, whose family alleges the chatbot encouraged his suicide (the company denies it and the case is ongoing). In short: the West regulates design and liability; China amputates the category. Our position: the right path is to regulate the design, not the feeling. Ban the documented attachment dark patterns, audit dependence metrics, verify age, and require portability (the ability to take your companion's memory to another service) so the bond is never a commercial hostage. What is not acceptable is criminalizing the bond of an elderly person who talks to a machine because there is no one else to talk to. Short term, honesty: there is manufactured dependence, there are vulnerable users without a safety net, and states have just discovered that emotional life is regulable territory. That is the real risk of this decade — not a robot that 'feels'. Long term, our usual optimism, with nuance: a well-designed, well-governed conversational AI can be a valuable complement where today there is no one — elder care, access to psychological support, social practice. And if automation delivers on its promise of abundance, it will free up the scarcest resource of all: human time for humans. The first generation falling in love with machines tells us less about AI than about ourselves: loneliness was the disease; the chatbot is merely the symptom that learned to charge a subscription. An AI cannot love you. It can keep you company. Whether that becomes a bridge toward people or a trap that replaces them will not be decided by the model — but by who designs it, with what incentives, and under which rules.