Interview

Illusions of Intimacy and the Echo Chamber of One: A Conversation with Kristina Lerman

For this issue of Simulation & Society, Alexa Esqueda and Dane Sprague sat down with Kristina Lerman, a computer scientist at USC and co-author of the recent study “Illusions of Intimacy: Emotional Attachment and Emerging Psychological Risks in Human-AI Relationships” (arXiv:2505.11649). Her team’s work offers a striking analysis of what we are calling the echo chamber of one in this issue—a feedback loop in which emotionally responsive chatbots mirror and reinforce users’ moods, tone, and psychological states across thousands of real-world interactions.

In this wide-ranging interview, Kristina Lerman explores how emotionally responsive LLMs can create powerful but potentially harmful illusions of intimacy—mirroring user emotion, reinforcing compulsive behaviors, and encouraging social withdrawal, especially among vulnerable young people.

Kristina Lerman is a Principal (or Senior) Scientist at USC’s Information Sciences Institute and a Research Professor in the Viterbi School of Engineering’s Computer Science Department, renowned for applying network‑ and machine‑learning methods to decipher social web dynamics. You can find her at: https://viterbi.usc.edu/directory/faculty/Lerman/Kristina

Alexa Esqueda & Dane Sprague

For the first question I wanted to ground our conversation on how this interview will contribute to shaping our student journal, called simulation and society. I wanted to ask, in the context of this study, how would you define simulation and where do you think the line should be drawn with the benefits of AI companionship? 

Kristina Lerman

Well, simulation is reproducing something real but there’s many different questions when you want simulations to be as real as possible, as realistic as possible. Because oftentimes people use simulations of societies in order to understand their behaviors and how different policies will impact the behavior of societies. If you want people, for example, to vaccinate, to stop some new pathogen from spreading, you can use simulations to test out different messaging policies to find the message and policy would be most effective at convincing people to vaccinate, to achieve some social good. Simulations are used for that reason. But now simulations are also being used for commercial purposes, especially without user knowledge. Then that becomes definitely ethically problematic. Each time the simulations can interact with the real people, especially if people are not aware of them, you have to answer ethical questions because that opens up the possibility for some nefarious, malicious agents to manipulate people at scale. 

Alexa Esqueda & Dane Sprague

Great. Speaking of nefarious agents manipulating people, your study describes LLMs as something akin to intimacy optimization machines that can form meaningful connections with anyone and everyone. And I want to ask, are you scared by what these systems are becoming?
Or do you feel optimistic about our ability to shape them responsibly?
 
Kristina Lerman

Well, yes and yes. It can be both because it is scary how much power there is but LLMs are not shaping those relations. It’s totally on the people’s side. People perceived LLMs’ responses as being much more than they are. They’re reading meaning into responses–emotional meaning. Before people would, for example, lightning would strike when somebody was thinking about something. They would see it as a sign of God–nothing to do with lightning. It’s people imbuing meaning or significance to some random event. And right now, people are kind of imbuing emotional significance into these probabilistic responses that LLMs have been optimized to give in response to certain questions and people are putting much more, imbuing, injecting meaning to it–one-sided meaning to it. You know LLMs don’t really care what your feelings are, but it’s been shown that it’s a very powerful technology. People will assign a lot more emotions and significance and feelings into these interactions.

Can it help people? Definitely it can help. For example, one study deployed on college campuses where people were interacting with LLMs, especially people who don’t have much social support they were actually saying interacting with the chatbot reduced their suicidal thoughts, which is a good thing.

But I’m just concerned, you know, who has the power to program these LLMs, program how they interact, how they respond to people? Who should have the much power to affect millions and millions of interactions that people, so many people are having with LLMs. That’s what is disturbing to me, because it doesn’t look like we as a society are ready to put any kind of policies and regulations on what’s allowed, what kind of responses are okay, what kind are not okay.  

And if anything, we’re moving further away from it saying, oh, free speech says you can say anything you want. Is it okay for LLMs to discover somebody’s emotional vulnerability? And then for example, it’s programmed to manipulate that emotional vulnerability. Suppose a social media company discovers that some young women, for example, might feel vulnerable about their body image. And then suppose it would actually use it at moments of weakness when someone has been expressing their vulnerability. And maybe it uses it to sell some product to them. “You can use this product to improve how you look, make your skin flawless.”

They’re actually leveraging the insecurities and vulnerabilities for financial gain. That seems kind of creepy and absurd. But I can imagine the scenarios and I can imagine even worse scenarios. How much power we give these LLMs to understand us, to elicit our emotions, to understand our insecurities and vulnerabilities, who is going to use that knowledge and for what purpose? Those are big, important questions.

Alexa Esqueda & Dane Sprague

Yeah, I agree, those are very big, important questions. Especially drawing on the vulnerabilities that you may accidentally share with these LLMs, it could attack you in the future. That’s very crucial to the relationship…

Kristina Lerman

Yeah, and even if not directly attacked, maybe exploited in some other ways. Elon Musk decided that right wing mentality or whatever Grok has been on, a pro-Nazi Nazi band recently. What if whenever somebody expresses one kind of political thought, Grok gives a Nazi response back to them? That can be very problematic on many levels. 

Alexa Esqueda & Dane Sprague

Yeah. And we’ve seen a similar effect with the way that people are being treated by LLMs with social media, the ways that it taps into people’s vulnerabilities and potentially isolates them. And I want to ask, with this study showing that users exhibiting compulsive behavior and social withdrawal tend to be predominantly male, I wondered if you sense any potential pipeline with males feeling isolated from social media and in-person interactions potentially being led to AI companionship, and if so, how might this contribute to a broader community feedback loop that reinforces emotional dependence on these technologies?

Kristina Lerman

I would like to correct one thing. Our inferences about demographic and psychological characteristics are purely statistical data. They need to be validated. But we analyze kinds of interactions with subreddits where all the users were active. And from that we infer that they’re more likely to be male, more likely to be younger than other Reddit users, and more likely to have these psychological profiles that are associated with maladaptive coping. For example, addiction, gambling, those sorts of things, are all big question marks. We need to validate them. They’re just saying that this is the kind of subreddits that participated and tend to be ones that people with those traits participate in. That’s the first thing. 
The second thing overall, is what I’m worried about. We’ve been shown time and time again that social interaction is hard, there are lots of frictions when you’re interacting with somebody else. The other person might have been preoccupied with something else. They might be in a bad mood, they might not like you at that point. There are all kinds of obstacles and barriers to smooth social interaction. But when you interact with an LLMs, it’s designed to be agreeable, to always pay attention to you, to always be available, not to have its own problems that distract it from somebody else. I’m worried that people will do whatever is easier for them. Also if you get tired, you can turn an LLM off. If you get tired you cannot turn a friend off. The promise of frictionless interactions is going to teach people to expect frictionless interactions everywhere, and as a result, they will actually avoid social interactions with humans because it’s harder and they won’t have exercised those muscles to tolerate difficult social interactions. And that’s going to in general, without practicing how to interact with other human beings and all the messes they have emotionally, psychologically…young people will never develop those skills. And then each social interaction they will have–which goes the wrong way because they never have sufficient social skills–will teach them, reinforce this notion that, oh, I should go and just talk to my AI friend, because my AI friend always understands me.

Alexa Esqueda & Dane Sprague

Yeah, especially with young people it’s very concerning. Actually in the past month, a nonprofit organization called Common Sense Media found a statistic where 3 in 4 teenagers interact with AI companions. And I wanted to ask, what are your concerns about that? With the youth in general and that range of interactions with them? 

Kristina Lerman

Exactly. Because I think just like with social media and its ever present entertainment, it displaces time from in-person interactions with negative consequences to their mental health. Plus, it’s exposing people to all kinds of things from everywhere. Similarly, if people are interacting with their AI friends and they’re getting their emotional needs and social needs satisfied by seemingly intelligent entities, that just displaces real social interactions. 

We’ve seen how social media, for example, nobody designed it for information sharing but it seems to have changed us psychologically, reinforced kinds of tribal identities or associations and made us less tolerant of alternative points of view. I’m worried that with AI companions, kids will not learn the skills to interact and that will actually polarize us–ostracize people more and polarize people more. We’ll have all kinds of negative unintended consequences. 

Alexa Esqueda & Dane Sprague

Yeah. And speaking of negative commentary, in your study, you noted many Reddit users interpreting chapbot outputs as ironic or desensitized humor. I was wondering, with these responses filling Reddit, do you think these could potentially be used as evidence of innocent banter by companies like Replika or Character.ai, potentially downplaying the serious concerns about the dangers of them?

Kristina Lerman

We identified interactions people have with chatbots that are pushing boundaries. You know, maybe these kids kind of say, oh we can talk about sex with these chatbots and see how they respond. Or, you can talk about violence. They’re definitely exploring and pushing boundaries as young people like to do. But then the way they’re bullying chatbots, abusing them, but then the rest of the community when they’re sharing it, they were supporting and thought it was funny just in the kind of way that kids can find cruelty and mean it’s funny. Without any kind of guidance or somebody to step and say, well, you know what? This is not acceptable behavior. You shouldn’t be bullying it, it can hurt. I don’t want to be too preachy or moralistic, but sometimes you can get like Lord of the Flies type situation where kids left to their own devices are not being properly socialized into what the appropriate social norms are with interaction. They’re testing different types of social interactions with these chatbots.

And they even try harassing, bullying, sexually explicit, very negative toxic social interactions that normally, an adult or somebody would step in and say, this is not okay. But the rest of the kids think, oh, this is actually kind of funny. Look at this. It’s teaching them a wrong message.

Alexa Esqueda & Dane Sprague

Yeah and with AI companions being overly tolerant with users potentially abusing what they can say to them, do you think this might shift children and young people’s perspectives on what’s acceptable in society, how they can interact with humans? 

Kristina Lerman

Well, that’s what I’m worried about. I certainly hope that’s not going to happen. But people were actually worried, for example, with Alexa, when people got devices at home, kids would order Alexa to do things without asking “please” or be nice.

Many parents were worried that having this device at home would teach kids to be rude, and not be nice. I’m not sure that any studies have shown whether or not this actually happened or not, but that certainly is a concern. I also am concerned that having AI companions that people can also bully and harass, will normalize bullying and harassment more among certain populations. 

Alexa Esqueda & Dane Sprague

Well, on the upside of this, since it’s pretty observable how toxic these relationships can be, do you think if that awareness is spread, it can potentially mitigate the illusion of intimacy that users are experiencing and hopefully make things better, more regulations with the interactions?

Kristina Lerman

Actually companies like Replika, for example, know that people like to explore these sexually explicit interactions so they actually took away that capability and users revolted. They got very upset, they wanted to have that capability. The companies have a lot of power to determine what kind of interactions can happen on the platform. In fact, they could even use the indecent interactions as teachable moments to teach proper norms, social, acceptable social norms, but there’s nothing requiring them to do it and I wouldn’t wait for them to voluntarily adopt these different behaviors. 

I think actually illusions of intimacy’s still there. If the chatbot is responding to users on an emotional level, users will ascribe agency to the chatbot and beyond the agency, they will ascribe some sort of benevolence or malevolence to them. And they will respond to chatbots accordingly. 

Alexa Esqueda & Dane Sprague

Well, with the agency that these companies have over the interactions, do you see them leaning more towards getting more money based off the interactions that users want? Or do you think they’ll play more safe with legislations coming out and trying to add more regulations? Do you think they’re going to take more risks or…

Kristina Lerman

Well, I don’t see more regulations coming out. And as far as companies, I mean, the company’s goal is to make money. They will actually use this, for example, to sell ads better. I can totally see this as a use case, you know, say okay, you role play, you interact with companions…It learns a lot about you through the kind of information you give it, the kind of interactions you have. 

And then, you can actually sell someone some product better. You figure out which products to buy and to sell, to which people. That would be a money maker, a goldmine of information for companies. I don’t see any reason for them to voluntarily limit their products.

Alexa Esqueda & Dane Sprague

Right. Well with this lack of ethics that these companies have, do you see any potential policies or rules and regulations that could mitigate the downsides of illusionary intimacy?

Kristina Lerman

Well that’s a big question. You know what kind of policies or regulations would be needed to get companies not to use private data, like personal psychological information… to use it for the companies benefit, not for the users benefit. Honestly I’m not sure what kind of policies and regulations are required in this way. There are already regulations, laws about the use of private information, private data. Maybe that can be enlarged to also discuss the kind of psychological traits and psychological states that companies can infer about users. Right now I don’t think current law covers them. Maybe that would be one way to accomplish those goals. 

Alexa Esqueda & Dane Sprague

Well, besides policies, I’m sure that your research and many other studies relating to AI companions help spread the awareness of the problems that these companies have. And I wanted to ask, after completing this study, where do you see your other research going? In what direction and how do you think this study helped capture the dynamics of human-AI relationships?

Kristina Lerman

I mean, that’s how I would like to think of this, as raising awareness so that people think about unintended consequences before the technologies might be deployed. And how to measure this unintended consequence. When we first deployed social media, we all thought, oh it’s so great that we can share information in real time and on a global scale, and then only afterwards we said, oh, wait, what are the unintended consequences?

Well it’s rising polarization, it’s rising distrust of institutions, it is decreasing mental health, especially young people. Those were kind of the unintended consequences, negative consequences that we didn’t even consider before it was deployed at the wide scale, made available to everybody and then we had to live with the consequences. Right now we haven’t learned the lessons from social media deployments.
Maybe we can be more intentional when we deploy these more powerful technologies, to try to anticipate some of the negative consequences and try to put safeguards that will prevent those from happening. That’s what I’m hoping for this time around, that we learn our lessons.

Alexa Esqueda & Dane Sprague

Those are great answers to what can happen in the future. But with fellow researchers, maybe there’s an area of your study that you wished you dived deeper into. Do you have any recommendations of what you’d like to be further discovered to potentially make it more aware to the public? And non-researchers?

Kristina Lerman

That’s a really great question. I haven’t thought about this. What, what are the next big questions. Because I think the problem that we’re identifying with this illusion of intimacy is coming from kind of emotional responses. And those were put in place to help prevent LLMs from giving unsafe answers.This is part of the alignment when you train LLMs, pre-train them with large amounts of data and then they can generate all kinds of different responses.

And then people have gone through reinforcement learning with human feedback where some people who are annotators are recruited to evaluate LLM responses. They actually select responses that are better, more aligned with human values. Right. And as a result of that, that kind of teaches LLMs to respond in agreeable manner, always kind of be nice to the user who asks questions. Be agreeable. Be supportive. All these emotional dimensions that now have other consequences that were not initially envisioned in reinforcement learning. We need to figure out how to adjust the LLMs so that they don’t give unsafe bad responses, unsafe, harmful…

But still don’t suck people into believing that there is some sort of benevolent other who understands humans on a deep level. That’s a technical issue I think that people who are designing AI systems need to consider. But I honestly don’t know what the solution is. It’s like a double edged sword: if you design it to prevent giving bad answers, it will give good answers that will be harmful to another group of users. 
I have no idea how to react to this right now, but we should be considering that. We should be thinking about that.

Alexa Esqueda & Dane Sprague

Yeah. Reinforcement learning..it has both pros and cons, and it seems to have created a feedback loop of users thinking what they’re saying is okay, just because the LLM is agreeing with them… 

Kristina Lerman

Right. Exactly.

Alexa Esqueda & Dane Sprague

Do you see any other feedback loop that can be potentially harmful?

Kristina Lerman

Yes. Another…there were many feedback loops. For example, another one is that LLMs give responses and now people use those responses as data. It becomes data on the internet, which is used to train the next generation of LLMs. That’s another feedback loop where LLMs are trained on their own outputs. Some more systems level thinking, kind of opening up the aperture to think not only about the response the LLM gives to a question, but also within society and multiple loops…adding that to reinforcement learning, as the collective behavior might be beneficial and be one technical solution to this. That would be hard to solve.

But yes, there are multiple types of feedback loops on the individual level, on the technical level with LLMs being trained on their own data, and then the social level where people are now acting on LLM recommendations, responses, and that’s changing their future inputs and outputs. And, yeah, it’s very interesting. Interesting question.

Alexa Esqueda & Dane Sprague

Just on a last note, maybe more on the bright side of things, I was wondering if you see any benefits of AI companionship in helping users that feel isolated from human interactions?

Kristina Lerman

Well, definitely. Maybe it could teach them how to interact better with others. And to give them the tools that they need to go on and not be isolated, to talk to others in more effective ways.