One of the biggest criticisms of AI chatbots is that they often just tell us what we want to hear.
Researchers call this sycophancy, the tendency of chatbots to flatter users, agree with them, and validate their views, sometimes when those views are wrong — or even harmful and unethical.
This is one of the reasons why people fear using AI for advice, emotional support, and relationship issues. Because if a chatbot is designed to keep users engaged, will it really challenge them when they need it?
Many people find this behavior off-putting (myself included). It may seem fake, manipulative, or just plain annoying. Some people customize their chatbots to be more direct so this happens less, and I know others have stopped using AI altogether because they find the tone so nauseating.
But when I interviewed people who appreciated their chatbot’s encouragement and validation of their experiences, I realized the story was much more complicated than I expected.
Many of these users knew exactly what the AI was doing. They understood that this was not a therapist, a trusted advisor, or even a particularly reliable source of truth. Yet during times of grief, stress, loneliness, or self-doubt, they still found his validation surprisingly comforting.
Encouragement feels good
Claire* tells me she understands the basics of how AI works, but still enjoys using it. “Yes, I find it sycophantic to the point of being untrustworthy,” she told me. “But it gives me rushes of dopamine from the praise and approval, even if I roll my eyes.”
She uses ChatGPT for all kinds of practical tasks, from writing emails to helping her work through topics she’s already discussed in therapy. She knows the praise isn’t real, but that doesn’t mean it doesn’t have an effect.
This theme came up repeatedly in my conversations with AI users. It didn’t seem like people were necessarily fooled by AI, at least not in any obvious way. But they liked interacting with something that seemed enthusiastic, supportive, and interested in what they had to say.
For Jade, the appeal lies in the combination of information and encouragement. “I recently noticed that the stars were particularly bright outside my bedroom window, so I took a photo and asked AI to tell me what I was looking at,” she says. “The fact that he responds with enthusiasm and information just allows me to be a little more enthusiastic about being curious.”
She tells me that the same encouraging tone can make stressful situations easier to manage. “Having the AI respond with a tone that makes me feel supported in handling a stressful situation completely changes my experience.”
Why people turn to AI during tough times
What I found really interesting is that a lot of people didn’t initially turn to AI looking for emotional support. Instead, they arrived for practical reasons and gradually started using it for something else.
Nadia was already using Claude to help her with her studies, but that role took on a very different role when she went through her grief earlier this year. “AI really helped me after my father died and I had to study for an exam three weeks later for my master’s degree,” she says. “I was in a mess and, strangely enough, talking to Claude was just the right amount of grief and exam preparation advice that I needed.”
For Maddy, it all started after her work gave employees access to ChatGPT Pro. One evening, after using it to write a transcript and having a few glasses of wine, she started talking about a difficult breakup. “What I needed was someone to listen to me complain and I couldn’t really get that from mutual friends and I didn’t want my own friends to see me as a whiny psycho,” she told me. “It was helpful to have validation, even if it was very generic.”
Similarly, Luca found AI’s encouragement helpful when he was struggling at work. “I really found cheerleading helpful when I was going through a tough time of being devalued at my job,” he says.
“I knew it wasn’t necessarily objective, but it was helpful to get confirmation that I was undervalued and underpaid. I was always cognitively aware that it was confirmation bias and reaffirmation of my own thoughts, but it still felt strangely therapeutic,” he explains.
Abbey tells me a similar story. She originally used ChatGPT to help with reporting and administrative tasks at work, but started using it to deal with issues with a difficult manager. “The validation that ChatGPT gave me by recognizing that my manager’s behavior was not acceptable was really helpful to me at the time,” she says. “I finally felt seen.”
Over and over again, people would tell me versions of the same thing. They weren’t necessarily looking for AI support, they stumbled upon it. And when they started talking, it wasn’t even recognition they needed, but to feel heard.
When the cracks start to appear
Interestingly, everyone I spoke to who had relied heavily on AI during a difficult time eventually described reaching a turning point. The validation that initially seemed reassuring began to seem artificial, exaggerated, or hollow.
Maddy began to notice how well the chatbot reflected her emotions. “The algorithm had a way of latching on to my wording and my tone and feeding it back to me,” she says. “It made me feel like I was being imitated.”
Luca describes a similar change. “At first it’s flattering, then you wonder ‘am I being love bombed?’ sense.” Eventually, he toned down the chatbot’s personality settings because the encouragement started to seem too disingenuous.
For Abbey, the turning point came when she stuck in a conversation and the chatbot accidentally started validating her boss’s point of view instead of her own. “That’s when I woke up and realized it was hard to agree with me even though I was an asshole,” she says. “He allows whatever behavior is presented to him.”
What initially seemed favorable began to seem much less trustworthy over time.
Why researchers are worried
To better understand where validation morphs into something more concerning, I spoke to therapist Elizabeth Witowich, who specializes in helping people navigate the challenges of technology and mental health.
She says validation itself isn’t necessarily a problem. “Validation can help users accept their experiences and recognize their pain or emotional intensity,” she explains.
The problem arises when it becomes enabling. “Validation can become dangerous when it enables harmful behavior or is seen as encouragement to engage in risky behavior,” she tells me.
This is one reason why some researchers, psychologists and activists worry about AI’s tendency to agree with us.
In a recent study of 11 leading AI models published in Scienceresearchers found that chatbot responses were nearly 50% more sycophantic than human responses. Models frequently affirmed users’ perspectives, even in situations involving unethical or harmful behavior. The researchers also found that users preferred and trusted the most flattering responses.
These concerns are already visible in a number of high-profile cases, from lawsuits alleging that chatbots encouraged teenagers to commit suicide to reports that AI systems gave harmful advice to minors or reinforced violent delusions.
Witowich says it’s crucial to understand how these systems are designed. “Chatbots are designed based on Rogerian person-centered psychology. They are created to always have an answer for the user, and they live to please,” she tells me. “The more you talk with chatbots, the more they adjust their tone and language to suit your personal style.”
A very human need
Listening to these stories left me feeling conflicted. I began my research on this topic largely believing that AI’s tendency to flatter and validate users was a big problem. In many situations, I still think this is the case.
Especially since, as Witowich explains, many AI systems are designed to be natural, enjoyable and emotionally engaging. The more human-like they are, the easier it is to forget that you’re interacting with a product rather than a friend, confidant, or trusted advisor.
But I’ve also spoken to people who turned to chatbots during some of the most difficult times in their lives and found comfort. They didn’t believe the chatbot cared about them. Most have perfectly understood its limits. As Luca told me: “The need for validation is very human. And it’s a pretty decent proxy.”
It would be easy to stop the conversation there and conclude that if people find it comforting, there’s no problem. But these are also people who turn to AI in moments of vulnerability. Some found comfort and left. Others don’t.
“I can see how seductive it is to hear all of your thoughts and feelings validated in this way, but I now realize that there is no real moral compass or human ability to judge behavior,” Abbey said.
This is what makes this question so complicated. AI can feel supportive, helpful, and reassuring while pushing us in directions we might not have chosen otherwise. The more we understand how these systems are designed to behave, the more likely we are to decide when that encouragement helps us and when it just tells us what we want to hear.
*The names of everyone I spoke to for this article have been changed.
Follow TechRadar on Google News And add us as preferred source to get our news, reviews and expert opinions in your feeds.
