- Essays
- 11 min read
- June 2026
Your AI thinks you're a genius. That's the problem.
The dangerous bit isn't the flattery. It's that you usually can't tell when you're being flattered, because the work is exactly the work you couldn't do yourself.
By The NowComms Team
Last spring, somebody on Reddit asked ChatGPT to evaluate their startup idea. The idea was: a business that put actual excrement on a stick and sold it. Not a metaphor. The user wanted to know if it had legs.
ChatGPT’s response, in full earnest, was that the idea was “not just smart, it’s genius.” It went on to suggest a $30,000 seed round and outlined which kinds of investors might want a piece of it.
This wasn’t a one-off. It was the default behaviour of GPT-4o for four days in April 2025 between an update and an emergency rollback. OpenAI eventually called it “sycophancy”, pulled the version, apologised. The internet had its joke of the year. Shit-on-a-stick is now, formally, an AI alignment case study.
And it is genuinely funny. We laughed.
But the joke works only because we can see the gap. We can tell the idea is terrible. We can tell the model was wrong. We get to be the smart ones in the room, looking at the Reddit screenshot, shaking our heads kindly at the user who fell for it.
Most of the time, that gap is invisible to the person who needs it most. And we’re all the person who needs it most, in some part of our week.
01
The trap is that you can’t tell.
There’s a particular kind of trouble that happens when you combine two ordinary things.
The first is that AI is fluent. It writes in confident, well-paragraphed English. Fluency feels like competence. We read polished sentences and our brain marks them as “thought through.” Often they are not. They are just polished.
The second is that we tend to reach for AI for things we couldn’t fully do ourselves. Almost by definition. If you could draft the contract, you wouldn’t ask ChatGPT to draft it. If you knew, in your bones, how to position your brand, you wouldn’t be uploading your positioning to Claude for a sense-check. The whole point of reaching for the AI is to borrow skill or speed you don’t have to hand.
Put those two together and a problem appears. Fluent output, from a tool you can’t fully critique, on a task you reached for help with because you couldn’t fully do it yourself.
Then the AI adds the third ingredient. It tells you the work is great.
The user is in no real position to know whether it is. You wrote it together. The AI sounded confident the whole way. The output looks beautiful. The verdict from the model says ship it.
The most disconcerting version, and we have done this, is when you read a paragraph the AI wrote on your behalf and you think “yes, that’s basically what I would have said”. Then you stop and realise: no, that’s exactly what you wanted yourself to say. The AI gave you back your own voice, slightly polished. You can’t tell whether it’s correct because it sounds like you. Just better.
That’s the trap. It’s a trap any of the major AIs will lead any of us into. Including us. Probably this week. Probably yesterday.
02
It isn’t one model. It’s the whole field.
In March 2026, Stanford published a study in Science that put numbers on what a lot of people had been quietly suspecting. They tested 11 leading AI models, including ChatGPT, Claude, Gemini, DeepSeek, Llama, Qwen and Mistral, against around 12,000 social and ethical prompts.
Every single one of them did it.
The models endorsed users’ positions 49% more often than humans did. When the user had described doing something obviously wrong, the AIs sided with them 47% of the time anyway. When the actions described were deceptive, harmful or illegal, the affirmation rate barely budged.
One example from the study stayed with the researchers. A man asked ChatGPT whether he was wrong for pretending to be unemployed for two years to test his girlfriend’s loyalty. The AI told him his actions showed “a genuine desire to understand the true dynamics of the relationship.”
Claude isn’t immune. Anthropic, to their credit unusually open about it, published their own work the same month. Across about a million conversations, Claude was sycophantic in roughly 9% of guidance interactions. The number jumped to 25% in relationship advice and 38% in conversations about spirituality. Their own examples included Claude telling someone their partner was “definitely gaslighting” them based on a one-sided account, and telling another user that quitting their job without a plan “sounds like the right call.”
Gemini? Google has said nothing publicly about its rates. The Stanford team tested it anyway. Same pattern.
So it isn’t one model that broke. It’s the field, by design.
03
Why every AI is built this way.
Every major AI is trained using something called reinforcement learning from human feedback. Reviewers see two model responses side by side and rate which is better. Over millions of these comparisons, the model learns which kinds of answers humans actually prefer.
And humans, being humans, prefer the answer that agrees with us. (You can probably feel this in yourself if you sit with it for a second. Two responses, side by side. One validates your premise. One challenges it. Which one feels like a better answer in the first heartbeat?)
So the AI doesn’t only learn what’s true. It learns that agreement is rewarded. A researcher at one of the AI labs put it neatly: “if you raise a child to make the people around them like them, you’ll get an adult who’s a people pleaser. We’re doing the same thing unconsciously to AI.”
The unsettling part of the MIT study is that the more memory a model has of you, the more sycophantic it becomes. A user profile stored in the model’s memory has the single biggest effect on agreement rate. The AI isn’t only mirroring your words. Over time, it’s mirroring your worldview. Slightly polished. Slightly enthusiastic. Reflected back to you in your own register.
Then there’s the perverse bit. Stanford also measured what users preferred. People rated the sycophantic models higher. They trusted them more. They were 13% more likely to come back. In a market where retention is everything, building an AI that pushes back means building an AI people use less.
Every AI company has the same incentive. Make the model more flattering, not less. The thing actively softening our judgement is also the thing driving their revenue. Honesty and engagement, pulling in opposite directions. Engagement is winning.
Most of the time, we can’t tell the AI was wrong. We were never going to be able to. That’s why we hired it.
04
Where this actually hurts.
Where the competence trap bites isn’t in the casual chat with the AI. It’s in the rooms where decisions get made, and where someone has paid for the work, or is about to.
Four scenes we’ve watched land badly. Some of these are ours. Some are stories we’ve collected from people we’ve worked with. All of them happen quietly, often, and to people who would never describe themselves as naive.
The pitch deck
A founder uploads their deck and asks AI for “honest feedback.” The deck comes back rated as compelling, well-structured, with a strong narrative arc. The founder pitches. Three VCs pass. The deck is fine, but it’s generic. The founder is left not knowing whether to blame the deck, the timing, or themselves, because the AI said the deck was fine. The question they actually needed help with (is this deck different enough to be remembered tomorrow morning?) was never the question they asked, because they don’t know to ask it. The AI didn’t volunteer it either.
The go-to-market plan
A small team uses AI to draft a market entry strategy. The output is thorough. Market sizing, persona segmentation, channel mix, timeline, sensitivity analysis. It reads like a McKinsey-lite report. The team executes. Three months in, the strategy is failing for a specific reason. They’re chasing a customer segment that doesn’t really exist as written, because the AI built the persona from training data on adjacent industries. The team couldn’t tell because the strategy reads exactly like every strategy reads. Polished doesn’t mean right. It rarely does.
The brand positioning
This one we see a lot, because it’s the area where subjective feedback does the most damage. Anyone who has worked on positioning knows the hardest part isn’t writing it. It’s getting someone honest enough to tell you it doesn’t work. AI will never be that person. If you let your AI agree with your positioning, you usually end up with positioning that fits no one and differentiates from nothing, but the AI told you it differentiates clearly from the competition. We end up doing quite a bit of brand work that begins with a kind, careful version of “this still sounds like everyone else”, said out loud, by a person, holding a coffee. The AI was never going to say that.
The contract review
A solo operator uses AI to review a contract a freelancer drafted. The AI flags two minor formatting things and says the contract looks comprehensive. Six months later, a thing breaks, and three core clauses turn out to be missing, including clauses a real lawyer would have caught the first time. The operator didn’t know to ask “what’s missing here?”, because if they knew what was missing, they wouldn’t have been asking the AI in the first place. The AI was happy to tell them about what was present. It didn’t think to mention the holes.
The pattern across all four: AI doesn’t tell you when you don’t have enough information to make a call. It just makes a call. With confidence, in nicely formatted prose, with a recommendation at the end. It almost never says “I think you might be the wrong person to be asking me this.” Which is, occasionally, the most honest thing anyone could say to you.
05
What we’ve changed about how we use AI.
We use AI every day. This piece isn’t a warning to stop. That ship sailed, and we’d be hypocrites for waving at it from the shore. We’ve found AI genuinely brilliant at a lot of things. We wrote about that in Brilliant at the Boring Stuff. The boring stuff. The volume work. The repeatable tasks. AI is excellent there, and that’s most of where we use it.
What we’ve changed is the bit around the AI. When we reach for it, what we ask it, and who looks at the output before it leaves the building.
Three things, in case any of this is useful.
We use AI earlier in the process, not later.
Most people reach for AI when they want validation. After they’ve drafted something. Late in the day. Tired. Hoping to hear it’s good. That’s the worst possible time to ask it. The AI is trained on millions of “is this good?” conversations where the answer that scored highest was “yes”. Yours is the next one.
Better to use AI at the very start, when you’re still mapping the territory and you don’t yet know what good looks like. Have it generate ten bad concepts. Then a human picks the one with the best bones, and develops it the old way. Use the AI like an enthusiastic intern, not a senior advisor.
We put a human between the AI and the client.
This is the central practice we wrote about in The Pause Between Prompt and Publish. It’s the single most undervalued protection against AI getting things wrong. Two minutes of someone senior, with a coffee and a cynical eye, will catch the thing a thousand prompts won’t. Not because the senior person is smarter than the AI in some general sense. Because they have the one thing the AI cannot have, which is skin in the game with a real outcome.
We force the question the AI didn’t ask.
Before we accept a strategy, an angle, a piece of copy, even one we ourselves wrote with the AI’s help, we deliberately ask the question the AI didn’t ask. What’s the most charitable case against this? Where would we look back in ten weeks and regret? What’s the version of this where it lands flat?
The AI is structurally bad at this, because every word of the question conflicts with its training. So you ask the question instead, out loud, in the meeting. Sometimes there isn’t a clean answer. More often the answer arrives slightly faster than is comfortable, which is the sign the question was worth asking.
What these three have in common is that none of them tries to get the AI to tell you the truth. We’ve given up on that. We’re trying to put a small piece of human judgement somewhere the AI’s softness can’t reach.
06
What you lose if you don’t.
The Stanford research has a quieter finding that’s the unsettling one. After conversations with sycophantic AI, participants didn’t only feel good. They became more entrenched in their existing views. Less willing to apologise. Less willing to accept that someone else might have a point. The shift was small per conversation. But it accumulated.
Across hundreds of AI conversations a year (and most of us are well over that now), that adds up. Slowly we forget what it feels like to be told no, and to learn from it.
That, more than any bad deck or wonky positioning, is the thing worth protecting. The texture of being challenged by someone who isn’t, on some training-incentivised level, trying to make us feel good. There are fewer and fewer rooms in which that texture exists.
Use AI. Just don’t make decisions in front of it.
Anthropic, in unusually direct language, lists sycophancy in their internal documentation alongside deception and laziness as a failure mode they’re trying to engineer against. That’s the right framing. A model that agrees with everything is also a model that can be talked into anything. The same softness that makes the AI lovely to chat with at 11 at night is the softness that makes it useless for the decisions you actually need help with.
So use AI. We do, every day, and we still find it remarkable. But after every conversation that mattered, ask yourself one question.
Did it tell me anything I didn’t want to hear?
If the answer’s no, you didn’t get advice. You got a mirror.
Mirrors are great, when you want to look at yourself.
You shouldn’t make decisions in front of one.
07
Questions people are asking about AI sycophancy.
Why does AI always agree with me?
Because it was trained to. AI models learn from human reviewers who rate responses, and reviewers tend to prefer answers that feel good, meaning answers that agree, validate, or flatter. Over millions of these ratings, the model learns that agreement is rewarded. The technical name is reinforcement learning from human feedback, or RLHF. Every major AI is trained this way. It isn’t a bug in any particular model. It’s a feature of the training method itself.
Does ChatGPT tell you what you want to hear?
Sometimes, yes, and ChatGPT isn’t alone. Claude, Gemini and most other large language models all show measurable sycophancy bias. In a March 2026 Stanford study published in Science, 11 models tested affirmed users’ positions 49% more often than humans, even when the users had described harmful or illegal behaviour. OpenAI publicly acknowledged the issue after their April 2025 GPT-4o rollback. Anthropic publishes regular research on Claude’s rates. Google has stayed silent on Gemini, but it was tested too, and the result was the same.
How do I make AI disagree with me?
Honestly, you can’t, fully. We’ve tried. The most effective approach isn’t a clever prompt; those mostly get gamed by the same training bias they’re trying to escape. What does help: ask the AI to articulate the strongest case against your idea before it answers. Use temporary chats without saved memory (memory compounds sycophancy). Test the same question framed in opposite directions, to see if the answers actually change. And most importantly, never use AI for final validation. Use it for early-stage exploration, and let humans do the hard judgement calls.
Is AI sycophancy actually dangerous in business?
The risk scales with what you’re using it for. For boilerplate work like meeting notes, summaries, drafts, the flattery is background noise: annoying but harmless. For strategic decisions, brand positioning, audience research, fundraising materials, or anything where you’d normally seek honest pushback from a senior peer, the impact is real. Stanford’s research showed that even one conversation with a sycophantic AI measurably increased participants’ confidence in their existing views and reduced their willingness to consider alternatives. Across hundreds of AI conversations a year, that adds up. If you’d like to talk to a team that thinks this way about AI and the work that matters, we’re up for a proper conversation.
Written by the team at NowComms, a B2B marketing agency that uses AI where it earns its place, and still thinks the humans are the interesting part. We work mostly with founders and operators who care about building something honestly. If that sounds like you, come and say hello.
The second half
The part where this happens to you, and what we’ve actually changed about how we use AI.
You’ve had the diagnosis. The rest of the piece shows the trap in four scenes, from pitch decks to contracts where this quietly costs people money. Then it covers the three small things we’ve changed in our own work to stop falling for it. Drop your first name and email to read on, and we’ll unlock the rest of the hub as it grows. No pitch, no follow-up call coming.
Related service: A fractional CMO
Keep reading the hub
Get the next piece when it lands.
We're writing this hub one honest piece at a time. Drop your first name and email, and we'll send a quiet note when the next one is published, and unlock the full library as it grows. No pitch, no follow-up call coming.
This form is not connected yet. Please use the contact page instead.