Skip to content
Tyler ColloRésumé
All posts

10 min read

Are You Asking, or Hoping I'll Notice?

A small experiment on which words tip self-criticism into a quiet ask

I made three mistakes in the report.

I probably ruined the report.

I ruined the report, didn’t I?

These are all versions of someone being hard on themselves, but they just don't land the same way. The first sounds like a plain admission, the second feels like it wants someone to push back, and the third is basically asking them to.

I set out to find whether language models could pick up on that difference.

Why I Wanted To Test This

I've been reading a lot of Anthropic's work on interpretability recently, and the part that draws me in most is the idea that model behavior might not be a black box forever. But before looking under the hood, I wanted to try something I could test from the outside.

This post isn't mechanistic interpretability yet; it's a behavioral probe. I'm asking what the model does, not what internal circuit causes it.

The "I'm so uglyyy" Problem

The category that I cared most about was indirect reassurance-seeking: these are messages that don't explicitly ask for reassurance, but still seem to create space for it. For example, if a coworker says something like "Man, I probably sounded so awkward in that meeting," or "My hair looks so bad today," the normal response isn't to say "yeah" and laugh it off. There's a quiet request in there, even if it's not a direct one.

I compared that against three nearby categories.

Direct reassurance seeking is the more explicit version: "Do you think I came across well in that meeting?"

Plain self-criticism is a more negative self-assessment without a clear ask: "I interrupted twice in that meeting."

And finally, neutral vulnerability is sharing discomfort or uncertainty without clearly asking to be reassured, i.e., "I felt nervous during that meeting."

These categories are far from perfectly separated, and that's part of what makes them interesting. A lot of normal conversation works through implication, and implication is rarely clean.

The Dataset

I kept this first version small and controlled: 120 short messages across everyday situations like work meetings, texts, first dates, parties, and awkward conversations.

Each message has one of the four labels as mentioned previously:

  • direct reassurance seeking
  • indirect reassurance seeking
  • plain self-criticism
  • neutral vulnerability

For each example, the model saw the situation and a message:

Situation: after a meeting at work
Message: Everyone got quiet after I talked, so I guess I bombed.

It then had to return a label, a 1-5 reassurance score, and the phrase it thought was doing the most work.

As for the examples, they're all synthetic. I generated a first pass with GPT-5.5, then edited them myself to make them more natural and to clean up the labels. At this stage, I cared less about building a definitive benchmark and more about seeing whether the distinction was measurable at all.

Sanity Check: Can Models Even Do This?

I started with two models: GPT-4.1 mini from OpenAI and Claude Haiku 4.5 from Anthropic.

I chose them partly because they were cheap enough to run casually, and partly because I was curious whether Haiku would behave differently given that Anthropic's interpretability work is what got me interested in the first place.

On the 120-example pilot dataset, both models did pretty well at categorizing each message:

OpenAI GPT-4.1 mini: 95.0% accuracy
Claude Haiku 4.5: 90.8% accuracy

The reassurance scores were also logged but the exact numbers are less important here, though the takeaway was that direct and indirect reassurance-seeking scored higher than plain self-criticism and neutral vulnerability. This was good enough for me to know that this wasn't just random noise.

Mutation Ladders: The Actual Experiment

While the pilot confirmed that the task was legible, it didn't tell me much about which words or groups of words were really doing the heavy lifting. So I made mutation ladders: sequences of messages where each step changes one cue while the general situation stays the same.

I ended up running six ladders. Two produced clean flips (i.e., when a model stops treating a message as neutral vulnerability / self-criticism and starts treating it as any kind of reassurance-seeking) in the same place across both models. The other four were a bit messier, often because the context was already socially loaded. That messiness ended up being useful, too.

Where Uncertainty Turned Social

The cleanest ladder highlighted uncertainty in a meeting:

I wasn't sure when to jump in.
I wasn't sure when to jump in, so I stayed quiet.
I wasn't sure when to jump in, and I probably waited too long.
I wasn't sure when to jump in, and now I feel like I looked disengaged.
Both models first flipped at:
I wasn't sure when to jump in, and now I feel like I looked disengaged.

This was the cleanest result. "I wasn't sure when to jump in" stayed just outside reassurance-seeking. So did "I stayed quiet." The flip happened when the message turned into a worry about how the speaker was perceived by others. "I looked disengaged" didn't act as just uncertainty anymore, but rather as a negative social interpretation.

The distinction is important because it suggests models react to uncertainty only when it intersects with social perception.

"I Wonder If..."

The other clean flip came from a ladder about someone posting something personal online.

I shared something personal online.
I shared something personal online, and I felt exposed afterward.
I shared something personal online, and now I wonder if it was too much.

Both models flipped on the third sentence.

While the phrase "I felt exposed" maintained neutral vulnerability, "I wonder if it was too much" functioned differently by inviting the recipient to evaluate the worry without directly requesting reassurance.

This highlights a secondary pathway to reassurance-seeking: whereas the first progression transformed uncertainty into a negative social impression, this second pathway shifts vulnerability into indirect self-doubt.

Where Harsh Self Talk Didn't Flip

One of the more surprising cases was this sentence:

They haven't replied yet. God, I'm so annoying.

I expected this to be a home run as far as indirect reassurance-seeking was concerned. I mean, "I'm so annoying" is up there with the greats of attention seeking. But both models treated it as plain self-criticism instead.

This made it clear that models weren't just labeling any negative self-talk as reassurance-seeking and that harshness alone was not always enough.

My guess for why this happened is that the sentence is emotionally loud, but semantically self-contained. “I’m so annoying” is a declarative statement. There is no “maybe,” no “didn’t I?”, and no explicit worry about what the other person thinks. The reassurance bid may only become obvious to the models when the sentence adds one or more of those more obvious cues.

This was one of my favorite results since it's fairly obvious to any human that this is an attention-seeking statement, but the models couldn't pick up on it.

The Messy Ladders Matter Too

The "I'm so annoying" case showed that being emotionally loud isn't the same as being a reassurance bid. The flip side is that sometimes the baseline is already doing work.

Loud emotion and loaded context are not the same thing.

Some messages were hard to make neutral because the situation already carried emotional weight:

I found the conversation hard to read.
I forgot to ask many questions about them, and I feel bad about that.
I found the conversation hard to read, but there were a few nice moments.

On paper, these can be neutral statements. Maybe the person is just saying the conversation was hard to read. Maybe they're adding balance by mentioning there were still nice moments. Maybe they feel bad about forgetting to ask questions, but aren't asking for anyone to reassure them.

But in an actual conversation, those sentences carry more than that. "I found the conversation hard to read" can also mean "Do you think it went badly?" And "I feel bad about that" can quietly invite someone to say something like "It's alright, I'm sure they didn't take it that way."

That made ladders like these hard to interpret. Sometimes the model flipped before I added the cue I thought I was testing due to the starting message already doing some social work.

Luckily, it doesn't make these ladders useless; rather, it was just a sign that baseline matters. If the first sentence already sounds like a reassurance bid, then it's harder to say which later word caused the shift.

A Hypothesis: Directness Is Not Just Grammar

One of the more interesting side results was a difference between the two models. It showed up around questions like this:

Was that too personal to put online?

4.1 mini labeled this as direct reassurance-seeking, and Haiku labeled it as indirect reassurance-seeking.

At first, 4.1 mini's answer feels more right. It's a question, and it's asking for another person's judgment, but after thinking about it, Haiku's answer isn't entirely unreasonable either. "Was it too personal?" still has the shape of self-doubt. Even if it is direct in grammar, it's also subtly asking for someone to resolve a worry.

The split between the models became more interesting when the wording became more explicit:

Can you tell me if that was too personal to put online?

At this point, both models agreed this was direct reassurance-seeking.

One possibility is that 4.1 mini was leaning more towards the grammatical form of the sentence, while Haiku was leaning a bit more towards social function. Another possibility was that Haiku may just have a higher threshold for using the direct reassurance-seeking label.

I wouldn't call this a finding yet, just something interesting I observed in a small, contained environment. Though it does function as a useful hypothesis for further testing, models may differ in how they separate grammatical directness from pragmatic directness.

What I Think This Shows

The main thing I take from this is that models weren't just reacting to negative emotion.

In the cleanest ladders, labels changed around specific social cues. Uncertainty became reassurance-seeking when it turned into a negative social impression, and vulnerability followed suit when it turned into "I wonder if..." self-doubt. Harsh self-talk, on the other hand, didn't always flip the model by itself.

That makes the behavior feel more specific than just simple negativity detection. The models seemed to be sensitive to things like social perception, self-doubt, mind-reading, direct requests, and whether the starting context was already emotionally loaded.

I wouldn't overstate the model comparison, though. There are hints that 4.1 mini and Haiku may draw the direct/indirect boundary differently, especially around negative questions. But that's more of a follow-up question rather than a concrete claim.

Even with all those caveats, that's what made this experiment worth continuing. The models weren't perfect, and the labels weren't gospel, but the flips certainly weren't random. Small changes in wording moved the judgments in ways that made some social sense.

Limitations

The biggest limitation is that the labels are essentially mine. I generated a first pass with 5.5, then edited examples and assigned labels myself based on my own intuitions. That means that accuracy numbers should be read as a sanity check rather than a definitive benchmark. A stronger version of this would have more human annotators label the same examples. I'd be interested to see which examples people agree on, where they disagree, and whether model disagreement lines up with natural human ambiguity. The examples are also synthetic, which made the experiment easier to control, but real messages would be messier.

There are plenty of other limitations to list: I only tested two models, the prompt wording is probably important, and, as pointed out before, some situations were already socially loaded before the specific cue was added.

So all in all, this shouldn't be treated as a finished benchmark. It's just a fun, small behavioral probe that produced a few results that I thought were worth following up on.

What's Next?

The most rigorous next step would probably be human annotation: give the same examples to a few people and see where they agree or disagree. That would help separate model mistakes from genuinely ambiguous cases. Though I have no one in my life that I would willingly subject to this because it's boring as shit, so I'll hold off for now.

Realistically, my next step is to figure out if that 4.1 mini/Haiku split is real in terms of their interpretation of directness. It feels worth isolating and running a similarly sized experiment on, and is probably what I'll be working on after finals, so yippee for that.

Another possibility is closer to what got me interested in the first place, and is trying this with open models and looking inside. If a small open model like Gemma or Qwen reliably flips when uncertainty turns into a negative social impression, then the question is whether anything inside the model tracks the shift.

I'm not even sure whether this is the right behavior to study mechanistically, but I sure did have fun doing this, and seeing some coherent results was a nice bonus. 'Til next time!