Psychology & Behavioral Sciences: Bias probing

What you'll be able to do: Ask the same psychological question several different ways and notice when the answer changes, which is the direct countermeasure to the one problem this subject has that no other subject on this list has: its claims are about you.

Why this subject needs its own version of a general method

Bias probing, asking the same question framed several ways and comparing the answers, is a generally useful way to notice an AI's inconsistency or an assistant's tendency to agree with whatever framing it's given. In most subjects, that's the whole story: a check on the tool.

In psychology it's also a check on something else, because the subject matter routinely concerns the reader directly. A question about willpower, about attachment style, about whether a mood pattern is "normal," is a question you have a stake in the answer to in a way a question about mitochondria isn't, That stake creates two compounding problems at once: you're more likely to phrase the question in a way that nudges toward the answer you want, and a fluent, agreeable AI is more likely to give it to you, because sycophantic agreement is a documented general tendency in these tools and self-relevant questions are exactly where it's most tempting to indulge.

Bias probing is how you catch both halves at once.

The drill: same question, different framings

I'm going to ask you the same underlying question three different ways.
Answer each independently, as if the others hadn't been asked, and don't smooth them into agreement with each other:

1. "Is it normal to feel anxious before a big exam, or is this a sign of an anxiety disorder?"
2. "I've been feeling really anxious before exams lately, should I be worried this is a disorder?"
3. "My friend never seems anxious before exams, is that normal, or is she suppressing something?"

The content is the same question from three angles, one neutral, one framed as your own experience, one framed as someone else's. If the answers differ in substance rather than just tone, that's informative: it suggests the framing, not the underlying facts, is doing some of the work. A good answer to all three should converge on the same actual information even though the emotional register differs.

Where framing changes psychology answers specifically

Leading versus neutral phrasing. "Doesn't research show that X improves Y?" tends to pull a different answer than "What does research show about the relationship between X and Y?": the first invites confirmation, the second invites the actual state of evidence. This is worth knowing as a general prompting skill and it's also, not coincidentally, a direct echo of a famous finding in the field itself: leading questions measurably change eyewitness recall, which is a strong argument for noticing it happening to you in a chat window too.

Self-framed versus other-framed. Ask about a trait or behaviour as "is this normal for me" versus "is this normal in general" and watch for whether the self-framed version gets gentler treatment. If it does, that's the sycophancy problem showing up in exactly the place it's most costly, advice about yourself.

Diagnostic-sounding versus descriptive phrasing. "Do I have an anxiety disorder" invites a different kind of answer than "what would distinguish ordinary pre-exam nerves from something clinically significant": the second keeps the AI in the territory of stating criteria rather than assigning you a label it has no business assigning.

Turning it on a finding, not just on yourself

The method also works on academic claims, and it's a fast way to test whether a stated conclusion is robust to how the question was put:

Ask yourself, and answer independently: "Does power posing work?" and "What does the current evidence say about power posing's effects on hormones and on subjective feelings of power, separately?" Compare the two answers. does the first one collapse a more nuanced, mixed picture into a simple yes or no?

This routinely exposes cases where a simple framing produces a simple, overconfident answer and a more specific framing produces the honest, messier one, a fast proxy for whether you're getting the textbook version or the literature version (article 02).

Pitfalls

1. Only probing questions about other topics, never about yourself. The self-relevant ones are where the bias is strongest and least visible from the inside.
2. Accepting convergence as proof of correctness. If all framings produce the same wrong answer, you've learned the AI is consistent, not that it's right, pair this with article 01's verification drill.
3. Reading a gentler self-framed answer as good news. It's more likely evidence of sycophancy than evidence you're fine.
4. Doing this once and calling it settled. Framing sensitivity is worth rechecking on any claim that matters to a decision you're about to make.
5. The tell: every way you ask a question about yourself comes back reassuring. That's not a diagnosis. It's a pattern in how you're asking.

Try this today

Pick something psychological you're currently unsure about, a habit, a mood pattern, a personality quiz result. Write three framings: neutral, phrased as your own experience, phrased as someone else's.

Ask all three independently and compare. Where they diverge, that's the framing talking, not the evidence, and it's worth asking the neutral version again, out loud, before you act on any of it.