Psychology & Behavioral Sciences: Assumption surfacing

What you'll be able to do: Find the assumptions a study is quietly making, that self-report is accurate, that the lab task means what it claims to, that the sample looks like people in general, which is exactly what evaluation marks target and exactly what a results paragraph never states.

Scope note. Engineering's version (article 01 there) surfaces assumptions in a calculation, what happens to the answer if a load turns out not to be static. History's version (article 05 there) surfaces assumptions in an interpretation of events. Psychology's version targets a study's internal machinery specifically: what has to be true about the sample, the measure, and the setting for the stated conclusion to follow from the actual data.

Where psychology hides its assumptions

Read a study's abstract and you get the conclusion. Read the method and you'll find, usually unstated, several decisions that the conclusion quietly depends on. This isn't sloppiness, every study has to make these calls to exist at all, but a stated, examined assumption is a strength and an unstated one is a hole, exactly as in engineering, and the exam rewards knowing the difference.

Self-report is accurate. A huge share of psychological data is someone telling a researcher, or a questionnaire, how they feel or how they behave, This assumes people can accurately introspect on the thing being asked about, aren't shaping their answer to look good (social desirability bias), and interpret the questionnaire items the way the researcher intended. All three routinely fail to some degree, and the size of the failure varies by topic, people are worse at accurately reporting, say, how often they lose their temper than how many hours they slept.

The lab task measures the real-world construct. A noise-blast willingness task is assumed to tell you something about aggression outside the lab. A speeded categorisation task is assumed to tell you something about implicit attitudes that show up in real decisions. The assumption is ecological validity, and it's often the single most attackable part of a classic study's design.

The sample represents people in general. Discussed at length in the primary article as the WEIRD problem, but worth surfacing per study, not just as a general caveat, because the direction of the distortion differs by topic, A finding about conformity from a culturally individualist sample may understate the effect in a collectivist one; a finding about memory from undergraduates may not generalise to older adults at all.

The setting doesn't change the behaviour being measured. Being watched, being in a lab, knowing you're a research participant, all of this can shift behaviour (demand characteristics, evaluation apprehension) in ways that the finding then reports as if it were natural behaviour.

A single measurement captures a construct that actually varies over time. A personality trait, a mood state, or an attitude measured once is assumed to be a stable property rather than a snapshot that could look different a week later.

How to run it

Here's a study's method and conclusion: [paste]. Don't evaluate whether the conclusion is right. List the assumptions it depends on but doesn't state. about the sample, the measure, the setting, and the stability of what's being measured. For each, what happens to the conclusion if it's false: does it weaken the finding, or does it undermine the whole design?

That second question is doing the real work, exactly as in engineering's version: some false assumptions shave a bit off an effect size; others mean the study never measured what it claimed to.

Sorting the output

1. Which you'd defend: reasonable calls any researcher would make.
2. Which you'd flag in an evaluation paragraph: defensible but limiting, and worth a sentence.
3. Which you didn't realise were being made at all. This category is the valuable one, and it's usually where marks were left on the table.

The exam version

Here's a study I've described in my answer and my evaluation of it. Which assumption-based evaluation points would I have earned, and which available ones did I miss, about the sample, the ecological validity of the task, or the reliability of the measure?

This is the cheapest category of marks to recover in the subject: you usually knew the underlying issue (that a lab task might not generalise, that self-report might be biased) and simply didn't think to name it as an assumption worth stating.

Pitfalls

1. Confusing an assumption with a flaw. Most are reasonable design choices; the goal is knowing they were made, not condemning the study for making them.
2. Listing trivial assumptions. "Participants could read the instructions" isn't the interesting kind. Ask for the ones the conclusion actually depends on.
3. Stopping at the list. The what-changes-if-it's-false question is the method; the list alone is just an inventory.
4. Only running this on studies you're suspicious of. Confident, frequently cited studies are exactly where unstated assumptions sit most comfortably, unexamined.
5. The tell: every assumption surfaced is one you'd already have named yourself. Give it a real study's actual method section, not a tidied textbook summary.

Try this today

Take a study you're currently studying, the real method section, not the textbook's one-paragraph summary. Ask what it's assuming about the sample, the measure, the setting, and the stability of the construct, that it never states.

For the one that seems most load-bearing: does the conclusion just weaken if it's false, or does the study stop measuring what it claims to?