Unit 4 revisits cognitive bias with a specific focus: how bias operates during data collection and in the presentation of findings. This is a more advanced treatment than Unit 3’s introduction to bias in reasoning generally — now you examine how bias can distort even carefully planned research.
Cognitive biases do not disappear once a researcher designs a good methodology. They can operate at every stage of the research process — from how data is collected, to how it is analysed and interpreted, to how findings are framed for an audience.
KEY TAKEAWAY: The presence of cognitive bias in research does not automatically invalidate findings — it is a risk to be managed, not a flaw to be hidden. High-quality research acknowledges potential sources of bias and describes steps taken to minimise them.
Observer bias (observer-expectancy effect):
When the researcher’s expectations influence what they observe or how they record behaviour. Classic in qualitative research — an interviewer who expects participants to express distress may unconsciously probe more deeply when hints of distress appear.
Participant bias (demand characteristics):
When participants guess what the researcher wants to find and behave accordingly. This is particularly acute in any research where participants know the hypothesis.
Social desirability bias:
Participants report what they think is socially acceptable rather than what is true (e.g., overreporting study hours, underreporting unhealthy behaviour).
Interviewer bias:
The way questions are asked — tone, sequence, wording — can influence responses even when the instrument is otherwise well-designed.
Confirmation bias in analysis:
The tendency to notice, recall and weight data that confirms the hypothesis while minimising contradictory data. This is extremely common and often unconscious.
Motivated reasoning:
When the researcher has an emotional or professional investment in a particular result, their analysis may unconsciously drift toward that outcome.
Pattern recognition error (apophenia):
Seeing patterns in data that are actually random — especially a risk with small samples where statistical noise can appear meaningful.
Overconfidence bias:
Interpreting ambiguous results as more conclusive than the evidence warrants.
EXAM TIP: Questions about bias in the presentation of findings are common. Key errors to identify: (1) selective reporting of results that support the hypothesis; (2) framing conclusions more strongly than the evidence supports; (3) failure to report limitations; (4) misleading data visualisations.
Selective reporting:
Reporting only the results that support the thesis and omitting results that do not. This is a form of research misconduct as well as a reasoning error.
Framing effects in writing:
How results are described shapes how they are perceived:
- “67% of participants improved” vs “33% showed no improvement” — same data, different impressions
- “A significant relationship was found” when the effect size is trivial
Misleading data visualisation:
- Truncated y-axes making small differences look large
- Selective choice of comparison groups
- Cherry-picking time ranges in trend data
Unacknowledged limitations:
Failing to mention methodological weaknesses that would reduce confidence in the conclusions.
| Bias | Mitigation Strategy |
|---|---|
| Confirmation bias | Pre-register hypothesis; actively seek disconfirming evidence |
| Observer bias | Use structured protocols; blind rating where possible |
| Social desirability | Use anonymous surveys; stress confidentiality |
| Selective reporting | Commit to reporting all results, including null findings |
| Motivated reasoning | Seek peer critique of your analysis; use structured analytical tools |
Your Extended Investigation Journal should include regular reflections on how bias may have affected your process:
- “I noticed I was paying more attention to sources that supported my hypothesis — I deliberately sought three that challenged it.”
- “My survey may have introduced social desirability bias because questions about study habits were phrased positively.”
This kind of metacognitive commentary is evidence of intellectual maturity and is directly valued in assessment.
COMMON MISTAKE: Discussing bias only in the limitations section of the report, as a defensive afterthought. Bias mitigation strategies should be discussed in the methods section and limitations. Showing that you anticipated bias and took steps to address it is more impressive than simply acknowledging it existed.
