We all all human. That means we are all prone to bias.

In a recent article in Nature entitled How scientists fool themselves – and how they can stop, Regina Nuzzo points out that our skill to fool ourselves may be contributing to the proliferation of non-reproducible research. In science, career advancement depends primarily on the number of published papers, and the large bias for publishing statistically significant results (i.e., p<0.05) serves as an excellent motivator for the brain to find what it’s primed to find!

Four cognitive biases were identified as particularly problematic for researchers:

• Hypothesis myopia. Collecting evidence to support a hypothesis, but not looking for evidence against it and ignoring other explanations.
• Texas sharpshooter. Focusing on random patterns in the data and mistaking them for interesting findings.
• Asymmetric attention. Rigorously checking unexpected results, but giving expected ones a free pass.
• Just-so storytelling. Finding stories and explanations after the fact to rationalize whatever the results turn out to be.

These cognitive biases influence decision making at a subconscious level and this makes them hard to defend against. Fortunately, scientists have developed various debiasing techniques so that we can avoid fooling ourselves.
Regina Nuzzo highlights four of these techniques:

• Devil’s advocate. Explicitly consider alternative hypotheses, then test them out head-to-head.
• Pre-commitment. Publicly declare a data collection and analysis plan before starting the study.
• Team of rivals. Invite your academic adversaries to collaborate with you on a study.
• Blind data analysis. Analyse data that look real but are not exactly what you collected, and then lift the blind.

Cognitive biases are difficult to identify, and at present there is little to no incentive for scientists to keep them in check. Hopefully this will change in the future and good scientific practices, including those that protect us from fooling ourselves, will be valued above good publication numbers.

Further reading on cognitive biases.

Thinking, Fast and Slow by Nobel laureate Daniel Kahneman.