The Conversation: Seven deadly sins of statistical misintepretation, and how to avoid them

The Conversation recently published a nice piece by Louis and Chapman on common statistical misinterpretations and how they can be avoided. Here is summary of the main points:

Problem Reason Solution
1. Assuming small differences are meaningful Most small differences are due to chance, not meaningful differences Ask for the margin of error (ie. half of the 95% CI): if the difference observed is smaller than the margin of error, the difference is probably due to random fluctuations in the data.
2. Equating statistical significance with real world significance Generalisations about between-group differences often ignore within-group variability or between-group similarities. Ask for the effect size of the difference between groups and its precision (eg. mean difference and 95% CI)
3. Neglecting to look at extremes For Normally distributed data, a small change in the group average accentuates differences at the extremes of the distribution more than differences within most of the bell shape. When you’re dealing with group averages (which is most of the time), small group differences don’t matter much.
4. Trusting coincidence You can nearly always find an interesting pattern or correlation if you massage the data hard enough. The authors cite a correlation between no. of drownings from falling into pools and films Nicholas Cage appeared in. Ask how reliable the observed association is: has this association only happened once or has it happened before? Can future associations be predicted?
5. Getting causation backward Correlation does not equal causation: Did drownings in pools cause Nicholas Cage to appear in more films, or did Nicholas Cage appearing in more films cause more drownings? Remember to think about reverse causality when you see an association.
6. Forgetting to consider outside causes Known as confounding: eg. people who eat at restaurants appear to be healthier, but they are often healthier because they are richer and can afford better health care. Remember to think about potential outside causes: when investigating a certain cause, think about what, in turn, causes that cause.
7. Deceptive graphs (This is a biggie.) Eg. vertical axis scaling can accentuate differences between groups even though differences are small. Check graph labels along axes. Be very skeptical about unlabeled graphs.

Reference

Louis and Chapman (2017) The seven deadly sins of statistical misinterpretation, and how to avoid them. The Conversation.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s