Why p values aren’t necessary: they are not evidence, and they can’t be replicated
Tag Archives: p values
Why p values aren’t necessary: They don’t tell us whether hypotheses are true
Top journals love their p-values
Researchers love p-values, especially when they are significant. It has previously been demonstrated that there is a disproportionate number of positive or significant p-values. In other words, many reported p-values must be false. That is to say, they are associated with false-positive findings. In a recent article, Cristae & Ioannidis (2018) investigated the characteristics of p-values reported in figures and
Read moreReflections on p-values and confidence intervals
When we run a statistical test, we almost always obtain a p-value. Many statistical tests will also generate a confidence interval. Unfortunately, many scientists report the p-value and ignore the confidence interval. As pointed by Rothman (2016) and the American Statistical Association, relying on p-values forces a false dichotomy between results that are significant and those that are non-significant. This
Read moreImplying “there’s a trend to statistical significance” is not trendy.
When a p value that fails to reach a threshold is reported, investigators sometimes imply there is a “trend towards statistical significance”. This interpretation expresses the view that if more subjects had been tested, the p value would have become more significant. Epidemiologists Wood and colleagues examined the probability of how the p value of a treatment effect changes when
Read moreIndirect evidence of reporting bias in a survey of medical research studies
Reporting bias (ie. bias arising when dissemination of research findings is influenced by the results) is thought to be common in biomedical and medical research. However, exactly how common it is has been difficult to quantify. Albarqouni and colleagues examined how commonly reporting bias occurs by examining the distribution of p values in medical research studies, and compared these distributions
Read more
Scientifically Sound 