Tag Archives: p values

Reflections on p-values and confidence intervals

When we run a statistical test, we almost always obtain a p-value. Many statistical tests will also generate a confidence interval. Unfortunately, many scientists report the p-value and ignore the confidence interval. As pointed by Rothman (2016) and the American Statistical Association, relying on p-values forces a false dichotomy between results that are significant and those that are non-significant. This

Read more

Implying “there’s a trend to statistical significance” is not trendy.

When a p value that fails to reach a threshold is reported, investigators sometimes imply there is a “trend towards statistical significance”. This interpretation expresses the view that if more subjects had been tested, the p value would have become more significant. Epidemiologists Wood and colleagues examined the probability of how the p value of a treatment effect changes when

Read more

Indirect evidence of reporting bias in a survey of medical research studies

Reporting bias (ie. bias arising when dissemination of research findings is influenced by the results) is thought to be common in biomedical and medical research. However, exactly how common it is has been difficult to quantify. Albarqouni and colleagues examined how commonly reporting bias occurs by examining the distribution of p values in medical research studies, and compared these distributions

Read more

P values and hypothesis tests cannot indicate the size or precision of effects

P values and hypothesis testing methods are frequently misused in clinical and experimental research, perhaps because of the misconception that they provide simple, objective tools to separate true from untrue facts. In a new paper, the cardiologist Daniel Mark and statisticians Kerry Lee and Frank Harrell explain the role and limitations of p values and hypothesis tests in clinical research.

Read more

How to calculate the confidence interval from a p value

Confidence intervals are widely reported in published research and are usually thought to provide more information than p values from significance tests because confidence intervals indicate how precise an estimate is. Sometimes, however, investigators report an estimate (eg. a mean) and p value, but not the confidence interval about the estimate. In a BMJ statistics note, statisticians Doug Altman and

Read more

Common misinterpretations of statistical tests

Researchers often use statistical tests to test hypotheses and/or infer properties of a population based on properties of a sample. A key idea is that all statistical tests assume a statistical model provides a complete and valid representation of variability in the data, and faithfully reflects how the study was conducted and the phenomena being tested. For example, when we

Read more
« Older Entries