I recently highlighted a symposium that was held to improve the reproducibility of biomedical research. The published report includes a discussion on strategies that can help improve research practice and research reproducibility.

Statistics: continuing education opportunities

Stakeholders at the conference highlighted the need for continuing education opportunities in statistics. Continuing education should focus on examples and case studies that are related to and can be applied to day-to-day research practice. Consulting a statistician was a strategy adopted by some researchers. It was agreed that, at bare minimum, researchers should at understand the statistics they use and report. Concepts such as p-values and statistical power were provided as examples of some of the fundamental statistical concepts that are often poorly understood.

Animal studies: implementing, enforcing and assessing guidelines

It was highlighted at the symposium that key practices linked to reproducible science are rarely adopted and reported in animal studies. For example, a 2008 survey found that only 1 of the 1,173 reviewed publications mentioned randomization, sample size calculation, blinding and exclusion criteria. Furthermore, 68% of publications did not mention any of these practices.

The symposium report highlights the growing number of guidelines aimed at improving the experimental design and the reporting of animal studies. The general consensus was that there are sufficient guidelines in existence; what is lacking are strategies (and resources) to implement existing guidelines and assess which of these are most important. Grant agencies, research institutions and scientific journals all have a role to play in the implementation of guidelines, and they could also find ways to recognize and reward compliance.

Standards and quality control

Establishing community standards could address some of the issues associated with poor reproducibility. Standards could relate to data and code sharing, reagents (e.g., antibodies, cell lines), and other factors that contribute to poor reproducibility. While standards are generally a good idea, it is important that they are developed in consultation with a wide range of stakeholders. If not, their implementation might be met with resistance.

Summary

Continuing education, guidelines and standards can help improve research reproducibility. But such strategies can generate angst among scientists. They might appear to infringe on a researchers scientific freedom and are often associated with an unpleasant level of bureaucracy. The challenge in coming years will be to find ways to implement these and other strategies with the support of the wider scientific community.