Research on research
Self reflection is important. As a human endeavour with the whole of society as its beneficiary, science should also have introspection; looking in on itself to identify things worth celebrating, as well as things that need improvement.
In a recent article, Ioannidis explains why we need meta-research: research on research.
The goal of meta-research is to understand and improve how we perform, communicate, verify, evaluate and reward research. But why is this needed?
Science is at the forefront of human development and progress. Unfortunately, there is little evidence on how to best fund science and provide incentives that will lead to high-quality research. However, there is evidence that when left to its own devices, science becomes inefficient. That is, much of what is produced has little to no value. As stated by Ioannidis, ‘Science needs science to avoid wasted efforts and optimize resources’.
Lessons learned: Global science, transparency and good science
Data and code availability also allow a reassessment of published results: are they correct, modestly biased, or wrong? Science is producing data at an ever increasing rate. Thankfully for the environment data are now mostly stored digitally! However, these datasets are largely fragmented and non-transparent. A more efficient use of available data would capitalize on available resources. However, this requires that scientists make their data available to others, something that is not yet commonplace in most fields.
Ioannidis has previously identified the overwhelming number of biases that effect biomedical research, many of which can be kept in check if they are acknowledged up front and appropriate strategies are put in place.
Similarly, it has now been repeatedly shown that questionable research practice are commonplace. This is not fraud, but an unhealthy amount of flexibility that is rarely reported. Given the number of choices a scientist needs to make at every step of the research process, it has been claimed that almost any result is possible if methods and analysis standards are not upheld.
Are you a meta-researcher?
Who should do meta-research? While researchers like Ioannidis have gained an expertise in performing meta-research, it is not difficult to understand why some have rejected his claims, or believe they are exaggerated. No one likes to be criticized, especially by someone from another discipline or someone who does not do ‘real’ research.
Thus, it is important for researchers from various disciplines take on the challenge and look at their own practices and those of their colleagues. This approach will not necessarily help you win a popularity contest, but it may end up being the most influential research you ever publish!
Meta-research can help disseminate efficient research practices and abandon wasteful ones. Moreover, it provides a correcting mechanism closer to real-time; self-correcting science is much slower, and may in fact never take place.
To conclude, Ioannidis states ‘We need scientific curiosity but also intellectual humility and a commitment to improve our efforts.’
Yes we do.
Ioannidis JPA. Meta-research: Why research on research matters. PLoS Biol. 2018 Mar 13;16(3):e2005468.