I assume that, if you are reading this post, you are familiar with research reproducibility and the issues that surround it. Nevertheless, you might not be familiar with the difficulties early career researchers face in this era of open science.

A recent paper published in PLOS Biology by Allen & Meller entitled Open science challenges, benefits and tips in early career and beyond highlights some of the challenges faced by early career researchers in a “publish or perish” world that is trying to make a move -willingly or not- towards open science.

Challenges

1. Restrictions on flexibility
2. The time cost
3. Incentive structure isn’t in place yet

It is true that by adopting open science practices, you loose some flexibility. You can’t keep slicing and dicing data until statistical analyses return significant p-values. You can’t postulate a hypotheses after the results of your study are known. You can’t omit outcomes or analyses from your scientific papers.

It is also true that open science requires more time when first starting out. You have to learn about open science and how to implement it into your research workflow, with some trial-and-error being inevitable. You have to carefully prepare (and hopefully register) your research protocols and analyses plans before you start collecting data. You have to curate your data and code so that you can make it available to other researchers. You may also have to collect (substantially) more data than in previous studies to achieve adequate statistical power.

Both these challenges will reduce the number of papers you can publish in a year. Reduced scientific output could hurt an early career researcher’s chances of getting a tenure track job or a grant. Moreover, unless you do your PhD and post-doc with more senior researchers that have already made the transition towards open science, it might be difficult to convince a senior scientist to take you on as a trainee if you inform them that your research will take longer to complete, and that there is a higher probability that the results will be negative.

Finally, it is very much the case that there is currently little incentive to transition to open science. While people are beginning to recognise and value open science practices, most incentives that are still in place (e.g. numbers of publications and publishing in journals with high impact factors) negatively impact the quality of published results. Only in the last few years has this started to change slightly. It will take a decade or more for institutional incentive structures to change, and for researchers and administrators to buy into this change (and forget the old flawed incentive schemes that led them to have apparent success).

The writing is on the wall. Open science and reproducible research will someday be the only game in town. But what are early career researchers to do in the meantime?

My personal feeling is that there are easy, incremental steps that researchers can take to make the transition towards open science. The key is to always be moving forwards, towards open science. Let’s start our next study with the mindset that we will be uploading our data with our published manuscript. Let’s plan for the study after that to also include the computer code used to analyse the data. Let’s challenge ourselves to pre-register the protocol and analysis plan of our next study with a clear and focused hypothesis.

Challenging ourselves in this way will improve the openness (and quality) of our research. And as pointed out by Allen & Meyer (2019), there are benefits to moving towards open science.

The impact of research registration

In their paper, Allen & Meyer (2019) present an astounding finding. Similar to previous research looking at the impact that research registration had on clinical trials, Allen & Meyer (2019) show that registered reports have a dramatically higher rate of null research findings when compared to non-registered studies.

As the figure below illustrates, the audit performed by Allen & Meyer (2019) found that traditional research reports have as their primary result a null finding 13% of the time. This is in stark contrast to the 55-65% of null findings from registered results.

While this lesson has been heard for clinical trials, this is one of the first clear examples of just how biased non-registered research is towards finding and reporting statistically significant results.

Fig 1. Percentages of null findings among RRs and traditional (non-RR) literature [46,47], with their respective 95% confidence intervals. In total, we extracted n = 153 hypotheses from RRs that were declared as replication attempts and n = 143 hypotheses that were declared as original research. The bounds of the confidence intervals shown for traditional literature were based on estimates (5% and 20%, respectively) of null findings that have been previously reported for traditional literature [46,47]. Data is available on the Open Science Framework (https://osf.io/wy2ek/) and in S1 Data. RR, registered report.

Figure from Allen C, Mehler DMA (2019). Open science challenges, benefits and tips in early career and beyond. PLoS Biol 17:e3000246.

https://doi.org/10.1371/journal.pbio.3000246.g001

Benefits

1. Greater faith in research
2. New helpful systems
3. Investment in your future

These benefits are genuine. As a researcher and a member of society, open science does bring me hope that research funds will cease to be wasted and genuine advances in our understanding of the world can be made. The various tools and resources that are being created are helping those who want to transition, and they are allowing for true collaboration and advancement to take take place. By deciding to do science differently, in an open way, early career researchers are setting themselves up to be the leaders of tomorrow.

I agree with Allen & Meyer (2019). However, at present, these benefits are idealistic or based on a long-term perspective. The scientists and academics of today, the ones in positions of power, have made their way and found success in an era of publish-or-perish. There are leaders who have seen the light, taken a sip of the Cool-Aid if you will, but these people are far and few between. I get inspired when I talk with them, I get a sense that genuine change is afoot.

However, my inspiration quickly turns to frustration when I take a glimpse of the wider scientific ecosystem. Moreover, I do not see how the average early career researcher keen on open science can currently compete against others who have been trained and groomed in the more traditional way of doing science.

Societal good is important. But so too is having a job and being able to pay your bills, eat and have a roof over your head. At present, the incentive system remains heavily skewed towards the old system. What is needed is a clear way to assess and incentivise research quality. Not impact, not novelty, but quality. Those who get hired, promoted and funded should do quality research first, impact and novelty should and must come second.

Summary

Early career researchers are the future of science, and they should be encouraged to pursue open science practices. Conversely, senior scientists should foster this curiosity and learn from the minds of tomorrow.

Reference

Allen C, Mehler DMA (2019). Open science challenges, benefits and tips in early career and beyond. PLoS Biol 17:e3000246.