Last year, Fang & Casadevall (2016) wrote an editorial entitled Research Funding: the Case for a Modified Lottery highlighting the chronic and severe lack of research funds and the resistance to change how these funds are allocated. While their proposal may seem drastic, especially to the lucky 10% or so who get their grants funded, the arguments put forth by Fang & Casadevall (2016) are sensible and their suggestion that funding decisions should be based around a modified lottery should be seriously considered.

The situation

In most countries researchers write grants and submit them to funding agencies. These grants are then reviewed by a panel, usually with 2-3 researchers responsible for reading the application in detail. Grants are then scored and ranked, and money is handed out in descending order until there is no more. The pool of money available varies each year, but in many countries it has shrunk to the point where only 10% of grants get funded. As pointed out by Fang & Casadevall (2016), this is in contrast to the 50% funding rates that existed when current funding models were put into place.

With scarce resources, funding decisions become extremely conservative and only grants with assured outcomes are successful. While there are special funds for truly innovative research, this does not address the issue that transformative breakthroughs are often only evident as such after the fact. Funding agencies and governments want to fund the best science, science that will lead to true innovation and impact. Unfortunately, evidence indicates that peer review grant processes do not select the most productive or influential research projects. Why is this?

The problem

Bias. Various sources of bias can influence funding decisions. Some of those highlighted by Fang & Casadevall (2016) are cronyism (ie. favouring certain colleagues or friends) and preference or disfavour for particular research areas, institutions, individual scientists, gender, race or professional status. With so few grants getting funded each year, grant panel members yield tremendous power. Some estimates have it that reviewer bias impacts as much as 25% of funding decisions.

Waste of resources and researchers burnout. Because of scarce research funds, researchers are spending more time writing grants and less time doing research. Furthermore, good grants that go unfunded have to be resubmitted year after year, with researchers spending countless hours tweaking grants in hopes that an extra figure or semi-colon may put them over the threshold. Conversely, panel members are forced to decide between equally meritorious applications, and may end up basing their final decision on grantsmanship, the latter showing no correlation with research productivity or innovation. Panel members have to make impossible decisions and often review the same meritorious grants year after year.

Uneven playing field. Established investigators have important advantages relative to new investigators. When funding is scarce, differences between established and new investigators can become artificially magnified to favour established investigators.

Lessons from finance

Fang & Casadevall (2016) highlight the work of the statistician Nassim Nicholas Taleb who has argued that the most influential events tend to be both highly improbable and unpredictable. In such situations, the best approach is to create a system that is sufficiently robust to withstand negative events and maximize the opportunity to benefit from positive ones.

When applied to science, this means it is pointless to try to predict which grant applications will produce unanticipated transformational discoveries. Such discoveries are the result of serendipity, inspiration and the convergence of disparate observations. Thus, a random strategy that distributes funding as broadly as possible is more likely to maximize the likelihood that such discoveries are made.

In his book The Black Swan, Nassim Taleb underscores the limits of human knowledge and warns against relying on the authority of experts. Novel discoveries often can only be explained in hindsight, with people consistently failing to accurately predict the future.

A possible improvement: a modified lottery

The authors argue that not all projects should be funded. Thus peer review will always be required; not necessarily to select among meritorious grant applications, but to exclude unsound and non-feasible projects. With a pool of meritorious projects identified, research funds should be distributed in a fair and transparent manner using mechanisms that limit or remove bias from funding decisions.

The authors propose a two-stage system:

• Meritorious applications are identified by peer review.
• Funding decisions are made on the basis of a computer-generated lottery.

The benefits of adopting a modified lottery would be many:

• Maintain the important role of peer review to decide what applications should be included in lottery.
• Convert current system with biases and arbitrariness into a more transparent process.
• Lessen the blow of grant rejection as it is easier for humans to rationalize bad luck (i.e., outcome of the lottery) than not making the cut due to lack of merit (i.e., not in the top 10%).
• Relieve reviewers from having to stratify top applications, which is not in fact possible.
• Meritorious but unfunded proposals can continue to have a chance of being funded in the future because they are returned to the lottery for the next round of funding.
• Less expensive to administer.
• Decrease cronyism and bias against women, racial minorities, and new investigators.
• Give administrators in research institutions the ability to make projections based on the percentage of investigators who qualify for the lottery.
• System will be less noisy, fairer, and more promote new areas of investigation by removing favoritism for established fields that are better represented in review panels.
• The realization that many meritorious projects remain unfunded may promote more serious efforts to improve research funding.

Conclusion

Similar to the growing evidence that much of the published literature is irreproducible, there is increasing awareness that current funding models are flawed: a new approach is needed. As always, change is never easy and those who resist it the most are usually those who have found a way to succeed in the current system. But change is possible, especially when the greater good is kept at the forefront of peoples minds.