How funding should be distributed

(I’m talking about UK or EU government funding.)

The problem

Thousands of researchers all across Europe are spending huge amounts of time writing detailed project proposals for projects that will never get funded. Success rates vary by the scheme, but around 10% get funded. The proposals are long and detailed and take a lot of time to write. UK national funding has the same problem, although rates are a bit higher and documents shorter. But they still waste a huge amount of time (of researchers who are highly trained, representing a big investment from society). Governments should avoid wasting researchers’ time writing detailed proposals for projects that will never be funded. In the UK case EPSRC explicitly say they want to slow people down by making them make a new proposal rather than resubmit (think of the case where you had one nasty reviewer, or a reviewer felt you had one missing component in your proposal - you can’t just add that and resubmit). It is not the case that proposals are simply bounced for improvement and subsequently funded. This only happens in a tiny number of cases. The vast majority of rejected project proposals will never be funded. They are a complete waste of time - a loss of researcher time that could be put to better use by society.

Even among those who get a lot of funding far too much of their time is spent in proposal writing when they could be better utilised directing and doing actual research. For some the whole year is planned in terms of what grants they will write, instead of doing any research. (The writing is largely about marketing, presentation, covering possible objections, I estimate that no more than 5% of this time would be spent on generating actual research ideas.)

The solution

There is always a lottery element to the decision. Acknowledge this.

Have the lottery at the start, save the time of the people from writing a detailed proposal. Researchers can simply enter their name for the lottery (optionally add two publications for credibility). Or if funders insist they could enter a short paragraph abstract.

Invite winners to develop the detail - have them on a track to get the money. They can work with reviewers to develop a proposal, much like a journal author works with feedback from reviewers and iterates, e.g. in the case of a major revision. The reviewers are there to help make it better, but the paper/proposal is already on track to acceptance.

Aside: for those who don't accept it is a lottery

For those who don’t accept that it is already a lottery, there are plenty of examples where a single proposal can score a 2 or 3, and a 6 (top mark in UK EPSRC) from different reviewers. They are drawn from a distribution. Here are two examples of actual samples from different EPSRC submissions: [3,5,6,6],[3,4,5,6],[1,4,5,6],[4,6,6] (all rejected). The [3,4,5,6] example had completely different comments on the same proposal. There wasn’t a single criticism that was in common among any two reviewers. What one guy says is good the other guy says is bad. 

 

Some may argue that some proposals are weak and rightly rejected, and that can be true, but most proposals are good, and are still rejected (Andrew Derrington says the same: "Most well-written grant applications get rejected." [2] "Funding rates are falling and sometimes perfect grants, grants that propose well-designed projects that will answer important questions, don’t get funded because there just isn’t enough money." [3]). The same proposals sent to two different panels, and different reviewers, would most likely result in a different ranking. At the boundary between those who get funded or not, the difference is down to feelings of the reviewers and panel at the time they act, and could have gone differently on another day. For well-written proposals the lottery element is the main factor. To quote Derrington again: "The failure rate makes it necessary to submit several applications to be reasonably certain of securing a grant. Even the best-written grant applications from the strongest applicants have a reasonably high chance of failure – maybe as high as 50%. This means it would be foolish to risk too much on a single application – or to be too disappointed by a single failure. I think that the best strategy is to submit four or five grant applications in quick succession, all based on the same set of ideas. Then, if you get five straight rejections, you can be reasonably sure that it is time to change your approach." [4]

Why do I quote Derrington so much? People might not believe it coming only from me. Derrington has more experience and this gives him more credibility. He is saying you need to write multiple applications spinning the same idea in different ways in order to get one grant. What a waste of time.

To give better chances there should be more small grants, and no too large ones. Small grants give more bang for the buck [1], but bureaucrats and politicians sometimes like big ones so they can boast about the great big research project they fund.

They could even weight it to favour established players if they want (though I think this idea is terrible).

Ways in which Leverhulme Funding is better

Small grants

Outline (short document)

No top down trying to tell academics what to research

How Government Agencies Could and Should Support Scientists Instead of Wasting Their Time

Currently the agencies are trying to have some top-down strategy of the direction of the research and themes and strategic priorities.

 

They shouldn’t even try, because for true research nobody knows which direction will bring the biggest benefit. Many times looking back one sees it was some obscure work that gave a breakthrough, and the celebrated approach of the day did not. Finding optima in a complex space needs a more random approach, a point which is well recognised in machine learning. Elitism and lack of diversity are recognised to have downsides in evolutionary computation.

 

Funders should be in the role of helping researchers to realise their vision. Researchers have the ideas and the vision for what they want to achieve over 10 years say, funders should be helping them, linking them with other researchers, equipment, resources, supporting them bottom-up to achieve their vision.

(Funders do occasionally do the right thing, e.g. some networks are really good to help people to link up. But mostly funders waste researchers’ time writing proposals that won’t be funded (majority won’t be funded), Inventing requirements for the document, specifying all the detail you need to write into something that won’t happen anyway.)

 

Stepping back and looking at the whole research effort in UK/EU – it is very sub-optimal.

Governments could be helping researchers rather than creating nonsense priorities/strategies and requirements.

[1] Fortin JM, Currie DJ (2013) Big Science vs. Little Science: How Scientific Impact Scales with Funding. PLOS ONE 8(6): e65263. https://doi.org/10.1371/journal.pone.0065263

[2] Andrew Derrington. https://parkerderrington.com/wp-content/uploads/2020/01/Handout.pdf

[3] Andrew Derrington. https://parkerderrington.com/how-to-deal-with-rejection-1/

[4] Andrew Derrington. https://parkerderrington.com/how-often-should-you-write-research-grant-applications/