Should there be a handicapping system when it comes to the allocation of research funds?Handicapping is a fixture in the world of sport, most prominently in the disciplines of golf and horse-racing. Whether it’s subtracting the number of strokes needed to get around a golf course, or adding weight to a champion horse, the principle is the same: it allows competition on an approximately level footing despite differences in experience or prior performance, and thereby equalises the chances of winning.

Would science funding also benefit from a handicap system? When allocating grant money, funding panels typically consider the past performance of the applicant in terms of publications, the likely success of the project based on the proposal and preliminary data, and the extent to which the local environment is likely to help or hinder that outcome.

“Success” is of course a very relative term – research projects often go in unexpected directions, so a likelihood of “success” essentially means the chances that the investment made by the funding body will realise publications similar to what is being proposed, not necessarily that 100% of what the applicant is proposing will be completed.

The problem though is that writing a good proposal and delivering on it are two different things, so there’s an inherent risk in the investment being made. How do you minimise that risk? Easy. Go with experience.

Trust people with a strong track record. Trust people based in famous institutions. In other words, trust the Matthew Effect. And funders do. 

That’s not to say they’re ignorant of the problem. It’s intuitively obvious that if you’re a junior group leader and trying to get started, the odds are stacked against you. Many funding bodies have consequently introduced schemes with strict eligibility cutoffs on the basis of experience in order to favour younger applicants, although they have their own set of problems.

As we’ve pointed out here already, if you’ve done a 6-year PhD and a 2-year postdoc, you are actually, in funding terms, 2 years younger than someone who did a 4-year PhD and a 4-year postdoc. For the current generation of young scientists, many of whom have had to endure longer-than-planned postdoctoral periods simply due to a lack of independent positions being available, those criteria may already invalidate their applications.

Regardless, sooner or later young scientists are going to end up applying for funds and competing with established groups that have been around a long time. And here’s where there’s a key disadvantage. Because when assessing track record, one element never shows up on those glittering publication lists testifying to exemplary past performance: how much money has been expended to generate them.

In other words, a large group of twenty people that publishes one high-impact paper a year (0.05 papers per capita per annum) is judged to be far more active and productive than a group of three people that publishes one high-impact paper every four years (0.08 papers per capita per annum). And as grants typically require annual reports and will run for 3-5 years, a small group is inherently disadvantaged when it comes to demonstrating value, even if they are achieving much greater value for money.

Why? Because the single most expensive thing in research is people – either direct employees within the group (PhD students, postdocs, technicians) or indirect employees in research facilities (paying a facility means you’re essentially hiring its staff to work for you). So the more money you have, the more people you can use, and the more you will be able to publish. But what won’t show up is how much bang you’re actually generating for every buck. As long as you have a large group and ample funding, you can be shockingly inefficient but still be hailed as a role-model – even if many of the people in your group leave without a first-author publication or quit prematurely.

Wayne Wahls recently and brilliantly pointed out the gross inequities in allocations of NIH research funding, and how even a modest redress could transform career prospects for an entire generation of young scientists. Papers per capita and papers per taxpayer dollar are more or less the same thing, so discouraging skewed funding distributions should increase overall efficiency. 

For young group leaders, collaboration is of course one way around this problem. It’s a way of increasing the size of your team without increasing your direct expenditure. But young group leaders who collaborate extensively can often be accused of lacking independence, especially if they are interacting with former mentors. And good collaborations are not always easy to find, or sustain. Also, collaboration networks are personal networks, so who you’re collaborating with may play a larger role than the collaboration’s actual utility when it comes to assessment. 

Would it be unfair to start handicapping larger groups with a diverse research portfolio? After all, not everybody in the group will necessarily be working on, or being funded by, grants from the research area that’s being targeted. Probably not. Everyone knows that the fundamental unit of research is the group, and it’s the group’s output that should be measured. If a group is up against a submission deadline or racing to beat a competitor, it’s not unheard of for everybody to chip in, regardless of what their primary projects are.

In this age of metrics, one of the simplest ways of providing a level playing field would be to require grant applicants to disclose their total income, their total number of personnel, and their total publications within a given time window – in other words, specify their publications per capita per taxpayer dollar for a particular time window. This would show who’s using taxpayer funds most efficiently. 

Would there be a cheeky work-around for those large groups to escape being penalised in such a system? Of course – make the postdocs in the group the primary applicants for research funds, rather than the group leaders. But this is no disadvantage at all. Rather, it would empower the postdocs, as obtaining a grant in their own name would be a big stepping step on the way to an independent position. And it wouldn’t explicitly bias against established groups either – if they were making efficient use of taxpayer money, then their per capita output would demonstrate exactly how fiscally responsible with the taxpayers’ funds they had been.

Enforcing disclosure of total funds and total group size alongside total publications wouldn’t strictly be a handicap system because it’s not actually penalising anybody, just adding a level of transparency that doesn’t currently exist. But it would almost certainly level up the playing field, and equalise the chances of success. Time to tee off.