Conference on use of AI in research management hears that if used carefully, the technology could mitigate current biases

5 March 2024

Using artificial intelligence (AI) tools in research funding could allow science to break out of a cycle where only a narrow pool of researchers receive grants because they have been previously successful, a conference on the use of AI in research management has heard.

Suze Kundu, director of researcher and community engagement at Digital Science, told the ‘AI Day’ event — held by the European Association of Research Managers and Administrators in Brussels on February 29 — that research sometimes suffered from a “snowball effect” where grant winners had an advantage in winning future funding.

If AI could be used in the early stages of grant processing to identify researchers or novel methods that may not have received attention otherwise, these effects could be mitigated, she said.

During a panel discussion, Kundu said that this idea ran counter to the argument that using AI in research management was a risk because it might amplify certain biases in funding.

“Funding is one part of a very biased system that we’re working within already. You tend to award funding to people who have proven themselves to a certain level, published papers in certain journals,” she said.

Kundu added that although it would be exciting to see AI and machine learning identify under-the-radar researchers, projects or teams, funders would need to be “incredibly careful and considerate” in building AI models for such a purpose. “If we’re using training data from research information that exists, it’s so riddled with biases already that we run the risk of propagating some really damaging traits,” she said.

Kasper Nørgaard, scientific director at the Novo Nordisk Foundation, a private research funder based in Denmark, said his organization has been exploring various ways that AI might be used to aid the funding process.

One possibility he raised was the potential for AI to allow the foundation to give feedback to unsuccessful grant applicants, something it could not do presently as it is “extremely time-consuming to give qualified rejection letters to everybody”.

Nørgaard said they were now looking at whether a large language model could use data held by the foundation, such as comments from human peer reviewers, to provide applicants with “cohesive feedback” so they “know potentially what to improve for next time”.

He also said that it might be possible to use AI to text-mine past grant applications and spot untapped growth areas for future funding streams co-developed by the foundation and researchers. Nørgaard stressed that in any use of AI in research funding, “of course we need to have human oversight, always”. But “without testing and trying things out, we will not find the methods that actually work”.

Kundu agreed that using machine-learning methods to spot emerging trends in research and the scientists involved in novel approaches was potentially a huge benefit of AI in research management.

“At the moment, we are kind of limited by the people we have on evaluation panels, or even in the sifting process” for grant applications, she said. So, there could be a propensity to “overlook” an application if the language it used to describe a proposal was unfamiliar to someone with built-in expectations and knowledge of a field. But, by training AI on past application data, “we really could avoid that overlooking of these many underexplored pockets of research”.