Preprint analysing automated scoring of applications for UKRI funding comes amid reports of AI-based rejections
Artificial intelligence could be used to filter out weaker proposals for funding, a study has suggested, after finding that large language models ranked applications broadly in line with expert reviewers.
The paper comes amid reports of researchers being rejected for grant applications after “AI-based triage”.
Published as a preprint on 21 September, so yet to go through peer review itself, the study compares scores given by human reviewers to 2,200 recent grant applications to the Economic and Social Research Council and the Engineering and Physical Sciences Research Council, with scores generated by six LLMs. The study was funded by the ESRC.
It found that while LLM scores are individually inaccurate, the average rankings they yield correlated with expert average scores, suggesting that LLMs “may usefully play a role in contemporary funding decisions”. It says LLMs could help triage weaker proposals, replace one human reviewer, check for bias or flag proposals for further scrutiny.
“Whilst they may not be accurate enough to play an important role…they might still be used in situations where higher accuracy is not available, or when efficiency can be improved without compromising the overall outcome,” the study says.
AI rejections reported
The results come shortly after UK Research and Innovation said it would begin “exploring the safe use of AI-assisted assessment” as part of efforts to reduce grant processing times, against a backdrop of rising volumes of applications.
In recent days, there have also been several reports on social media of applicants seemingly being rejected after “AI-based triage”. The funding, while originating from UKRI, is reported to have been devolved so the triage process was not carried out by the funder. Research Professional News has approached UKRI for clarification.
The preprint study notes a swathe of implications from adopting AI in the grant review process, including that funders may “need to reassure their stakeholders” with academics reluctant to allow AI to evaluate them.
Mike Thelwall, a professor of data science at the University of Sheffield and lead author of the study, told RPN: “I worry about too much AI influence on research decisions.
“But I also think the increasing volume of submissions for grants with sophisticated AI systems for proposal writing seem to be making the current system unsustainable, so it seems important to carefully and reflexively try alternative solutions,” Thelwall said.
Responsible use in research
A separate study, published on 23 September by the Innovation and Research Caucus, suggested UKRI should assess not only whether researchers use AI but how responsibly they use it.
The study, also funded by ESRC, analysed more than 3,000 UKRI-funded AI-related social science projects and found responsible AI concerns were “present but unevenly distributed” across the funder’s portfolio.
Jack Stilgoe, a professor of science and technology studies at University College London, told RPN it is “great to see” that some researchers are embracing responsible innovation approaches.
“Progress on this will always be uneven,” he said. “I think we need to keep building the evidence that good, trustworthy science should be underpinned by a robust discussion of all the questions—ethical, epistemological, political—raised by AI as new a research tool.
“AI is forcing some of these questions into the open, and researchers are used to having to adapt to new tools,” Stilgoe said.
A recent survey also found that public support for UK taxpayer-funded R&D drops dramatically when generative artificial intelligence is involved.
Leave a Reply