Analysing Grant Peer Review Reports Using Machine Learning

Peer review plays an essential role in grant evaluation. External peer review reports by international experts contribute to assessing the feasibility and quality of grant applications and provide an essential basis for funding decisions. In addition, they help justify rejections and provide feedback, which may help applicants improve their research. Peer review thus has the power to influence which researchers and what kind of research receives funding and can subsequently be conducted. For funding organisations, peer review must fulfil these functions. Peer review reports should also be in line with their understanding of quality. Peer review should also enable fair, transparent, and efficient funding decisions and foster diversity in research (ideas, methodologies, and approaches) and researchers.

This research project is a collaboration between University College Dublin and the Swiss National Science Foundation (opens in new tab). The project will analyse the texts of anonymised grant review reports along several dimensions using human coding and machine learning. We seek to conceptualise characteristics of grant peer review reports and classify a large corpus of review reports. The project investigates whether strategic initiatives and new evaluation procedures have the desired effects on the content and structure of review reports.