Stefan Müller
Associate Professor
University College Dublin
I am an Associate Professor in the School of Politics and International Relations at University College Dublin. My research focuses on political representation, party competition, political communication, public opinion, and quantitative text analysis. My work has been published, among others, in the American Political Science Review, The Journal of Politics, the British Journal of Political Science, Political Communication, the European Journal of Political Research, and Political Science Research and Methods.
I lead two funded research projects. The first project assesses environmental and energy policies in comparative perspective. The project is embedded into the multidisciplinary energy research programme NexSys. The second project, funded by the Swiss National Science Foundation, analyses grant peer review reports using computational text analysis and machine learning.
I am a core member of the Connected_Politics Lab, co-author of the quanteda R package for quantitative text analysis, and maintainer of the Irish Polling Indicator. I was selected by the Royal Irish Academy to join the Young Academy Ireland. I established the Text and Policy Research Group and regularly contribute to national and international media on Irish and European politics.
Text and Policy Research Group
Publications
Google Scholar | ORCID | Scopus
Peer-Reviewed Journal Articles
2025. “Catalysts for Progress? Mapping Policy Insights From Energy Research.” Energy Research & Social Science 121: 103955 (with Brian Boyle, Yen-Chieh Liao, Sarah King, and Robin Rauner).PDF | Data and Code | Summary
Abstract
This article measures policy relevance in the abstracts of papers published between 2010 and 2023 in the top 100 journals covering energy research. Communicating the impact of research beyond academia is key to overcoming the evidence-policy divide. Yet, policy engagement is shaped by structural factors and poses unresolved dilemmas for researchers. Qualitative analyses of how research findings are presented in publications are inherently limited in scope, while simple search queries miss contributions that do not refer to ‘policy’ explicitly. Undertaking a large-scale bibliometric analysis, we use computational methods to evaluate over 270,000 abstracts by applying a carefully validated keyword-based dictionary approach. Overall, we find that 15 % of abstracts contain policy-relevant statements, with considerable differences among journals mentioning policy in their aims and scope. We also observe geographic variation by authorship and the funding agencies that sponsored research projects. Finally, we apply unsupervised topic models to identify distinct themes in policy-relevant abstracts. Our analysis reveals that the topics of renewable energy and implementation are most prevalent but have declined since 2010, while the focus on energy systems and emissions has gradually increased. These findings inform ongoing discussions about bridging the gap between research and policy impact in a field that will play a pivotal role in developing pathways to net zero.
PDF | Data and Code
Abstract
Do policy priorities that candidates emphasize during election campaigns predict their subsequent legislative activities? We study this question by assembling novel data on legislative leadership posts held by Japanese politicians and using a fine-tuned transformer-based machine learning model to classify policy areas in over 46,900 statements from 1270 candidate manifestos across five elections. We find that a higher emphasis on a policy issue increases the probability of securing a legislative post in the same area. This relationship remains consistent across multiple elections and persists even when accounting for candidates’ previous legislative leadership roles. We also discover greater congruence in distributive policy areas. Our findings indicate that campaigns provide meaningful signals of policy priorities.
PDF | Data and Code | PolNos Datasets | The Conversation
Abstract
Traditional research on political parties pays little attention to the temporal focus of communication. It usually concentrates on promises, issue attention, and policy positions. This lack of scholarly attention is surprising, given that voters respond to nostalgic rhetoric and may even adjust issue positions when policy is framed in nostalgic terms. This article presents a novel dataset, PolNos, which contains six text-based measures of nostalgic rhetoric in 1,648 party manifestos across 24 European democracies from 1946 to 2018. The measures combine dictionaries, word embeddings, sentiment approaches, and supervised machine learning. Our analysis yields a consistent result: nostalgia is most prevalent in manifestos of culturally conservative parties, notably Christian democratic, nationalist, and radical right parties. However, substantial variation remains regarding regional differences and whether nostalgia concerns the economy or culture. We discuss the implications and use of our dataset for studying political parties, party competition, and elections.
PDF | Data and Code
Abstract
Do private interests predict politicians’ rhetoric? Focusing on housing policy, we compare issue emphasis and positions of landlord politicians and politicians who do not own multiple properties. Ireland provides a unique opportunity to study legislating landlords’ behavior as housing has become one of the most important political issues. We construct a novel dataset of politicians’ homeownership status between 2013 and 2022, a period characterized by rising rent and property prices. We fine-tune a transformer-based machine learning model and apply text scaling and sentiment analysis to identify issue salience and positions on housing in over 870,000 tweets and parliamentary questions. Contrary to our expectations, landlord politicians do not avoid the topic of housing nor take different positions. We also find that government status does not influence this relationship. The results imply that private financial interests do not influence rhetoric on housing policy.
PDF | Data and Code | ECPR The Loop
Abstract
To achieve foreign policy goals and boost prestige, states try to influence how foreign publics perceive them. Particularly during crises, the imperative to mitigate a negative image may see states mobilize resources to change the global narrative. This paper investigates whether China’s ‘mask diplomacy’ efforts influenced portrayals of the country in the early days of the Covid-19 pandemic. We validate and apply a semi-supervised scaling method to 1.5 million English statements in newspapers around the world mentioning China and Covid-19. Multi-period difference-in-differences models reveal that media tone improved significantly after mask diplomacy engagement. Using its Covid-19 White Paper to determine China’s preferred external narratives, we also find that a country’s domestic media reproduced key terms more after the country received PRC support.
PDF | Data and Code | Nature Q&A
Abstract
The Journal Impact Factor is often used as a proxy measure for journal quality, but the empirical evidence is scarce. In particular, it is unclear how peer review characteristics for a journal relate to its impact factor. We analysed 10,000 peer review reports submitted to 1,644 biomedical journals with impact factors ranging from 0.21 to 74.7. Two researchers hand-coded sentences using categories of content related to the thoroughness of the review (Materials and Methods, Presentation and Reporting, Results and Discussion, Importance and Relevance) and helpfulness (Suggestion and Solution, Examples, Praise, Criticism). We fine-tuned and validated transformer machine learning language models to classify sentences. We then examined the association between the number and percentage of sentences addressing different content categories and 10 groups defined by the Journal Impact Factor. The median length of reviews increased with higher impact factor, from 185 words (group 1) to 387 words (group 10). The percentage of sentences addressing Materials and Methods was greater in the highest Journal Impact Factor journals than in the lowest Journal Impact Factor group. The results for Presentation and Reporting went in the opposite direction, with the highest Journal Impact Factor journals giving less emphasis to such content. For helpfulness, reviews for higher impact factor journals devoted relatively less attention to Suggestion and Solution than lower impact factor journals. In conclusion, peer review in journals with higher impact factors tends to be more thorough, particularly in addressing study methods while giving relatively less emphasis to presentation or suggesting solutions. Differences were modest and variability high, indicating that the Journal Impact Factor is a bad predictor of the quality of peer review of an individual manuscript.
PDF | Data and Code
Abstract
Many citizens support the involvement of experts in political decision-making, yet we know little about how citizens react to expert opinions. Bridging recent evidence on technocratic attitudes and deliberative democracy, we study citizen responses to experts during influential deliberative mini-publics. Combining automated speech transcription of over 380,000 spoken words and quantitative text analysis, we estimate the topic prevalence in all expert testimonials, Q&A sessions, and other agenda items in the Irish Citizens’ Assembly (2016–2018), one of the prime examples of impactful deliberative forums. We find that inputs of experts structure subsequent discussions but do not dominate them. This correlation persists with various measures of topic prevalence and is robust to several modelling approaches. We also find that participants tended to react less strongly to testimonials by female experts. These conditional effects should encourage organisers to invite experts with diverse backgrounds in order to enhance inclusive decision-making.
PDF | Data and Code | ECPR The Loop | Reuters
Abstract
The expectation that voters behave rationally has been challenged through studies suggesting that “irrelevant events” like natural disasters and sports results change voting behavior. We test the effect of irrelevant events by matching candidate-level election results from Irish general (1922–2020) and local elections (1942–2019) with games in the men’s Gaelic football and hurling championships, the most popular sports in Ireland. Although Irish citizens care deeply about sports, we fail to find any relationship between match results and support for incumbents or politicians of government parties. These findings hold when applying an “unexpected event during survey design” to two representative surveys. Our results contribute to the literature on political accountability and point to conditional effects of irrelevant events.
PDF | Data and Code
Abstract
This research report measures changes in China’s public diplomacy after a May 2021 collective study session of the Chinese Communist Party Politburo. The session examined the country’s global communications strategy and fuelled speculation about what might change in China’s external communications, particularly with regard to its “wolf warrior” diplomats. Combining hand-coding and quantitative text analysis, we develop and validate a measure of “wolf warrior diplomacy” rhetoric and apply it to over 200,000 tweets from nearly 200 institutional, media and diplomatic Twitter accounts. Using a difference-in-difference research design, we evaluate if the session led to a noticeable change in the tweets of diplomats based in OECD countries. After the announcement, PRC diplomats in the OECD moderated their tweets in comparison to non-OECD diplomats, but we do not detect a major re-orientation of PRC communication strategies. These findings have relevance for scholars of Chinese foreign policy, nationalism and public diplomacy.
PDF | Data and Code
Abstract
Lectures and seminars increasingly strive for continuous interactions between learners and the instructor. I study whether the communication program Slack contributes to these goals by analyzing daily activity statistics in methodological and project-based postgraduate courses at an Irish university. Both semester-long courses were taught online during the coronavirus pandemic (Covid-19) and in person. The quantitative analysis reveals three insights. First, students are active on Slack throughout the term. Second, students post messages in public channels and extensively use private channels and direct messages. Third, many students follow the conversations, ensuring transparent and fair communication between students and instructors. Open-ended responses suggest that Slack created “team spirit.” I conclude with five recommendations: students should sign up in the first week of term; create channels for different aspects of the module; explain when students can expect a response; encourage private conversations between students; and monitor activities regularly.
PDF | Data and Code
Abstract
The 2019 Swiss national elections were characterized by the unusual prominence of two issues, environment and gender, whereas two staples of Swiss politics, immigration and Europe, were less dominant compared to previous elections. We study how, in this context, the media and party agenda were linked to issue ownership. Specifically, we consider whether political parties that own an issue could lead the media agenda and the agenda of other parties. Our analysis relies on all tweets and press releases of major Swiss political parties from January to October 2019 and 37,225 newspaper articles published during the same period. Results show, first, that the agenda-setting capacity of parties was restricted to the gender issue, and second, that the link between issue ownership and agenda setting is ambiguous. These findings suggest that during election campaigns, agenda setting may be largely exogenous to both parties and media.
Best Paper Award, Manifesto Corpus Conference (2018)
PDF | Data and Code | JOP Blog
Abstract
Experiences from the past and present influence decision-making. Voting behavior at elections also involves retrospective and prospective considerations. Yet, we do not know the degree to which parties react to these considerations by emphasizing the past, present, and future. I posit that parties do not only make promises but face incentives to discuss the past and present. I also expect that incumbency status conditions emotive rhetoric across these temporal dimensions. Using supervised machine learning, I uncover the temporal rhetorical focus in 621 party manifestos published in nine countries between 1949 and 2017. Parties devote, on average, half of a manifesto to future promises, while the other half describes the past and present. I also show that statements on the past and present drive previously observed differences in sentiment between incumbents and opposition parties. The findings underscore how the temporal dimension of campaign communication enhances our understanding of party competition.
PDF | Data and Code
Abstract
What is the role of social media in political agenda setting? Digital platforms have reduced the gatekeeping power of traditional media and, potentially, they have increased the capacity of various kinds of actors to shape the agenda. We study this question in the Swiss context by examining the connections between three agendas: the traditional media agenda, the social media agenda of parties, and the social media agenda of politicians. Specifically, we validate and apply supervised machine learning classifiers to categorize 2.78 million articles published in 84 newspapers, 6,500 tweets posted on official party accounts, and 210,000 tweets posted by politicians on their own accounts from January 2018 until December 2019. We first use the classifier to measure the salience of the four most relevant issues of the period: the environment, Europe, gender equality, and immigration. Then, using a vector autoregression (VAR) approach, we analyze the relationship between the three agendas. Results show that not only do the traditional media agenda, the social media agenda of parties, and the social media agenda of politicians influence one another but, overall, no agenda leads the others more than it is led by them. There is one important exception: for the environment issue, the social media agenda of parties is more predictive of the traditional media agenda than vice-versa. These findings underscore how closely different agendas are tied together, but also show that advocacy campaigns may play an important role in both constraining and enabling parties to push their specific agendas.
PDF | Data and Code
Abstract
Can voters in multi-party systems predict which coalition will form the government with any degree of accuracy? To date, studies which explore voter expectations of coalition formation have emphasized individual level attributes, such as education, but the complexity of the environment at the time the coalitions are forming should also be consequential in enabling (or handicapping) voters in forming expectations. We examine the relative effects of individual level attributes (e.g., education, cognitive mobilization) versus contextual factors (e.g., information availability) in 19 German state elections and 3 German general elections between 2009 and 2017. We find that the ease of identifiability of alternative future governments varies significantly across multi-party systems. We find that respondents are more likely to predict governments that they would like to see in office, that have a higher probability of receiving a majority of seats, and that consist of ideologically proximate parties. Combining survey data with a novel indicator of coalition signals, measured through a quantitative text analysis of newspaper coverage, we also find that voters consider positive pre-election coalition signals when predicting the government. Finally, we find that the information environment is much more relevant for correct coalition predictions than individual-level characteristics of respondents. Although individual attributes do influence predictive ability, these factors are strongly dominated by the context in which the prediction is taking place. The information environment has by far the largest effect on predicting coalition outcomes. Our results have implications for the literature on strategic voting in multi-party settings, as well as the literature on accountability.
PDF | Data and Code
Abstract
The relationship between digital technology and politics is an important phenomenon that remains poorly understood due to several structural problems. A key issue is the lack of adequate research infrastructures or the lack of access. This article discusses the challenges many social scientists face and presents the infrastructure we built in Switzerland to overcome them, using COVID-19 as an example. We conclude by discussing seven lessons we learned: automatization is key; avoid data hoarding; outsource some parts of the infrastructure but not others; focus on substantive questions; share data in the context of collaborations; engage in targeted public outreach; and collaboration is more promising than competition. We hope that our experience is helpful to other researchers pursuing similar goals.
Walter Lippmann Best Article of the Year Award, APSA Political Communication Section (2022)
PDF | Data and Code | Interview
Abstract
Voters evaluate politicians not just by what they say, but also how they say it, via facial displays of emotions and vocal pitch. Candidate characteristics can shape how leaders use—and how voters react to—nonverbal cues. Drawing on role congruity expectations, we study how the use of and reactions to facial, vocal, and textual communication in political debates varies by candidate gender. Relying on full-length videos of four German federal election debates (2005–2017) and a minor party debate, we use video, audio, and text data to measure candidate facial displays of emotion, vocal pitch, and speech sentiment. Consistent with our expectations, Angela Merkel expresses less anger than her male opponents, but she is just as emotive in other respects. Combining these measures of emotional expression with continuous responses recorded by live audiences, we find that voters punish Merkel for anger displays and reward her happiness and general emotional displays.
Best Paper Award, PSAI Annual Conference (Elizabeth Meehan Prize, 2021)
PDF | Data and Code | Podcast
Abstract
The Irish party system has been an outlier in comparative politics. Ireland never had a left-right divide in parliament, and for decades, the dominant centrist political parties competed around a centre-right policy agenda. The absence of an explicit left-right divide in party competition suggested that Irish voters, on average, occupy centre-right policy preferences. Combining survey data since 1973 and all Irish election studies between 2002 and 2020, we show that the average Irish voter now leans to the centre-left. We also show that income has recently emerged as a predictor of left-right self-placement, and that left-right positions increasingly structure vote choice. These patterns hold when using policy preferences on taxes, spending, and government interventions to reduce inequality as alternative indicators. We outline potential explanations for this leftward shift, and conclude that these developments might be anchored in economic inequalities and the left populist strategies of Sinn Féin.
PDF | Data and Code
Abstract
Do candidates in local elections benefit from an incumbency advantage? And which factors moderate the strength of this incumbency bonus? Analyzing seven decades of Irish local elections (1942–2019) conducted under proportional representation through the single transferable vote, we reassess and extend the mixed evidence on the incumbency advantage under proportional representation and in second-order elections. By applying the Regression Discontinuity Design, we find that the incumbency advantage is at least as strong in Irish local as in general elections, which are conducted under the identical electoral system. We also show that marginally elected candidates in local elections have much higher reelection probabilities when they do not face a high-quality candidate in their local electoral area after getting elected. The findings point to the importance of name recognition as a major driver of the incumbency advantage in local elections.
PDF | Data and Code
Abstract
We study the role of social media in debates regarding two policy responses to COVID-19 in Switzerland: face-mask rules and contact-tracing apps. We use a dictionary classifier to categorize 612’177 tweets by parties, politicians, and the public as well as 441’458 articles published in 76 newspapers between February and August 2020. We distinguish between “problem” (COVID-19) and “solutions” (face masks and contact-tracing apps) and, using a vector autoregression approach, we analyze the relationship between their salience on social and traditional media, as well as among different groups on social media. We find that overall attention to COVID-19 was not driven by endogenous dynamics between the different actors. By contrast, the debate on face masks was led by the attentive public and by politicians, whereas parties and newspapers followed. The results illustrate how social media challenge the capacity of party and media elites to craft a consensus regarding the appropriateness of different measures as responses to a major crisis.
PDF | Data and Code
Abstract
Previous studies conclude that governments fulfill a large share of their campaign pledges. However, only a minority of voters believe that politicians try to keep their promises, and many voters struggle to recall the fulfillment or breaking of salient campaign pledges accurately. I argue that this disparity between the public perception and empirical evidence is influenced by the information voters receive throughout the electoral cycle. I expect that the media extensively inform readers about political promises. In addition, I posit that news outlets focus more on broken than on fulfilled promises and that the focus on broken promises has increased over time. I find strong support for these expectations based on a new text corpus of over 430,000 statements on political commitments published between 1979 and 2017 in 22 newspapers during 33 electoral cycles in Australia, Canada, Ireland, and the United Kingdom. Newspapers inform voters regularly about announced, broken, and fulfilled promises. Yet, across the four countries, newspapers report at least twice as much on broken than on fulfilled promises. Moreover, this negativity bias in reports on political promises has increased substantively. The results have implications for studying campaign promises, negative information in mass media, and the linkages between voters and parties.
PDF | Data and Code | LSE: EUROPP
Abstract
Does government party support decline in a monotonic fashion throughout the legislative cycle or do we observe a u-shaped “electoral cycle effect”? Moving beyond the study of midterm election results, this is the first comparative study to assess the cyclical pulse of government party support in parliamentary democracies based on voting intention polls from 171 cycles in 22 countries. On average, government parties lose support during the first half of the electoral cycle, but at most partially recover from their initial losses. Under single-party government and when prime ministers control cabinet dissolution, support tends to follow the previously assumed u-shaped pattern more strongly. Finally, we find that government parties hardly recover from early losses since the 2000s.
PDF | Data and Code
Abstract
Can voters learn meaningful information about candidates from their electoral campaigns? As with job market hiring, voters, like employers, cannot know the productivity of candidates, especially challengers, when they elect them. The real productivity of representatives only reveals itself after the election. We explore if the information revealed during the “hiring process” is a good signal of the legislative effort of elected representatives. In the incomplete information environment of election campaigns, candidates should turn to credible signals to indicate their “type” to voters. Campaigns—and campaigning—are means by which candidates can, in principle, signal their motivations to voters. Is a candidate’s behavior on the campaign trail informative about his or her behavior and effort as a legislator? Does it, for example, reveal whether a candidate will be more hard working and legislatively active? Using evidence from the European Parliament, we show that campaign activity prior to the election is not related to policy-seeking behavior in the legislature post-election. The finding also holds in two national-level settings and across a variety of measures of legislative effort. Those who campaign harder do seem more likely to win the election, but campaign effort seems to provide a poor guide to what the winner does once elected.
PDF | Data and Code
Abstract
Which voters prefer having more choice between parties and candidates in an election? To provide an answer to this question, we analyse the case of a radical change from a closed-list PR system to a highly complex open-list PR system with cumulative voting in the German states of Bremen and Hamburg. We argue that the approval of a personalised electoral system is structured in similar ways as support for direct democracy. Using representative surveys conducted prior to all four state elections under cumulative voting in 2011 and 2015, we analyse which individual factors determine the approval, disapproval, or indifference towards the new electoral law. The results indicate that younger voters as well as supporters of left parties are much more likely to support a personalised electoral system. In contrast to previous studies, political interest only has an impact on the indifference towards the electoral system. More generally, our results show that a large proportion of voters does not appreciate personalised preferential electoral systems which seems to be a result of the complexity and magnitude of choice between parties and candidates.
Society for Political Methodology Statistical Software Award (2020)
PDF | Software Repository
Abstract
quanteda is an R package providing a comprehensive workflow and toolkit for natural language processing tasks such as corpus management, tokenization, analysis, and visualization. It has extensive functions for applying dictionary analysis, exploring texts using keywords-in-context, computing document and feature similarities, and discovering multi-word expressions through collocation scoring. Based entirely on sparse operations, it provides highly efficient methods for compiling document-feature matrices and for manipulating these or using them in further quantitative analysis. Using C++ and multi-threading extensively, quanteda is also considerably faster and more efficient than other R and Python packages in processing large textual data. The package is designed for R users needing to apply natural language processing to texts, from documents to final analysis. Its capabilities match or exceed those provided in many end-user software applications, many of which are expensive and not open source. The package is therefore of great benefit to researchers, students, and other analysts with fewer financial resources. While using quanteda requires R programming knowledge, its API is designed to enable powerful, efficient analysis with a minimum of steps. By emphasizing consistent design, furthermore, quanteda lowers the barriers to learning and using NLP and quantitative text analysis even for proficient R programmers.
Abstract
Many electoral systems constrain voters to one or two votes at election time. Reformers often see this as a failing because voters’ preferences are both broader and more varied than the number of choices allowed. New electoral systems therefore often permit more preferences to be expressed. In this paper we examine what happens when cumulative voting is introduced in two German states. Even when we allow for tactical considerations, we find that the principle of unconstrained choice is not widely embraced by voters, although in practice, too, many seem to have preferences for more than just one party. This finding has implications for arguments relating to electoral reform as well as how to conceive of party affiliations in multi-party systems.
PDF | Data and Code
Abstract
The home advantage in various sports has been well documented. So far, we lack knowledge whether playing in neutral venues indeed removes many, if not all, theoretically assumed advantages of playing at home. Analysing over 3,500 senior men’s Gaelic football and hurling matches – field games with the highest participation rates in Ireland – between 2009 and 2018, we test the potential moderating influence of neutral venues. In hurling and Gaelic football, a considerable share of matches is played at neutral venues. We test the influence of neutral venues based on descriptive statistics, and multilevel logistic and multinomial regressions controlling for team strength, the importance of the match, the year, and the sport. With predicted probabilities ranging between 0.8 and 0.9, the favourite team is very likely to win home matches. The predicted probability drops below 0.6 for away matches. At neutral venues, the favourite team has a predicted probability of winning of 0.7. A Coarsened Exact Matching (CEM) approach also reveals very substantive and significant effects for the “treatment” of neutral venues. Overall, neutral venues appear to be an under-utilised option for creating fairer and less predictable competition, especially in single-game knock-out matches.
Book Chapters
Forthcoming. “What the 2024 Party Manifestos Reveal: Issue Salience and Left-Right Positions” (with Gail McElroy), in: Michael Gallagher, Theresa Reidy, and Eoin O’Malley (eds.): How Ireland Voted 2024: The New Normal? Palgrave Macmillan.
Other Publications
2025. Narrative CV Toolkit: Resources and Examples: Young Academy Ireland (with Kevin G Daly and Fabrice Kämpfen).
PDF
2023. Quanteda Tutorials (with Kohei Watanabe).
Website
Datasets
2025. Irish Polling Indicator Datasets (with Tom Louwerse).
Stable Version (1987–2024) | Development Version (2025)
2024. Irish Politics Data: An Overview of Datasets Related to Irish Politics.
Website
2024. Irish Local and European Parliament Election Manifestos (with Artur Baranov and Paula Montano).
Dashboard | Data
2024. Irish Demographic Polling Datasets (with Thomas Pluck and Paula Montano).
Dashboard | Data
Current Research
Under Review
“A Supervised Machine Learning Approach for Assessing Grant Peer Review Reports.” Quantitative Science Studies: revise & resubmit (with Gabriel Okasa, Alberto de León, Michaela Strinzel, Anne Jorstad, Katrin Milzow, and Matthias Egger).Preprint (PDF) | Classifiers | Code | Data Management Plan
Abstract
Peer review in grant evaluation informs funding decisions, but the contents of peer review reports are rarely analyzed. In this work, we develop a thoroughly tested pipeline to analyze the texts of grant peer review reports using methods from applied Natural Language Processing (NLP) and machine learning. We start by developing twelve categories reflecting content of grant peer review reports that are of interest to research funders. This is followed by multiple human annotators’ iterative annotation of these categories in a novel text corpus of grant peer review reports submitted to the Swiss National Science Foundation. After validating the human annotation, we use the annotated texts to fine-tune pre-trained transformer models to classify these categories at scale, while conducting several robustness and validation checks. Our results show that many categories can be reliably identified by human annotators and machine learning approaches. However, the choice of text classification approach considerably influences the classification performance. We also find a high correspondence between out-of-sample classification performance and human annotators’ perceived difficulty in identifying categories. Our results and publicly available fine-tuned transformer models will allow researchers and research funders and anybody interested in peer review to examine and report on the contents of these reports in a structured manner. Ultimately, we hope our approach can contribute to ensuring the quality and trustworthiness of grant peer review.
“Mapping Digital Campaign Strategies: How Political Candidates Use Social Media to Communicate Constituency Connection and Policy Stance.” Computational Communication Research: revise & resubmit (with James P Cross, Derek Greene, and Martijn Schoonvelde).
Working Papers
“Gender and Discipline Shape Length, Content and Tone of Grant Peer Review Reports” (with Gabriel Okasa, Michaela Strinzel, Anne Jorstad, Katrin Milzow, and Matthias Egger).Preprint (PDF) | Classifiers | Code | Data Management Plan
Abstract
Peer review by experts is central to the evaluation of grant proposals, but little is known about how gender and disciplinary differences shape the content and tone of grant peer review reports. We analyzed 39,280 review reports submitted to the Swiss National Science Foundation between 2016 and 2023, covering 11,385 proposals for project funding across 21 disciplines from the Social Sciences and Humanities (SSH), Life Sciences (LS), and Mathematics, Informatics, Natural Sciences, and Technology (MINT). Using supervised machine learning, we classified over 1.3 million sentences by evaluation criteria and sentiment. Reviews in SSH were significantly longer and more critical, with less focus on the applicant’s track record, while those in MINT were more concise and positive, with a higher focus on the track record, as compared to those in LS. Compared to male reviewers, female reviewers write longer reviews that more closely align with the evaluation criteria and express more positive sentiments. Female applicants tend to receive reviews with slightly more positive sentiment than male applicants. Gender and disciplinary culture influence how grant proposals are reviewed—shaping the tone, length, and focus of peer review reports. These differences have important implications for fairness and consistency in research funding.
“From Manifesto to Mandate: Tracking Environmental Policy Evolution with Language Models” (with Robin Rauner, Cara Augustenborg, and Isabelle Hardgrave).
“Selecting and Validating Classifiers for Multilingual and Cross-Domain Stance Detection” (with Yen-Chieh Liao).
“Do Legislators Learn how to be Legislators? The Life Cycle to Parliamentary Rhetoric” (with Shaun Bowler, Gail McElroy, and Jihed Ncib).
If you would like to get access to the latest version of a paper, feel free to send me an e-mail.
Projects
Assessing and Explaining Environmental and Energy Policies in Comparative Perspective
Project Summary: Political parties, politicians, companies, and interest groups increasingly discuss how to achieve a net-zero carbon emissions future, but systematic evidence that tracks these political debates is still lacking. The project “Assessing and Explaining Environmental and Energy Policies in Comparative Perspective” seeks to identify the problems political actors raise and solutions they offer regarding renewable energy, sustainability, and water treatment. The project will also assess how companies and interest groups aim to reduce greenhouse gas emissions and help mitigate the impacts of climate change. By combining quantitative text analysis, human coding, and supervised machine learning, it will define and map (proposed) policies relating to the environment and sustainability, and provide recommendations for policymakers.
The project is part of NexSys, an All Island SFI Strategic Partnership Programme. NexSys focuses on the transition to a net zero carbon energy system. It is a unique partnership bringing together a multidisciplinary research team, industry, and policymakers to tackle fundamental research questions to be addressed as part of the transition to net Zero. Hosted by UCD Energy Institute, NexSys brings together academics from nine institutions across the Island of Ireland (University College Dublin, Trinity College Dublin, Dublin City University, ESRI, Maynooth University, University College Cork, NUI Galway, Ulster University and Queen’s University Belfast) to work together to meet the unprecedented scale and complexity of the challenges associated with the energy transition.
Funding Volume: €183,018 (total funding: €16,000,000)
Interviews, Reports, and Outputs:
- ‘Researcher Story’ with Dr Stefan Müller
- Blog post on the NexSys website about our project.
- Report and case study about NexSys and its industry partners.
Analysing Grant Peer Review Reports Using Machine Learning
Project Summary: Peer review plays an essential role in grant evaluation. External peer review reports by international experts contribute to assessing the feasibility and quality of grant applications and provide an essential basis for funding decisions. In addition, they help justify rejections and provide feedback, which may help applicants improve their research. Peer review thus has the power to influence which researchers and what kind of research receives funding and can subsequently be conducted. For funding organisations, peer review must fulfil these functions. Peer review reports should also be in line with their understanding of quality. Peer review should also enable fair, transparent, and efficient funding decisions and foster diversity in research (ideas, methodologies, and approaches) and researchers.
This research project is a collaboration between University College Dublin and the Swiss National Science Foundation. The project will analyse the texts of anonymised grant review reports along several dimensions using human coding and machine learning. We seek to conceptualise characteristics of grant peer review reports and classify a large corpus of review reports. The project investigates whether strategic initiatives and new evaluation procedures have the desired effects on the content and structure of review reports.
Funding Volume: €276,099
Team
Current Team Members
- Sarah King: PhD Researcher (Funding: UCD Iseult Honohan Doctoral Scholarship)
- Mafalda Zúquete: PhD Researcher (Funding: Portuguese Foundation for Science and Technology)
- Robin Rauner: Research Scientist (Funding: NexSys – Science Foundation Ireland)
Former Team Members
- Brian Boyle, PhD: Lecturer with tenure at Newcastle University
- Alberto de León, PhD: Postdoctoral Researcher at Universidad Carlos III de Madrid
- Yen-Chieh Liao, PhD: Research Fellow at the University of Birmingham
- Jihed Ncib, PhD: Postdoctoral Researcher at University College Dublin, School of Computer Science
You find more information about the team and our current projects on the website of the Text and Policy Research Group.
Teaching
Syllabi of all modules semester-long modules I have taught between 2019 and 2025 are available in this public GitHub Repository.
Module Instructor: Undergraduate Level
Understanding Government: Theory and Reality (University College Dublin: Autumn 2025)
Parties and Party Competition (University College Dublin: Spring 2020, Autumn 2021, Spring 2023).
SyllabusRepresentation and Party Competition (University of Zurich: Autumn 2019, Spring 2020).
SyllabusHope (Discovery Module; coordinated by Imelda Maher, UCD Sutherland School of Law: Spring 2023, Spring 2024, Spring 2025)
Module Instructor: Postgraduate Level
Quantitative Text Analysis (University of Zurich: Spring 2019, Autumn 2019; University College Dublin: Spring 2022, Spring 2023, Spring 2024, Spring 2025).
SyllabusApplied Data Wrangling and Visualisation (University College Dublin: Autumn 2024)
SyllabusIntroduction to Statistics (University College Dublin: Autumn 2020, Autumn 2021, Autumn 2022).
SyllabusConnected_Politics (University College Dublin; Spring 2021, Spring 2022).
SyllabusPolitical Representation and Policy Preferences (University of Zurich: Autumn 2019, Spring 2020).
Syllabus
I maintain and continuously update a GitHub repository with the syllabi of all modules.
Workshop Instructor
Quantitative Text Analysis for Beginners: University of Bergen (2023, 2019); UCD Geary Institute for Public Policy (2022); University of Lucerne (2021); University of Hamburg (2021); Scottish Graduate School of Social Science (2020); University of Bremen (2020, 2018); COMPTEXT (2020: Innsbruck; 2019: Tokyo); Swiss National Science Foundation (2019); Kobe University (2019); University of Düsseldorf (2019); Trinity College Dublin (2018); WZB Berlin (2018)
Creating Academic Personal Websites: Geneva Graduate Institute (2022); University of Lucerne (2022); University College Dublin (2020)
Data Wrangling and Visualisation: University College Dublin (2020); Trinity College Dublin (2017)
Reproducible Research with Git and GitHub: University College Dublin (2023, 2021)
I could also teach these workshops at your institution. Do not hesitate to contact me.
Teaching and Supervision Qualifications
- Research Supervisor Support and Development Programme (RSSDp), University College Dublin (2023)
- Professional Certificate in University Teaching & Learning, University College Dublin (2021–2022)
- Certificate in Academic Teaching & Supporting Learning, Centre for Academic Practice and eLearning, Trinity College Dublin (2016)
Teaching Awards
- University Teaching Excellence Award, University College Dublin (2023)
- College Teaching Excellence Award, University College Dublin, College of Social Sciences and Law (2022)
- Teaching and Learning Prize, Political Studies Association of Ireland (2022)
- Dermot McAleese Award for Teaching Excellence, Trinity College Dublin (2018)
Consulting
I provide consulting services through my affiliation with ConsultUCD, UCD’s Managed Consultancy Service, offering expertise in text analysis, large language models (LLMs), and data visualisation. My work spans academia, industry, and public engagement, with a focus on making complex analytical tools both accessible and effective.
Bespoke Workshops and Training
I design and deliver tailored workshops on applied data visualisation, text analysis, LLMs, and large-scale survey data analysis. These workshops cater to researchers, businesses, and public sector organisations looking to harness cutting-edge computational methods.
Implementation of Local LLMs
I support organisations in deploying local LLMs to analyse open-ended text responses and longer documents. My work with the Swiss pollster LeeWas has focused on developing AI-driven text analysis solutions while protecting sensitive data. I have worked with Ireland Thinks on using LLMs to reliably analyse open-ended survey responses at scale. Our analyses have been published in the Sunday Independent, covering views on cabinet reshuffles and referendums.
Get in Touch
If you need customised training on data visualisation, quantitative text analysis, the analysis of survey data, or support in deploying local LLMs, please contact me or ConsultUCD.
Contact
BlueSky (@stefanmueller.bsky.social)
Room G312, Newman Building, University College Dublin, Belfield, Dublin 4, Ireland
Office Hours: Thursday, 12:30–14:00 (Schedule a meeting via Calendly)