European agencies reform research assessment to prioritize quality

A study of 10 funding agencies highlights growing adoption of qualitative criteria and the discontinuation of metrics such as journal impact factors

Editorial staff

Textured watercolor-style illustration showing researchers walking through a valley where large stone monoliths engraved with numbers and bar chart symbols stand covered in moss and vegetation, like abandoned ruins of the past. The researchers follow a path toward golden light on the horizon. In the blue sky float abstract elements—atomic orbital diagrams, open books with visible pages, and a hand outstretched reaching toward another, symbolizing collaboration and shared knowledge.

Discussions on reforming research assessment have gained momentum in recent years, driven by the advance of open science and responsible assessment practices. Across different countries, institutions, and academic fields, there is growing support for greater transparency, equity, and inclusion, as well as for more qualitative forms of assessment supported by—rather than dominated by—quantitative indicators. This is one of the conclusions of the report “Emerging Practices in Research Assessment: Cross-Case Analysis of 10 International Funders,” prepared by researchers at Aalborg University in Denmark on the occasion of the European Union Presidency High-Level Conference on the Reform of Research Assessment (CeRRA), held in Copenhagen in December.

Qualitative review surpasses traditional metrics

The report compiles case studies of European public and private research funding agencies, independent philanthropic organizations, and mission-oriented institutions. It draws on public documents such as strategic plans, guidelines, calls for proposals, application forms, and policy statements, as well as interviews with funding managers and additional background research on each funding entity. The aim was to map how these institutions structure and apply their assessment models, identify points of convergence and divergence, and situate their practices within the global debate on responsible evaluation of research.

The analyses reveal both shared trends and distinct approaches to responsible assessment practices and principles. All institutions maintain peer review and qualitative expert assessment as the primary mechanisms for evaluating funding proposals. The structure varies from individual reviews to multi-stage processes involving evaluation panels. In all models, qualitative judgment remains central. Some funders, such as the United Kingdom’s Wellcome Trust and the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation), explicitly discourage the use of journal-based metrics such as the impact factor (which measures how often a journal’s articles are cited) and the h-index (which quantifies a researcher’s output and citations).

The criticism is that these quantitative indicators fail to capture the true quality of research and may distort scientific priorities.

Alignment with international reform initiatives also varies among funders. The Wellcome Trust, the DFG, the Volkswagen Foundation, the Independent Research Fund Denmark and the Villum Fonden demonstrate explicit engagement with the Coalition for Advancing Research Assessment (CoARA) and the San Francisco Declaration on Research Assessment (DORA), which in 2012 called for elimination of the impact factor as an indicator of article quality.

Several Danish funders, while recognizing the importance of research quality and societal impact, do not formally participate in CoARA. This is true for the Novo Nordisk Foundation and the Augustinus Foundation, which nevertheless incorporate elements of responsible assessment into their processes, such as qualitative review and recognition of interdisciplinary and collaborative contributions.

Diversity and equity gain ground in policies

The Aalborg University study also identified growing attention among these institutions to diversity, equity, and inclusion. Some agencies have adopted structured gender-balance rules for their evaluation panels, including Sweden’s Riksbankens Jubileumsfond, the Wellcome Trust, and the Volkswagen Foundation. Others monitor diversity indicators or incorporate them into institutional strategies, as seen with the Villum Fonden, the Velux Foundation, the Novo Nordisk Foundation, the DFG, and the Independent Research Fund Denmark. Some smaller or mission-orientated foundations, however, only mention diversity and inclusion to a limited extent in their public guidelines.

A small group of funders remains largely agnostic about the future of responsible assessment. Most of the institutions analyzed, however, have active policies in this area and are working to promote more equitable, open, and collaborative research cultures. The resulting ecosystem is heterogeneous but shows convergence around common norms, reflecting different institutional histories, mandates, and strategic priorities. “By bringing together these case studies within a single analytical framework, we sought to support discussions on the reform of research assessment in Denmark and across Europe, providing references for continued learning and contributing to the emerging debate on the role of funders in shaping the research ecosystem,” the authors wrote.

* This article may be republished online under the CC-BY-NC-ND Creative Commons license.
The text must not be edited and the author(s) and source (Science Arena) must be credited.