REF 2028: First reflections on the FRAP review

Today saw the publication of the eagerly anticipated ‘initial decisions’ of the Future Research Assessment Programme. The result of extensive consultation and deliberation, it lays out the proposed shape of the next Research Assessment Framework (REF).

There are some very important developments, many of them aligned with the consultation response the NCCPE developed with the public engagement community in 2022. They mark a very significant shift in how the relationship between research, engagement and culture is understood.

Reframing the narrative

So what are the big shifts? We have identified three significant changes:

• Research as an open and inclusive endeavour, in dialogue with society: the definition of what counts as research is much broader, encouraging much more diverse approaches to building and sharing valuable knowledge
• Acknowledging the vital contribution of engagement to the creation of impact: for the first time, there is recognition that the processes of engagement need to be factored into assessing the social impact of research
• Putting people and culture front and centre: for the first time, supporting a healthy research culture is to be an underpinning principle of the REF, influencing all aspects of its design and conduct

These shifts in emphasis open up great opportunities for better recognising the contribution of public engagement, which we have summarised below:

A major change in how the Environment element is assessed

• The focus of the environment element will be expanded to People, Culture and Environment and it will now make up 25% of the profile (a very significant increase from 15% in REF 2021).
• There will be substantial changes to the way this element is assessed: a much ‘tighter’, questionnaire-style template will be used, and a separate template will be developed to enable people to describe their engagement and impact strategy.
• The role of ‘Research-enabling’ staff will be more explicitly recognised.

A broadening of the Outputs element

• A greater diversity of research outputs will be encouraged in REF 2028 – and the Outputs element will be renamed as ‘Contribution to Knowledge and Understanding’ to reflect this.
• HEIs will also now be expected to submit a structured explanatory statement, describing the institution’s wider contribution to knowledge and understanding in the disciplinary area.
• The Contribution to Knowledge and Understanding will make up 50% of the profile (down from 60% in 2021)
• There is an acknowledgement of the need for research outputs to reach diverse audiences, and not to be limited to conventional platforms and formats: ‘Reaching businesses, policymakers and citizens requires outputs in different formats, such as policy summaries or video or audio content’

A greater focus on the contribution of engagement to impact

• The Impact element will be renamed as ‘Engagement and Impact’ to recognise and reward a wider range of impact-enabling activities
• Given the increased focus on engagement the panels will be invited to consider the suggestion made by the NCCPE and others that the criterion of ‘rigour’ be added alongside ‘reach’ and ‘significance’. This will help to ensure that the appropriate focus is placed on the process of delivering impact
• An impact narrative will be reinstated to recognise and reward approaches to maximising the impact of research.
• The assessment of impact case studies will be supplemented by quantitative and qualitative evidence, focused on outcomes and supported by data where appropriate, which sets out the wider contribution of the research activities to society and the economy
• The requirement that the research underpinning impact case studies be of a minimum 2* quality will be removed, to encourage a broader base of underpinning research activity

All in all, these developments are really encouraging to see. They provide opportunities to push for greater recognition of the vital contribution of public engagement, and of professional staff working in this area. It’s also pleasing to see how they have taken on board feedback from lots of people working in the engagement space, and which we highlighted in our FRAP consultation response in May 2022.

Building a more inclusive picture

There are of course things that are missing that we hoped to see. Perhaps the most significant is greater recognition of Equity, Diversity and Inclusion in the assessment of engagement and impact. We argued in our initial FRAP response for more explicit incentives to be built into the assessment, to actively promote equity and challenge inequality, not just avoid discrimination.

We would also like to see more account being taken of the very different place-based and institutional disparities which are currently hidden from view in the REF’s assessment approach. There isn’t a level playing field in terms of the investment and capabilities that different HEIs can draw upon to underpin their production of outputs and to invest in impact-generating activity. We would like to see a ‘Context’ section, building upon the lessons learned from developing clusters in the KEF, to help fairer judgements to be made about what institutions have achieved with the resources at their disposal.

Seizing the moment

There will be further consultation and evidence gathering over the next two years, and the new REF panels will be charged with developing many of these suggestions into the new guidance, which will be published for consultation in winter 2024/25.

The NCCPE will continue to bring people together to feed into this work.

We’re planning a mix of events and desk research (including a deep dive into the REF 2021 case studies and environment statements). Some of the questions we need to address include the following – and we look forward to your help in tackling them:

• How might the criterion of ‘rigour’ be operationalised in assessing engagement and impact?
• What are the indicators of a healthy, ‘engaged’ research culture?
• What are the characteristics of excellent impact and engagement strategies?
• How can the contribution of research-enabling PE Professionals be recognised?
• What quantitative and qualitative evidence can be used to assess impacts arising from public engagement?
• How can EDI be built more explicitly into the assessment of engagement and impact?

We look forward to your help with this important work – and would love to hear your thoughts about the FRAP.

Related resources

• Research Excellence Framework 2028: initial decisions and issues for further consultation (FRAP)
• NCCPE FRAP consultation response 2022