REF 2021 guidance published: what’s the public engagement angle?

The consultation on REF 2021 is over and the final guidance has been published – NCCPE Director Paul Manners explains what has changed and what this means for public engagement.

It was with bated breath that we opened the PDFs of the final guidance for the REF 2021 to see what had changed. The NCCPE had put a lot of work into our response to the consultation, with the help of many people across the sector, and were keen to see what difference this had made.

Therefore it was initially disappointing to see that very little had changed from the draft published back in July 2018.  Sadly our work had little impact on this occasion! 

The NCCPE’s consultation response had identified some critical areas pertinent to public engagement, areas where we thought the guidance could be improved.

Equality and diversity

We argued that the guidance should highlight the importance of considering equality and diversity in relation to impact, not just internal HEI staffing policies. We argued for a clear expectation being set by the panels that equality and diversity be addressed in impact planning (in the ‘environment’), and, where relevant, within impact case studies. We recommended that the 2010 Equality Act and the Public Duty regarding socio-economic inequalities were referenced to provide an appropriate framing of this expectation.

Whilst this hasn’t happened in the way we had hoped, there is a small but significant change in the Environment template, which now includes an additional prompt in Section 4 (Collaboration and Contribution to the Research Base, Economy and Society) inviting information about:

'How the unit engages with diverse communities and publics through its research' (Panel Criteria, paragraph 358)

We were also pleased to see that the Panel Criteria and Working Methods have slightly refined their guidance about impacts arising from public engagement.  Their definition of ‘reach’ now encompasses demography – not just numbers of participants. The additional phrase is highlighted below:

'Panels expect that case studies based on public engagement will demonstrate both reach (for example, through audience or participant figures and demographics) and significance…' (Panel Criteria, paragraph 299)

Both are a step in the right direction, and we shouldn't underestimate how important even small changes in wording can be.

Areas and indicators of impact

We also suggested various ways in which guidance about public engagement could be made more explicit and coherent. We recommended some changes to the framing of the areas and indicators of impact (Annex A of the Draft Criteria and Working Methods) to ensure that impacts arising from public engagement are considered more holistically.  The draft table saw outcomes linked to public engagement largely parked in just one area – ‘Impacts on Understanding, Learning and Participation’.  We argued that public engagement can provide a pathway to impact in all of the other impact areas – for instance by enabling public perspectives to influence policy making and professional practice, or through patient involvement in health. However the table remains virtually the same, and we are concerned that this sends a message that public engagement is all about influencing public understanding – and stops people thinking about other kinds of outcomes it can realise, for instance influencing social structures and social practices.

However, always on the lookout for positives, we were pleased to see the Impact Area ‘Impacts on practitioners and professional services’ reframed as ‘impacts on practitioners and delivery of professional services, enhanced performance and ethical practice’.  We think that this more expansive framing could encourage approaches which actively engage publics as service users and stakeholders.

We were also pleased to see the Guidance on submissions emphasises the value of co-produced research in paragraph 319 c:

'The relationship between research and impact can be indirect or non-linear (for example, co-produced research)'.

Evaluation

Lastly, we did push hard for greater emphasis on the vital role of evaluation in our response.  We recommended that the use of robust evaluation was signalled more explicitly.  That hasn’t happened.  However, there is a welcome softening in Panel A’s position regarding qualitative data and a strong, Panel-wide endorsement of the value of qualitative evidence:

309. The panels anticipate that impact case studies will refer to a wide range of types of evidence, including qualitative, quantitative and tangible or material evidence, as appropriate. Individual case studies may draw on a variety of forms of evidence and indicators. [] A diversity of evidence is welcome, and no type of evidence is inherently preferred over another (Panel Criteria and Working Methods)

There is also clarification of how the panels will approach the inclusion of testimonials:

310. Testimonials should draw on statements of fact and relate specifically to the impact(s) claimed. There may be occasions where opinion-based testimonials are appropriate (for example, where the impact is on public understanding of an issue). 

The right direction of travel

Overall, we are really pleased with the direction of travel from 2014 to 2021.  The guidance lays a solid foundation to encourage people to feature impacts arising from public engagement.  We look forward to working with the sector to seize this opportunity.