Reviewing public engagement with physics and engineering in REF2021

The NCCPE was commissioned by the Ogden Trust and the Science and Technology Facilities Council to review the Impact Case Studies and Environment statements submitted for the 2021 Research Excellence Framework (REF) by physics departments in UK Higher Education Institutions (HEI). The project was later expanded to extend the review to Impact Case Studies submitted for Engineering.

Our aim was to show what the submissions reveal about the ‘state of play’ of public engagement policies and practices in physics and engineering at the point when ICS were submitted to REF2021, understand the focuses of activity and identify gaps for future prioritisation.

We conducted the research in three phases:

1. A landscape summary of the ‘state of play’ in public engagement policy and practice in physics at the time of submissions to REF 2021. The landscape summary laid out the context against which HEI submitted their REF2021 returns.
2. An in-depth critical review of the ICS, building on the rapid analysis conducted in phase 1
3. Focus groups with members of physics departments and informal conversations with a small number of panel reviewers to derive experiences of the REF process.

Almost half (77/169 or 46%) of the physics ICS produced for REF 2021 mention public engagement activity. While a direct comparison is not possible, due to changes in the reporting approach, this represents a significant increase from REF2014 when 18% of the ICS mentioned public engagement.

Of the 44 institutions that submitted physics ICS, 31 included at least one public engagement-focused ICS. While this represents majority ‘take up’ of public engagement in physics departments, it leaves room for growth.
 

In engineering, public engagement is mentioned in very few ICS (22/391 or 5%). Of those 22, only two ICS are primarily about public engagement

There appear to be few differences in approach among the four home nations.

Key priorities for public engagement in physics are raising awareness and understanding of physics, supporting school-level physics teachers to address low participation in science and influencing policy (UK and elsewhere). This shows little change between REF2014 and REF2021.

Key priorities for public engagement in engineering are influencing government policy and contributing to the development of professional standards

Not all physics ICS or ES define their target audiences for public engagement activity (albeit the format of the templates does not require this). Those that do focus on students with lower educational attainment, schools in deprived areas, ‘widening participation’ schools and communities that are under-represented in STEM subjects (such as students living in socio-economically deprived areas and people who identify as female)

Very few engineering ICS define a target audience, however it is clear from context that the main target audiences are policy-makers, professional bodies and the ‘general' public

Public engagement activity in physics is often about broadening access to equipment (e.g. mobile planetariums, telescopes) and sharing knowledge, rather than engaging specific publics in specific academic research projects

Common modes of public engagement are mass public events, media appearances, involvement in science festivals, teacher training events/workshops, public exhibitions and in engineering, policy engagement

Less common modes are digital engagement (although the COVID pandemic increased this), partnership projects (e.g. with third sector organisations and public libraries), arts and cultural projects, citizen science and long-term engagement with schools

A handful of examples of ICS used robust evidence to demonstrate the contribution of public engagement – in physics we identified six examples which we felt were strong examples of evidenced engagement and in engineering, one example. However, this is a small minority; most ICS either don’t mention evaluation or contain very brief summaries which makes it difficult for a reader to judge the robustness and validity of the evidence.

The use of evidence/and the role of evaluation is an area that could be developed within ICS – at present the focus is on evidencing positive outcomes rather than reflection on what might be improved. Evaluation also takes place at the department-level and we wonder about the potential of group/network evaluation approaches.

There are some significant changes proposed for REF 2029; it is likely to look very different to REF2021, with potential redistribution of weighting across the elements, an increased emphasis on people, research culture and environment and greater recognition of the range of roles and people that are involved in research. 

This new profile is likely to set higher expectations about how institutions and units support inclusive, reflective and engaged practice and strengthen their approach to planning, evaluation and evidencing the impact of public engagement. 

We think this will require universities and departments to formulate long-term strategic approaches to planning, evaluating and evidencing the impacts of public engagement, sustained by funding for specialist support, resources, training and guidance for evidencing impact and creating supportive institutional cultures.