Evaluation

These split into two main areas – practical challenges and challenges with evidencing impacts:

Practical challenges are often down to lack of resources especially time, and lack of knowledge about how to evaluate well. This can sometimes lead to evaluation being an afterthought, and not being useful. Strategic planning from the beginning can minimise resources needed and ensure any evaluation that is done is effective. We recommend costing evaluation into your project plans to ensure that you have resource needed to do this well – many funders will expect to see this as an essential part of effective work.  A common pitfall is not including enough funding to do this well.  

Evidencing impacts provides several specific challenges. Given the complexity of the contexts within which engagement and culture change work take place, it can be difficult to prove attribution (crudely put, that Intervention A led to Outcome B). It can be useful to think of the contribution your work makes, rather than trying to prove that your work directly led to specific impacts.  

Long term tracking of individuals involved in engagement work can be difficult, and the resources needed may outweigh the benefits of collecting the data. Determining suitable proxy indicators of impact can provide a helpful way of managing this. A proxy indicator is one you can measure more easily at the point of intervention, which provides confidence that impacts will happen (for example, drawing on existing data sets that link learning at an event, to if and how that learning may effect change).  

Evidencing impacts in longer term projects (e.g. a culture change initiative) can be slightly easier, as the participants are available over a longer period of time. The EDGE tool is a classic culture change framework that you can use to assess how your work is going. Creating a baseline by surveying staff, and then repeating this survey biennually, is a great way to both inform the initial approach to culture change work, to assess if and how it is going, and to identify areas where more targeted interventions are needed

Whether you want to develop the evaluation yourself, or work with an expert evaluator, then help is available to you. Don’t forget to engage with your public engagement professional if you have one – as they are likely to have evaluation expertise and may be able to inform your approach. They might even have existing resources and tools you can draw on, that have been developed within your institutional context.  

Deciding whether to commission an evaluation expert will depend on the scale of your project, and the type of evaluation approach you want to take.  

An evaluator may take several roles in your project. They might lead the design and execution of the evaluation, with you and your teams support, sometimes called collaborative evaluation; they might work alongside you and your team in a participatory evaluation – where you co-develop the approach; or you might lead the evaluation, with the evaluator providing advice and support as a member of your team. The approach you take will depend on the needs and interests you have – for example a participatory approach can help you develop your own skills and confidence in evaluation but is more time intensive.   

Whether you go it alone, or bring in others to help, you will need to be clear about the purpose of your evaluation, the questions you hope evaluation will help answer, and the resource you can contribute to it.