LGBTQ+ Action Plan for Wales: evaluability assessment

Main findings

A process evaluation and theory-based impact evaluation, both with a qualitative focus, are recommended for evaluating the LGBTQ+ Action Plan.

Important messages

A process evaluation should be carried out promptly so any recommendations can be implemented to increase the LGBTQ+ Action Plan’s impact. 

The impact evaluation should comprise an initial data collection stage in the first instance, followed by further data collection and evaluation in several years’ time. This will allow measurement of any gradual shift in people’s perceptions and experiences. 

A Theory of Change was created. This highlights key assumptions about how activities will lead to outputs, outcomes and impacts, as well as potential risks. The Theory of Change will need to be updated as new outputs or outcomes become apparent.

There is currently little quantitative data that can be used to measure the LGBTQ+ Action Plan’s outcomes and impact. Improvement in data volume and quality is needed for analysis of those with intersecting identities, for example looking into experiences of Black LGBTQ+ people and disabled LGBTQ+ people.

Experimental or quasi-experimental techniques were not advised for an impact evaluation due to ethical concerns and limited data availability.

Some theory-based approaches for an impact evaluation were ruled out. This was due to high demands on stakeholders. It was also because these approaches are unable to capture the complexities of the LGBTQ+ Action Plan for Wales.

An economic evaluation was not recommended. This was because of a lack of data available to monetise outcomes and the complexity of quantifying LGBTQ+ experiences.