Anti-racist Wales Action Plan: Impact Measurement Framework

Background and purpose

The Anti-racist Wales Action Plan (ArWAP) sets out the vision for Wales to be an anti-racist country. Its aim is to work together to make a measurable difference to the lives of Black, Asian and Minority Ethnic people. The plan was co-produced with Black, Asian and Minority Ethnic people and there was a clear message that people don't just want to see things done, they want to see improvements to their lives because of this plan.

The impact measurement framework will set out the key set of data (numbers and statistics) that will be assessed regularly to help us understand whether the plan is making a difference.

The impact measurement framework is 1 part of the many evidence sources we will use to assess whether change has taken place when evaluating the overall impact of the Anti-racist Wales Action Plan.

This document sets out the areas that the framework will monitor, both collectively across all chapters of ArWAP and also for individual chapters. A technical document will follow in early 2025 which will set out the data sources that are available, their current level of granularity, limitations in their use and development or consultation planned to change or develop new sources.

The Anti-racist Wales Action Plan logic model

The impact measurement framework is based on the logic model for the Anti-racist Wales Action Plan set out in the plan under Race Disparity Evidence Unit: how we will measure change.

The Anti-racist Wales Action Plan logic model is a map that represents the shared relationships among the resources, activities, outputs, outcomes, and impact for the Plan. The model connects how the delivery of actions will lead to the overall intended effect of the Plan, an anti-racist Wales, by 2030. The logic model provides a consistent and comparable approach across chapters to measuring the overall impact of the Plan.

An evaluation plan: a plan for measuring change

The evidence we need to measure change will be a mixture of data (numbers and statistics), as set out in the impact measurement framework, and people’s experiences, including lived experience. We will set out all the evidence we will use to measure change in our evaluation plan which will include the impact measurement framework.

Lived experience will play a key role in measuring change. Lived experience is information about individual experiences, gathered in a way that it can be used to reflect what is happening in people’s lives. It will provide more comprehensive, fair, and impactful findings, focussing on people’s experiences and not just overall numbers.  Whilst we will use existing data where possible for our measures it may take some time to develop new data sources; collecting lived experience will be a way of assessing whether change is taking place for individuals in the first few years of the plan.

There is recognition of the lack of evidence in relation to inequalities in Wales; the COVID-19 pandemic in particular highlighted the gaps in evidence for Black, Asian and Minority Ethnic people. Whilst improvements in evidence have been made, it is not currently possible to break down all existing data sources by ethnicity; this would be the ideal starting point for understanding the impact of the Anti-racist Wales Action Plan on all groups. There are also gaps in areas that need to be measured, for example complaints data is not collected or available consistently across health, social care, education and other policy areas before considering an ethnicity characteristic in relation to the data.

The Equality Race and Disability Evidence Units are carrying out a number of projects to make improvements to data which aim to change the availability of evidence on ethnicity and other equality characteristics. A list of projects and progress is shared in a regular update to stakeholders.

This document sets out the areas we will measure; a technical document will follow which will set out in more detail the data that is currently available, the level of granularity, limitations in use and development or consultation planned. It will set out the extent to which measures have an established mechanism for collection and reporting that can be used for ongoing tracking. As existing data sources change or new data sources come into existence, the detail of the impact measurement framework will evolve; the technical document will be refreshed on a regular basis to consider changes in data sources. This will be followed by a data dashboard which will show the existing data in 1 place so that it can be used to track change as part of the evidence suite for evaluating the impact of the plan.

Common areas

These are common issues across chapters which we need to look at in a consistent way. Some of examples of the common areas to monitor are:

• well-being
• representation and workforce diversity
• access to services
• bullying and harassment
• complaints 

We are assessing the evidence that exists in these areas across the chapters in order to understand what information exists, how it is collected and will set out more detail in the accompanying technical document.

Impact measures by chapter

For most policy areas the Welsh Government has already set out outcome measures which help us understand people’s experiences. If these outcome measures can be broken down by ethnicity, we will be able to understand how people’s experiences and outcomes differ and be able to track change for different ethnicity groups.

Chapter: Leadership

Key areas to measure

• Representation of leaders across the public sector

Chapter: Education and Welsh language

Key areas to measure

• Welsh speakers
• Educational attainment
• Exclusions and absenteeism

Chapter: Local government

Key areas to measure

• Representation of elected members
• People’s perception of their community and services provided

Chapter: Culture, heritage and sport

Key areas to measure

• Participation in culture, heritage and sports activities

Chapter: Employability and Skills, including Social Partnership and Fair Work and Entrepreneurship

Key areas to measure

• Employment and unemployment
• Pay gaps

Chapter: Health

Key areas to measure

• Healthy behaviours
• Mental health
• Health outcomes

Chapter: Nation of sanctuary

Key areas to measure

Key migrant outcome measures relating to: 

• Social connections
• Safety and stability

Chapter: Social care

Key areas to measure

• Children and adults at risk
• Care experienced children

Chapter: Crime and justice

Key areas to measure

• Hate crime

Chapter: Homes and places

Key areas to measure

• Housing situation
• Homelessness

Chapter: Childcare and play

Key areas to measure

• Early years
• Children’s development

Chapter: Rural affairs, climate change and sustainability

Key areas to measure

• Environmental impact in areas where Black, Asian and Minority Ethnic people live