Scoping study for the evaluation of the curriculum and assessment reforms in Wales: government response

On 13 July 2022, the Welsh Government published the Government’s Social Research Report (GSR), developed by Arad Research in partnership with The Open University in Wales, Scoping study for the evaluation of the curriculum and assessment reforms in Wales: final report, which presented the findings of a scoping study relating to the evaluation of the curriculum and assessment reforms in Wales.

Throughout the legislative and implementation stages of Curriculum for Wales, the Welsh Government has committed to undertaking a rigorous and transparent evaluation of our curriculum and assessment reforms, both to understand how the reforms are working and to examine the extent to which they are having the desired impact for all learners, regardless of their background or needs.

The paper below presents the Welsh Government’s initial response to the scoping report and its recommendations. This report is an important step in helping us to shape our evaluation plans.

The aim of the scoping study is to articulate the theory, assumptions, evidence and readiness underpinning the curriculum and assessment reforms and to use this information to develop recommendations for a robust monitoring and evaluation programme.

The report identifies the data and other evidence gathering that is in place or planned and assesses the extent to which these would allow the research and evaluation questions to be answered, and where key gaps would remain. A key finding of the report is that the available data would not provide sufficient evidence to answer the research and evaluation questions. Consequently, there is a need to design and deliver a programme of research and evaluation that produces the necessary data and insights to understand the effectiveness and outcomes of the curriculum reform process.

The report recommends a comprehensive suite of studies which, collectively, enable an examination of the progress and impact of the curriculum reforms over time. Nine studies are proposed, some of which emphasise a broad and long view of research into the success of the curriculum reforms, and several that recognise the importance of in-depth investigation of more focused aspects of the work.

The scoping report recommends nine areas of work that the research and evaluation programme should focus on. This includes:

1. Formative and process evaluation of curriculum realisation
2. National sampling study
3. National Cohort Study: “Growing Up with Curriculum for Wales”
4. National Study of Schools as Learning Organisations
5. System Coherence Review
6. Case Studies: Curriculum Elements Foci
7. Case study research: Theory of change activities / outcomes foci
8. Intervention studies: Research and Development
9. Practitioner Enquiry Collaborations

The scoping report also recommends that the programme of research and evaluation should be underpinned by a set of principles to guide research activity. The proposed principles state that the programme of research and evaluation should:

• examine whether the vision and requirements set out in the national CfW framework are realised through the actions of schools and the wider education system
• have learners – and understanding learner progression at its heart
• focus on activities, outcomes and the relationships between them
• take a whole-system view and focus on understanding system change/improvement
• take a long view, including longitudinal research designs alongside other methodologies
• take a broad view – incorporating detailed research into novel aspects of the reform alongside more established (traditional) challenges
• inform system learning continuously – and include both planned and responsive activity
• recognise the diverse linguistic contexts across the system and reflect the commitments by the Welsh Government and partners to support learners’ Welsh language skills development
• build trust in the evaluation process - among those who contribute to it and among audiences for the research
• use efficiencies and draw on existing expertise to support the evaluation process.

The Welsh Government accepts:

• the need for an ambitious and wide-ranging programme of research, monitoring and evaluation proportionate to the scale of our transformational new curriculum
• the authors’ presentation of the programme’s theory of change as set out within the scoping report and the need to regularly review the theory of change to take account of policy developments and other external factors likely to influence how the reforms are working
• that the research and evaluation questions identified are appropriate to understand how the reforms are working, who they are working for and in what contexts, and whether the expected outcomes and impacts of the reforms are being realised in the short, medium and longer term
• that, even allowing for adapting existing evidence sources, these sources would only partially address the research and evaluation questions and that extensive additional research and monitoring will be required, including in particular to understand how learners are progressing and if and how educational standards are improving in Wales
• that a programme of research and evaluation must take both a broad and long view of the success of curriculum reform, while also investigating some aspects in more detail, and should start as soon as possible
• that research and evaluation activity should be underpinned by the 10 principles set out in the evaluation scoping report

The Welsh Government will consider the specific areas of work recommended and will publish an update on developments in autumn term 2022 and aim for detailed evaluation plan to be published in summer term 2023. This will set out a detailed programme of research, evaluation, and monitoring activity which will enable us to understand and describe the improvement or change in educational standards across Wales, including in relation to specific cohorts of learners. 

This is to ensure that our evaluation is aligned with the recommendations from a complementary project being undertaken to identify the data and information needs across the school system in Wales, which will report in the autumn, and to develop long-term, integrated plans. However, in order to ensure that we are able to understand the experience and learning from the first year of curriculum implementation, we will undertake initial, smaller-scale research from autumn 2022.

Given the central role of Curriculum for Wales within our broader education reforms, we recognise the important links between the recommended evidence projects and work being undertaken in other parts of the system, and more widely across the Programme for Government. Our evaluation plan will ensure that research and evaluation activity is aligned and integrated where appropriate to provide a holistic picture, reduce burden on research participants and make best use of resources.

Additionally, the evaluation plan will consider how the work will be resourced and undertaken. In recognition of the of scale of the work and the national and international interest in the reforms, as well as commissioning work through the Government’s Social Research service it will be appropriate to engage with the wider research community, including drawing on the expertise of researchers across the education system, academics and other research bodies.