Evaluation of the Additional Learning Needs System: scoping report (summary)

The overarching aim of this evaluation is to examine system implementation, identify emerging impacts, and explore the extent to which expectations are being met. The evaluation also aims to identify barriers and facilitators to implementation, and to identify measures that could be put in place to support implementation and the realisation of policy objectives. The evaluation also aims to identify barriers and facilitators to implementation, and to identify measures that could be put in place to support implementation and the realisation of policy objectives.

This paper summarises findings from the scoping phase of the evaluation. It includes a theory of change for the reforms, key findings from a synthesis of current evidence on ALN system implementation, and outcomes of a data mapping exercise. The report also identifies priorities for future phases of the evaluation.

The ALN system is the new system for supporting children and young people aged 0 to 25 in Wales with ALN. The ALN system replaces the previous special educational needs (SEN) system and the system for supporting young people with learning difficulties and/or disabilities (LDD).^{[footnote 1]}

The ALN legislative framework was created by the Additional Learning Needs and Education Tribunal (Wales) Act 2018 (the Act), The Additional Learning Needs Code for Wales 2021 (the ALN Code) and regulations made under the Act. The Act aims to ensure children and young people have access to the support and opportunities they need to fulfil their potential, by ensuring that they receive the provision called for by their ALN in a timely and efficient way, enabling them to participate in and benefit from learning.

The Act creates:

• a single legislative framework to support all children and young people aged 0 to 25 who have ALN
• an integrated, collaborative process of assessment, planning and monitoring of the support provided to ALN learners
• a fair and transparent system for providing information and advice, and for resolving concerns and appeals

Under the new system, all children and young people with ALN, regardless of the severity or complexity of their learning difficulty or disability, will generally be entitled to a statutory support plan called an individual development plan (IDP). Children and young people with ALN will receive support called additional learning provision (ALP) which will be set out in their IDP.

This scoping phase is the first of five phases of the evaluation

Outline of evaluation phases

Phase 1

Theory of change and scoping phase (March to October 2023)

Phase 2

Implementation evaluation: ALN area studies (November 2023 to September 2024)

Phase 3

Implementation evaluation: surveys of practitioners and parents/carers (April to November 2024)

Phase 4

Early impacts and progress evaluation – ALN area studies (September 2024 to September 2025)

Phase 5

Early impacts and progress evaluation – surveys of practitioners (November 2025 to September 2026)

The full scoping report includes a theory of change (ToC) for the ALN system, drawing on desk research, interviews and workshops with stakeholders during the spring and summer of 2023. The ToC is intended to help identify the key research questions and priority areas of investigation for the evaluation. 

The report also presents a synthesis of current information and evidence in relation to the implementation of the ALN system. This aimed to identify issues requiring particular attention during fieldwork with practitioners, stakeholders, learners and parents/carers.

Finally, the report outlines the data sources that are currently available to enable an exploration of the inputs, activities, outcomes and impacts set out in the ToC.

Collectively, these separate sections are intended to inform the priorities for future phases of the evaluation.

The research methods included:

• the development of a theory of change (ToC) for the reforms through an iterative and collaborative process, such as:
  • a review of policy and guidance documents relating to the ALN system
  • a series of scoping interviews and workshops with Welsh Government officials, and scoping interviews with representatives from further education, local authorities, local health boards, the National Academy for Educational Leadership, and the third sector; in total, 21 interviews and two workshops were held
• an evidence synthesis, drawing on recent publications by Estyn, the National Academy for Educational Leadership, academic papers, and unpublished reports and papers on ALN implementation
• a desk-based review of data sources to identify datasets that could be mapped against items included in the theory of change; this stage of the scoping phase included interviews with Welsh Government statisticians and data managers

A key focus of the report is the presentation of a theory of change (ToC), which explains how the ALN system is intended to achieve its aims and objectives, by making explicit the expected inputs, activities, outputs, outcomes, and impacts, as well as the assumed links between them. The ToC will be used to inform the approach to the evaluation, to support the design of research tools, and to provide a reference point to frame evaluation findings.

As noted in the methodology section, the ToC has been informed principally by available documentation and by input from strategic stakeholders. It is our intention to review and refine the ToC throughout the evaluation. An important next step in this refinement process will be to test the theory and underlying assumptions through engagement with practitioners at different levels across all sectors in addition to learners, and their families. 

Overall the available literature about ALN system implementation is limited and mainly concerns the experiences of mainstream primary and secondary schools. Insights from other education settings (including early years), the health sector, local authorities, learners, parents and carers is lacking. A key priority for future phases of the evaluation will be to explore the perspectives and experience of these groups in relation to the ALN system.

The sample sizes involved in many of the key sources of evidence reviewed are relatively small, limiting the ability to generalise findings and draw definitive conclusions about ALN system implementation to date. Instead, the findings of the evidence synthesis chapter should be viewed as being illustrative of issues requiring further exploration during the evaluation.

The synthesis is presented in two parts and considers evidence under themed subheadings relating to: (i) the work undertaken to support and prepare for the ALN reforms, and (ii) the ongoing implementation of the ALN system. A summary of key findings is presented below.

Part one: activities to support ALN reform

Workforce development and professional learning

Current evidence is predominately focused on opportunities for education practitioners, highlighting the need to explore training provision for health practitioners, local authorities and other stakeholders involved in the implementation of the ALN system. The literature stresses the positive impact of professional learning opportunities, especially for Additional Learning Needs Coordinators (ALNCos), whilst also emphasising the barriers faced by practitioners in accessing these opportunities. The evidence also suggests a need to develop further professional development opportunities, particularly aimed at those entering the education profession and continuing professional development focused on pedagogical practice to support the teaching and learning of children with ALN.  

The review of current evidence underscores the need for the evaluation to explore workforce development and professional learning in more detail, especially in relation to opportunities for those in the health sector, early years settings and in local authorities, and the effectiveness and scope thereof.  

Information, advice, and guidance

Welsh Government and other organisations have published information and guidance to support awareness-raising and understanding of the implementation of the ALN reforms. Evidence indicates that this guidance is being used by education practitioners, however there is limited information available on the perspectives of other stakeholders across the system. Current evidence suggests there is variability in the use and dissemination of guidance materials, resulting in limited awareness among some professionals and parents about the ALN reforms. Concerns regarding the consistency of guidance and the understanding of inclusive educational practices are also reported. This has the potential to result in variations in approaches to implementing aspects of the reforms and is an area that the evaluation will need to explore in more depth.

Funding

The evidence highlights funding as a critical input in the ALN system. However, the available evidence points to concerns around the sufficiency and transparency of funding. While the Welsh Government has increased funding for SEN and ALN, the evidence also reports on significant additional costs incurred by local authorities and schools in implementing the reforms. In spite of the limitations of the evidence base, and the fact that the available evidence draws largely on perceptions of the sufficiency of funding from school leaders, it is clear that the evaluation should explore views from across the system on the efficacy of funding allocations for ALN. This should include an exploration of how perceived or actual funding constraints are impacting on the successful implementation of the ALN system. Finally, the emerging evidence also points to the need for a comprehensive assessment of the transparency of funding allocations to support the ALN system.

Part two: ALN system implementation

The role of the ALNCO

Based on the available evidence, it is clear that the ALNCo role is vital to the delivery of the ALN system, both in terms of the ALNCo’s own statutory responsibilities and as a source of guidance and support to other practitioners. However, challenges are noted in the literature, particularly with regard to the workload of ALNCos, and difficulties reported in cases where ALNCos are not members of school senior management teams. The evaluation will explore concerns relating to ALNCO workload and potential impacts thereof on ALN system implementation.

Interpretation of ALN and ALP

Evidence highlights that there may be inconsistencies in how definitions of ALN and ALP are interpreted, potentially resulting in different applications of these terms in practice. In addition, the evidence also points to the emergence of terminology, not explicitly defined in the ALN code (e.g. ‘universal provision’) that is potentially being utilised and applied by practitioners without a common understanding across schools and settings and which parents and carers may find confusing. The evaluation will therefore seek to explore the factors underpinning the reported lack of consistency in the interpretation of the definitions and the implications of this across the system.

SEN/ALN numbers

The most recent data relating to the number of learners identified as having SEN/ALN appears to contradict the original policy assumption that there would not be a significant change in the number of learners identified as having SEN/ALN following the introduction of the ALN system. The data also shows that there appears to be regional variation in the percentage of learners with IDPs. It is important to note that implementation is still underway and thus the number of IDPs is anticipated to increase further as implementation progresses. 

Nevertheless, a key priority for the evaluation should be to explore the potential factors influencing the observed changes in SEN/ALN numbers and regional variability thereof. There are likely to be a complex array of interconnecting factors underpinning this, and it is not possible to draw conclusions on the basis of the current evidence base. However, considering the evidence pointing to potential inconsistencies in how ALN and ALP are understood, the extent to which this is potentially impacting upon SEN/ALN numbers, including regional variation in the percentage of learners with IDPs, should be a key line of investigation for the evaluation. In light of the parallel implementation of the CfW and ALN reforms it will also be important for the evaluation to consider this wider context and how it influences ALN system implementation. 

IDPs

The evidence notes that, although IDPs enable a personalised approach to address learner needs, the work involved in preparing IDPs in line with statutory timescales places significant demands on resources, particularly on ALNCos. The literature identifies some cases in which there also appears to be a lack of clarity among practitioners over who holds responsibility for maintaining IDPs: the evidence suggests a need to explore the different approaches taken by local authorities to determine who maintains an IDP as part of this evaluation, including a focus on the communication of these different approaches to schools, parents, carers and learners.

Engaging with parents and carers

The evidence suggests that many education practitioners believe the increased use of person-centred planning approaches has contributed to stronger relationships between educational institutions and the parents/carers of children with ALN. However, the available evidence from schools and emerging evidence from parents also points to persistent challenges, including variability in the quality and accessibility of information for parents, underscoring the need for better communication and transparency. Importantly, the fieldwork planned as part of the next phase of this evaluation (particularly the ALN area studies) will provide opportunities to explore parents’, carers’ and learners’ experiences of the ALN system.

Collaboration and cluster-working

Despite collaboration and multi-agency working being vital components of ALN implementation, the evidence highlights that collaboration and information sharing between agencies overall is too variable. Many schools are reported to have developed positive collaborative approaches with local authorities and the introduction of key roles, such as the Early Years ALNLO and Designated Education Clinical Lead Officer (DECLO) was welcomed. However, the evidence suggests that there are ongoing issues of capacity that may be hindering the ability of health partners to engage fully with education settings, and that information sharing during transitions from one educational setting to another is an important challenge. 

Based on the current evidence available, there is a need for the evaluation to examine how partners across the ALN system collaborate and share information to ensure effective planning and provision that meets learners’ needs. The evaluation should prioritise understanding the features of effective collaboration and identifying what factors facilitate and inhibit joint working, including between education and health professionals.

Post-16 learners

The evidence sets out concerns that have been raised about the interpretation of the two-year entitlement to further education and training, particularly for learners with complex needs. The literature emphasises that ensuring consistency in transition protocols and sufficient capacity in further education institutions is crucial for equitable access. The emerging evidence, therefore, highlights the need for the evaluation to carry out research on the interpretation of post-16 entitlements and on the consistency of ALN implementation in practice for this age group. 

Welsh-medium provision

The evidence shows there are clear challenges to realising the core aim of creating a bilingual ALN system. These challenges include weaknesses in the strategic planning to support and develop Welsh-medium ALP, limited Welsh language resources, and a shortage of qualified staff able to support ALN learners through the medium of Welsh. The evaluation will need to explore how these issues are being addressed and whether there is effective implementation of the ALN system in Welsh-medium settings.

Main findings

The findings from the data mapping show that there is currently either a limited or partial evidence base for answering the majority of the research questions identified. None of the current data sources identified, enable the research questions set out in the ToC to be answered fully.

Gaps in the data

There are gaps in data that illustrates the experiences of learners, parents and carers of the ALN system as well as data on the views of professionals in local authorities, the health sector, early years and post-16 settings. This shows the evaluation will need to gather evidence from these audiences during its next phases. 

Data mapping

Additionally, the data mapping highlights gaps in evidence which go beyond the scope of the evaluation. Addressing these gaps will require ongoing activity by the Welsh Government and stakeholder organisations, to monitor and analyse data gathered through secondary sources. 

Main findings

The findings of the scoping study indicate that the area studies (Phases 2 and 4 of the evaluation) and the surveys (Phases 3 and 5) planned as part of the evaluation should include a focus on the research questions identified through the ToC.

The findings of the evidence review additionally suggest that Phases 2 to 5 of the evaluation should explore several specific themes including:

• the transparency and effectiveness of ALN funding allocations and also the sufficiency of resources, both financial and workforce capacity, to implement the ALN system
• the relationship between the ALN reforms and workforce capacity, staff well-being, and professional development, especially for ALNCos
• the extent to which there is a clear and consistent understanding and interpretation of ALN and ALP among practitioners, settings and across local authorities
• factors contributing to the apparent drop in SEN/ALN numbers, in addition to variations across Wales in the numbers of IDPs
• examining the data on the numbers of IDPs and factors contributing to variations across Wales
• consideration of the implications of the parallel implementation of the CfW and ALN reforms for ALN system implementation
• how parents/carers and learners are being engaged about the ALN system and supported to understand their rights
• local authorities' approaches in maintaining IDPs and how these approaches are communicated to schools, parents, caregivers, and learners
• specific post-16 learner issues, including the transition of responsibilities from the Welsh Government to local authorities, ensuring equitable access to education and training for this cohort of learners
• challenges to realising the core aim of creating a bilingual ALN system and explore whether there is effective implementation of the ALN system in Welsh-medium settings

[1] The ALN Act replaces the existing special educational needs (SEN) legislation (provided for in Part 4 of the Education Act 1996), the Special Educational Needs Code of Practice for Wales and learning difficulties and/or disabilities (LDD) legislation (provided for in the Learning and Skills Act 2000).

Report authors: Duggan, B

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Schools Research Branch
Social Research and Information Division
Email: schoolsresearch@gov.wales

Social research number: 116/2023
Digital ISBN 978-1-83577-274-4

Image