Written Statement - Formative Evaluation of the Youth Engagement and Progression Framework

Today sees the publication of two reports – Formative Evaluation of the Youth Engagement and Progression Framework and Youth Engagement and Progression Framework:  An Approach to an Impact Evaluation.

The formative evaluation seeks to understand the way the Youth Engagement and Progression Framework has been implemented; inform its continued development; and support the development of a detailed specification through which the Framework’s impact can be measured.

To complete this work the contractors, ICF International undertook a number of activities including:

• Interviews with 162 implementation staff across all local authority areas including representatives of post-16 providers; schools; Careers Wales; Jobcentre Plus (JCP); and voluntary sector organisations.
• Detailed case study research in eight local authorities focusing on a specific strand of the Framework’s implementation where strong progress has been made.
• A stakeholder survey of participants in the implementation of the Framework to gauge perceptions of involvement and improvement in arrangements and systems since the launch of the Framework.

The overarching findings of the evaluation are that all local authorities have made progress with implementing the Framework since it was launched in October 2013.  Many interviewees praised the Framework as the best guidance/strategy related to young people produced by the Welsh Government.  Most interviewees believe that the Framework has made a difference to the collaboration between all partners as well as the engagement of the key organisations’ leaders.
Specific findings include:

• All local authorities have developed action plans for implementing the Youth Engagement and Progression Framework and have appointed an Engagement and Progression Coordinator to coordinate delivery of the action plan.
• Generally local authorities have taken steps to develop effective strategic governance arrangements for the Framework.  Although it is difficult to fully assess the effectiveness of steering arrangements, there is evidence that more strategic groups are functioning well with good attendance, examples of active leadership, good partnership working and improved sharing of data and information.
• As of February 2015 nearly every authority in Wales (20 out of 22) had developed an early identification system for young people at risk of becoming disengaged from education.
• Around two thirds of local authorities have developed plans and arrangements for lead working. There have been challenges with this linked to delays in establishing information sharing protocols; difficulties in conveying the message that lead working is already taking place and is not a new approach; and difficulties in getting all partners to participate in operational groups to facilitate the brokerage of support.  The local authorities that have implemented arrangements for lead working have overcome most of these challenges.
• In general, local authorities that have a data-led early identification system in place for young people at school pre-16 have established processes for follow up and review on a termly basis at least.
• All local authorities have made progress with mapping provision against the five tier model of engagement in line with guidance from the Welsh Government.
• Although generally no new employability projects have been developed under the Framework, local authorities and partners provided examples of existing initiatives that provided young people with knowledge about careers and work experience opportunities which they will maintain or seek to extend.
• In general LAs and partners had found the communications, networking opportunities and support provided by the Welsh Government’s programme team to be timely and useful.

Findings from the case studies highlighted:

• The feedback from schools and LA staff on the introduction of the early identification tool is positive. It was felt that the introduction of a standard tool for the whole local authority ‘helped to prevent some young people falling through the cracks as it identified some young people who would not have been seen as at risk previously’ (LA interviewee).
• The introduction of the NEET operational group to discuss the young people identified as at risk has reduced duplication of services and improved coordination between agencies. It has also improved accountability, as agencies report back to the group on the progress they have made with young people.
• As a result of the actions taken since the introduction of the Framework, as well as strengthening of existing arrangements, there has been improved coordination and joint working between partners. Partners are considered to be working well together to share information so that young people are identified early on and then provided with appropriate support that is coordinated by a lead worker.
• As a result of the mapping of provision for 19-24 year olds and subsequent changes to referrals, stakeholders reported improved processes for referrals from JCP to Careers Wales and improved sharing of information between partners.

The report makes 23 recommendations which cover suggested changes to monitoring arrangements for the operational aspects of the Youth Engagement and Progression Framework, changes to the advice, support and guidance provided to partners, and suggestions for what steps need to be undertaken to ensure that the subsequent planned impact evaluation can be undertaken successfully. Officials will be using the findings in both of these reports to inform the design of an evaluation of the Framework.

The Framework has a two year implementation phase which ends in October 2015.  I want to maintain the positive momentum and build on the progress made to date. As part of the development of actions for the Framework’s post-implementation phase, I have asked my officials to consider the evaluation report’s findings and recommendations, alongside those set out in the Wales Audit Office report ‘Young people not in education, employment or training’ and the Enterprise and Business Committee report ‘Assisting young people into work’.  

I will provide an update on this work as it progresses.