Evaluation of Apprenticeships Programme: final report (summary)

In 2015, Welsh Government secured funding from the European Social Fund (ESF) to support part of the delivery of its Apprenticeship Programme which includes four ESF operations across Wales for January 2015 to March 2019, since extended to 2023. Funding was awarded to four projects under two ESF Specific Objectives (SO) in Priority Axis 2 (Skills for Growth).

1. Specific Objective 1 (SO1): to increase the skills levels, including work relevant skills, of those in the workforce with no or low skills.
2. Specific Objective 2 (SO2): to increase the number of people in the workforce with technical and job specific skills at an intermediate and higher level.

These objectives are sub-divided into two European Union-designated NUTS2[1] regions: West Wales and the Valleys, and East Wales. Performance against these objectives is assessed relative to targets set for the Apprenticeship Programme in the ESF Business Plans (January 2015, regularly revised). For targets relating to total number of apprentices, absolute targets are set, in terms of number of enrolments. Targets for participation in apprenticeships among sub-groups, or for outcomes of apprenticeships, are expressed in terms of percentages of apprentices. It should be noted that this report’s findings therefore focus on apprentices funded by the ESF programme, and the data and findings exclude apprentices that are not funded by ESF initiatives.

The report sets out the findings from an evaluation of the Apprenticeship Programme, covering Apprenticeships taking place between January 2015 and March 2019. The aim of the evaluation was to assess the effectiveness, efficiency and impact of the Programme over that period.

The evaluation

The focus of the evaluation was on the performance and impact of the Programme, with some review of design and delivery processes. This report summarises how the Programme performed in practice, how well learners, providers, employers and stakeholders think it has been delivered, and the impact it has had on them.

The evaluation is based on a range of information sources, including primary research with learners, training providers and stakeholders, as well as secondary data, summarised in Chapter 2 of the report. Stakeholders taking part in this primary research included public bodies, training federations, third sector organisations, and representative bodies for training providers and employers, as well as Welsh Government policy officials.

[1] NUTS2 (Nomenclature of Territorial Units for Statistics Level 2) regions are geographical areas identified by the European Union (EU), and used for planning and monitoring many EU initiatives.

Programme design: administration and commissioning

Policy on apprenticeships in Wales was last updated in February 2017, in the Apprenticeships Skills Policy Plan[2] (‘Aligning the apprenticeship model to the needs of the Welsh economy)’. A summary of Welsh Government policies at the time of writing is provided in Chapter 3.

Overall, the process for commissioning the 2015-2019 Apprenticeships Programme was commonly seen by stakeholders and providers taking part in qualitative interviews as robust, fair and transparent, albeit lengthy and resource-intensive. Uncertainty around the bidding process was, however, acute for providers where apprenticeships made up a large proportion of their overall business.

Providers also had concerns regarding the introduction of some new initiatives. The implementation of Essential Skills Wales (ESW) was particularly criticised. The in-depth interviews with providers suggested that many found fitting the changes to working practices into budgets designed before the introduction of ESW difficult.

Programme design: structure and duration

The employer survey showed positive views on apprenticeship structure and duration; three-quarters (75%) of employers felt that the structure and length fitted their needs. Many employers taking part in the in-depth interviews were in favour of the approach of apprenticeships leading to accredited qualifications, in order to guarantee that apprentices had been adequately trained and to enable them to progress onto more advanced roles within the business. Learners interviewed generally said that the structure and length fitted their needs.

Programme design: subjects, levels and curriculum

Generally, employers surveyed were satisfied with the range of subject areas and levels available (73%), and with the content of frameworks (76%). Most felt the move towards more STEM apprenticeships was positive for the Welsh economy, but businesses not offering STEM subjects were concerned that apprenticeships they offer might be deprioritised.

Programme design: Welsh-medium and bilingual apprenticeships

Stakeholders and, to a lesser extent, providers were highly supportive of the policy of introducing and encouraging Welsh-medium and bilingual apprenticeships. Employers interviewed were fairly positive, although those most interested were customers or clients who spoke the language, and awareness of the existence of qualifications delivered through the medium of Welsh seemed limited.

Programme design: apprenticeship levy[3]

Overall, stakeholders felt that the UK Government Apprenticeship Levy Tax had increased employer demand for apprenticeships. Some providers were concerned about tension between the demands of levy-paying employers, and their contractual obligation to deliver a focus on priority routes and move away from Level 2 provision.

Programme design: areas for improvement

Some stakeholders felt a key improvement would be to facilitate better apprenticeship progression routes by connecting apprenticeships across levels. This would encourage apprentices, employers and providers to see the Programme more holistically and view it in a similar way to academic qualifications. Some other stakeholders and providers interviewed felt that by working with careers advisors, school pupils and more widely with young people and parents, a parity of esteem between apprenticeships and university could be achieved.

Providers’ priorities, when surveyed, were more around the administration of the Programme; by far the most popular response when asked what improvements they would like to see was a reassessment of the design of ESW requirements (30%).

Programme delivery and implementation: partnerships and promotion

Overall, relationships between organisations involved in the delivery of the apprenticeship Programme are viewed positively. Stakeholders were satisfied with the structures which have evolved (or been created) to facilitate communications between organisations delivering apprenticeships.

Some stakeholders felt the Welsh Government could be doing more to strengthen its relationship with employers. Employer-provider relationships were strong; however, getting employers engaged with the Programme in the first place remained a challenge.

In the qualitative interviews, employers commonly said the only information source they consulted about apprenticeships was local training providers. While employers seemed well-informed about what they needed to do to help their current apprentices, there was evidence of knowledge gaps among some employers in terms of the range of apprenticeships available, verified by stakeholder interviews.

Providers commonly felt the Welsh Government should do more at a national level to promote apprenticeships to all audiences. However, some stakeholders were quite critical of providers in this area and felt they should take more responsibility for the promotion of apprenticeships.

Programme delivery and implementation: provider performance

Nearly three-quarters (72%) of employers were positive about their main apprenticeship provider, rating them 4 or 5 out of five at a headline level. Overall, the vast majority of apprentices surveyed were satisfied with their training provider (88%) and employer (85%).

Stakeholders within and outside Welsh Government who were interviewed for the research had mixed views on training providers. Although most of these respondents felt, overall, that training providers met the needs of the Programme, some believed that areas of provision – the quality of teaching and employer engagement – were inconsistent.

Programme delivery and implementation: barriers to delivery

Most employers (86%) felt it was likely they would continue to provide apprenticeships in the future, with nearly two-thirds of these employers (64%) stating it is likely they would increase the number of apprenticeships they provide.

Just over a quarter of training providers said they had encountered barriers in achieving gender equality (27%), recruiting more apprentices with disabilities or learning difficulties (12%) or from ethnic minorities (14%). More than half of employers (56%) reported that they had an imbalance by gender among apprentices; usually they felt this was due to the nature of the sector/industry (49%). Far fewer mentioned difficulties employing apprentices with disabilities or learning difficulties (12%). Nearly half of these (48%) stated this was because apprentice roles at their organisation were not suitable for individuals with disabilities.

Many training providers (66%) had encountered barriers to delivering apprenticeships through the medium of Welsh or bilingually, usually attributed to a shortage of Welsh speaking staff (60%). Some stakeholders argued that learning providers were not creating enough opportunities for bilingual learning, based on a misconception of low demand. Findings from employer interviews imply that awareness may be limited among this group, with many responding to questions as if Welsh-medium or bilingual apprenticeships were not already available.

Programme performance

In general, stakeholders and employers were satisfied with the way in which the programme was performing. Targets not met at a total level related to gender balance and the proportion of learners with a work-limiting health condition or disability, each by a small margin. The separate SO1 and SO2 targets for younger learners were missed by large margins; however, this is largely due to the introduction of all-age Apprenticeships, which was not anticipated at the time targets were set. Nevertheless, it is notable that the absolute number of learners aged 19 to 24 has been in decline since 2015, and the number aged under 19 has increased by only 9%.

It is likely that this reduction in younger learners has some relationship to the targets on volumes of apprenticeships in Strategic Operation 1 in both East Wales and West Wales and the Valleys being missed by significant margins. This Strategic Operation consists primarily of learning at Level 2 (82% of all apprenticeships in SO1), programmes which tend to be more likely to be taken by younger learners. The reduction in Level 2 apprenticeships seen among learners under 25 is not being compensated for by any increase in use of Level 3 apprenticeships among the same group.

In general, however, the programme was perceived as good value for money for both stakeholders and employers, although complex administrative structures detracted from this for some employers and providers.

Some providers felt there should be more geographical and sectoral nuance in the targets set. It could be argued that some targets (Black, Asian, Minority Ethnic participation, and Welsh-medium apprenticeships in particular) might need adjustment to take into account the demographic composition of the area which the target applies to.

In terms of potential for improvement, the wide variation in terms of performance by Sector Subject Area suggests that initiatives dedicated to individual sectors may be productive. Further research may be justified into the detailed reasons why the reduction in people aged under 25 taking Level 2 Apprenticeships is not being significantly counterbalanced by an increase in uptake of Level 3 Apprenticeships among the same age group.

Programme impacts: apprentices

Most employers who took part in the employer survey felt the Programme had had a broadly positive impact on the apprentice(s) they had employed (86%). Most apprentices surveyed in the ESF Participants Survey for Wales were also positive about the improvement to their situation, for example in terms of improved confidence in their abilities (85%), as well as improved career progression (76%).

This was backed up by the benefit cost analysis using income and employment data from LEO, which indicated a strong benefit to cost ratio for the Programme (1.48 to 1.59, depending on the assumptions used) relative to non-apprenticeship provision (the ‘counterfactual’), even on a very short (two year) time horizon[4]. The impacts of the Programme according to this analysis included a 29 % point increase in the job entry rate relative to other provision (the ‘counterfactual’), and, in the first year after completion, an increase of 119 days in employment and £7,866 in earnings.

There were some areas for improvement; some apprentices interviewed were frustrated by having to complete ESW elements of their course, in that they felt they had learned little they did not already know. This may relate to the difficulties training providers reported with the process for accrediting prior qualifications.

Programme impacts: employers

For employers, the ability to shape apprentices to meet changing business needs and address skills gaps were the most positive impacts of the Programme. The majority of employers in the quantitative survey reported positive impacts for themselves (76%) and the wider sector (65%). If the Apprenticeship Programme did not exist, most employers felt they would be able to find alternative ways to recruit and train current and prospective employees, although none of those interviewed in the qualitative discussions believed this would be a better option.

Programme impacts: skills

In terms of overall desired impact, the two ESF operations fundamentally aim to improve skills in the workforce at two levels.

1. Specific Objective One: to increase the skills levels, including work relevant skills, of those in the workforce with no or low skills, and
2. Specific Objective Two: to increase the number of people in the workforce with technical and job specific skills at an intermediate and higher level.

Although generally positive, some stakeholders felt the reduced funding of Level 2 apprenticeships had compromised the first aim, while progress on the second had improved since 2015 with all-age apprenticeships and the focus on higher levels. Results from the ESF Participants Survey for Wales of apprentices also suggest that the Programme is supporting these aims in practice. Most apprentices (82%) stated they had acquired job-specific skills as a result of their course.

A number of apprentices participating in in-depth interviews suggested they had already learnt skills their apprenticeship provided through their prior experience. Some, however, were pleased that the Programme confirmed their knowledge, and drew confidence from this. They also felt that it was useful in itself to gain a qualification recognising their pre-existing skills.

Stakeholders, employers and training providers were asked the extent to which the Programme had made learning accessible to a broader cross-section of society. Although most were positive, few felt they could give a definitive opinion.

The Programme was also felt to have a positive impact on the wider Welsh economy, in terms of raising the skills levels of people who have low or no skills and increasing the number of people in the workforce who have job-specific skills, although stakeholders found this difficult to evidence in concrete terms.

[2] Welsh Government. (2017). Apprenticeships Policy Plan: Aligning the Apprenticeship model to the needs of the Welsh economy. February 2017.

[3] The Apprenticeship Levy is a UK-wide levy on employers that came into effect in April 2017. Employers with an annual pay bill of £3m or more pay the Levy, which corresponds to 0.5% of an employer’s pay bill.

[4] Data limitations mean that we can currently only assess outcomes for just two years after an apprenticeship has been completed; with time, the assessment of longer-term impacts will become possible.

Rowan Foster, Mark Winterbotham, Aoife ni Luanaigh, Sam Morris, Christabel Downing and Jonnie Felton, IFF Research

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government

For further information please contact:

Hannah Davies
Social Research and Information Division
Knowledge and Analytical Services
Welsh Government
Cathays Park
Cardiff
CF10 3NQ

Email: hannah.davies018@gov.wales

Image

Digital ISBN 978-1-80082-714-1