Evaluation of the Degree Apprenticeship programme: final report (summary)

In February 2020 the Welsh Government commissioned Wavehill and the Learning and Work Institute to undertake a formative evaluation of the Degree Apprenticeship Programme in Wales, to assess its effectiveness, efficiency, and impact.

Following its identification as a key aim in the Welsh Government’s Apprenticeship and Skills policy plan, the Welsh Government initially grant-funded the Higher Education Funding Council for Wales (HEFCW) approximately £20 million for the first three years of the Degree Apprenticeship Programme with the first apprentices expected to enrol in the programme in September 2018.

For the 2018 to 2021 programme, degree apprenticeship frameworks in Digital, and in Engineering and Advanced Manufacturing were commissioned by the Welsh Government. The following occupational pathways within those frameworks were identified as fitting with Welsh Government priorities:

• control and instrumentation
• data science
• civil engineering
• digital media
• software engineering
• mechanical engineering
• manufacturing engineering
• cybersecurity

There were delays in the completion of the Engineering and Advanced Manufacturing framework. Consequently, proposals for degree apprenticeships in the first year of the programme were associated with the Digital framework only.

All bar one of the HEFCW-funded Higher Education Institutions (HEIs) submitted a proposal to the programme for the £3million of funding in the 2018/19 academic year. In March 2019, HEFCW-funded institutions were invited to submit proposals for £5million of funding for 2019/20 against the three priority areas of Digital and Engineering and Advanced Manufacturing. In 2020 the programme secured £12million of funding for 2020/21. The initial phase of funding for the Degree Apprenticeship Programme came to an end in 2021.

In June 2021, the Welsh Government made a further commitment to support the Degree Apprenticeship programme in Wales for 2021/22 to the value of £9.5m for existing and new degree apprenticeships in the existing priority areas.

Evaluation approach

The evaluation has been delivered over two phases. The scoping phase involved:

• Scoping interviews with 22 key representatives of the Welsh Government, the HEFCW, and each of the HEFCW funded HEIs in Wales.
• Desk research encompassing a review of key documentation associated with the programme design alongside evidence gathered as part of a Senedd inquiry into Degree Apprenticeships and literature on degree apprenticeships and apprenticeships at higher levels globally and within other home nations of the UK.
• A workshop, with representatives from the Welsh Government, the HEFCW, and Universities Wales to explore and assist in the development of the theory of change for the programme.

The final phase of the evaluation, which commenced in the spring of 2021 involved:

• Interviews with 40 stakeholders (in late summer/early autumn 2021) involved in the management and delivery of degree apprenticeships across Wales. This included representatives of HEFCW funded HEIs and from Further Education (FE) Colleges that partner HEIs in the delivery of Degree Apprenticeships
• An online survey of 199 Apprentices (representing a response rate of 34%) who had enrolled on the programme, to capture feedback on their motivations to enrol on a degree apprenticeship and their experience and perceived value of the degree apprenticeship following enrolment.
• Survey participants who consented to a telephone follow-up interview were interviewed in September to October 2021 to explore their experience of and journey through the Degree Apprenticeship Programme in greater depth. Forty-seven (63% of those apprentices who expressed a willingness to do so) were successfully engaged in a telephone interview.
• In August/September 2021, telephone interviews were also undertaken with employers to capture perspectives on their experience of the degree apprenticeship programme. A total of 76 employers responded to the survey, representing 38% of employers involved with the Degree Apprenticeship Programme.
• Interviews with 11 Stakeholders identified by the HEFCW and Welsh Government as likely to have an informed, external/strategic perspective on the Degree Apprenticeships Programme (in February/March 2022)
• Analysis of monitoring returns submitted by each HEI to the HEFCW to explore the profile, patterns and trends of employers engaged and individuals enrolled, by framework and by HEI to the Degree Apprenticeship Programme.

The approach also informed the development of an outline impact framework and cost benefit analysis model for future usage on the degree apprenticeship programme. The approach has been designed in consultation with representatives from the Welsh Government..

Following its launch in 2018 and despite initial delays with the engineering and advanced manufacturing framework, the Degree Apprenticeship Programme has made steady progress with, in 2021, over 600 currently enrolled on a programme delivered by eight of the nine HEFCW funded HEIs and six FE Colleges in Wales.

Certain HEIs dominate degree apprenticeship provision, 43% of degree apprentices enrolled on the engineering and advance manufacturing framework are enrolled with the University of Wales Trinity Saint David, a further 28% with University of South Wales.  Thirty-two% of those enrolled on the digital framework are enrolled with UWTSD, 24% with Swansea University.

Programme design and implementation

The programme launched without a national campaign and with limited associated promotional activity due to concerns that this may lead to levels of demand that the available budget would be unable to fulfil. With the programme launched for a three-year period only, some stakeholders (HEIs, FE Colleges and employers) viewed it as a pilot and with an associated degree of caution, thereby restricting their levels of investment (in personnel, partnerships, infrastructure, and applicants) in the scheme. At the culmination of the initial three-year programme period in 2021, the programme had attracted 55% of the target allocation of degree apprentices (76% of those allocated to the Engineering Framework and 46% of those allocated to the Digital Framework).

Annualised funding awards, often late within the academic year, reinforced the sense that the programme was temporary in nature and had implications for the nature of employers engaged in the programme. It led to a tendency for HEIs and FE Colleges to pursue larger employers who may offer a greater number of apprentice enrolments. It also encouraged these institutions to target employers with whom they had a pre-existing relationship (as securing engagement to a programme with no guarantee of funding proved more challenging with those where there was no prior relationship).

Programme design also influenced the nature of employees engaged as apprentices on the programme. Tight timeframes following funding announcements limited the extent to which a degree apprenticeship opportunity could be linked to an employer recruitment exercise. This has led to existing employees representing 86% of those who enrolled onto the programme with less than one fifth (19%) of degree apprentices under the age of 21.

Amongst SMEs, awareness of degree apprenticeships was reported as being particularly poor. HEI’s promotional opportunities were predominantly driven by one-to-one engagements with employers and liaising with employers through industrial events. These methods were, however, time intensive, and therefore limited in reach.

Broadening the reach of the programme through a more expansive marketing campaign may also bring about benefits in terms of the programme’s ambitions around widening participation. At present, this is an area where the programme is struggling to deliver against its ambitions. Feedback from management and delivery staff suggests this has in part been driven by the overreliance of the programme on employment as a pathway into the programme, as the profile of entrants is therefore determined by the profile of employees within employer settings. As such, broadening the reach of the programme to individuals who are not currently engaged in employment may facilitate access to the programme to a broader profile of individuals.

Recommendations

• Welsh Government has set out plans to continue with Degree Apprenticeships within the Programme for Government. The programme’s ongoing implementation needs to be strategically aligned with a sustained, long-term commitment to programme investment.
• Whilst it may be necessary for funding allocations to remain annualised, these should be committed earlier in the year or outline indications of future budgets given in preceding years.
• Welsh Government should support promotion of the programme through the provision of a wrap-around campaign to raise awareness of Degree Apprenticeship among employers and potential apprentices and within schools and FE Colleges.
• HEIs should continue to support promotion through direct engagement with employers, via one-to-one interactions and engagement through employer liaison forums alongside school and FE College engagement.
• If a strategic programme of investment is announced, progress and performance indicators associated with employer type and apprentice type should be considered to help increase levels and diversity in the type of employers and apprentices engaged.

Programme delivery

The degree apprenticeships programme has been positively received by degree apprentices, their employers, Higher Education providers, and wider stakeholders. The vast majority of degree apprentices (96%; 187/195) would recommend a degree apprenticeship to someone else, and most employers (80%; 61/76) anticipate that they will enrol additional staff on to the programme.

The programme is credited with bringing about a range of positive benefits, including contributing to the skillsets of employees/degree apprentices, and in turn bringing benefits to their employers such as a more skilled workforce and improvements in staff retention. The programme was frequently championed by employers as a ‘win-win’ for employees/degree apprentices, employers and for Wales.

The degree apprenticeship programme has also strengthened partnership between FE and HE albeit with limited examples of new partnerships being forged in response to the programme. In addition, learning pathways to Degree Apprenticeships appear to be fairly well established on the engineering and advanced manufacturing framework with over half having undertaken an apprenticeship previously.

The COVID pandemic impacted on programme delivery. It led to the generation of new delivery methods and processes, generating benefits (increased flexibility, increased efficiency) and challenges (increased isolation and in separating work and home life). 

Areas of improvement

Whilst the programme has received a positive reception from employers, wider stakeholders and degree apprentices, there are several areas where the programme would benefit from further refinement.

Improvements to workload

Prior to enrolment apprentices were most concerned about balancing work and life – their concern was well-founded as a substantial proportion of degree apprentices (67%; 131/199) reported that they had experienced challenges managing workload between their degree apprenticeship and pre-existing in-work demands. This was also one of the main challenges reported by employers.

Whilst many degree apprentices remained positive about the programme even withstanding these challenges, there was evidence that difficulties balancing workload were resulting in stress among some degree apprentices. There were also a small number of reports from both degree apprentices, their employers and teaching staff, of cases where difficulties balancing work and study had resulted in impacts on the wellbeing of degree apprentices.

Evidence also emerged of variability in the time that degree apprentices had made available to them to dedicate to study. Within some employment settings degree apprentices were receiving regular paid study days to support their study, whilst some degree apprentices were expected to work back their hours. At present the programme places an expectation on employers to release staff for 20% of their working week. However, feedback from both degree apprentices and management and delivery staff suggests the extent to which employers adhere to this varies.

Recommendations

• Gaining clarity on expectations of employers as part of the Degree Apprenticeship ‘deal’ would be beneficial to apprentices and may help ensure greater consistency with regards to day release for degree apprenticeship participation.
• Strengthened monitoring of completion and retention rates across delivery profiles and employer settings to identify best practice in supporting apprentices.

Almost a quarter of apprentices who participated in telephone interviews reported that they wanted the delivery period of the course to be extended or for additional time to submit assignments. As the programme moves forward, further exploration to identify whether best practice can be developed in terms of degree apprenticeship delivery models that both meet employer need and support good work-life balance would be beneficial.

It is also possible that workload issues have in some cases been exacerbated by the over-reliance of employers on putting current employees forward for the programme. Most apprentices were employed before enrolling on the degree apprenticeships programme, and many of these appeared to be at relatively senior levels within their organisations (reflecting the level of skills provision being delivered through the apprenticeships). Some of these roles may therefore have been more challenging for employers to back-fill, which would likely be less of an issue where new roles have been created for the degree apprenticeships programme.

Opportunities to improve the alignment between the course and work-based learning

Over one third of degree apprentices (36%; 14/47) reported that they would like to see the course deliver content that was more relevant to their job or industry. Currently there is a high degree of variation in how degree apprenticeships approach the work-based learning element of provision and how they engage with employers.

There was clear evidence that HEIs were, in the main, working closely with employers to understand their requirements and align the course to meet these. At many HEIs, updates with employers and Industrial Liaison Panels were used as an opportunity to understand employer requirements. However, practise varied across HEIs, and employers did not always feel they had been able to influence course content. Indeed, 39% (29/74) of employers reported that they had not able to influence the course’s content.

HEIs, however, reported that this was an area of challenge or tension in delivering degree apprenticeships. HEIs were presented with the difficulty of offering a course that met the differing (and sometimes contradictory) requirements of different employers, and the apprenticeship frameworks as well as the requirements of professional, statutory, and regulatory bodies (PSRBs).

Broadening the reach of Degree Apprenticeships

Evidence shows that those on degree apprenticeships are far more likely to be the first generation of their family to participate in higher education than those on first degrees. Beyond this, there is little indication, at a programme-wide level, of success in widening participation (in part linked to some of the challenges around programme design outlined above). However, some HEIs and FE Colleges have enjoyed far greater success in widening participation than others, suggesting there is scope for sharing performance and good practice amongst HEIs to aid progress against this agenda.

Recommendation

• The sharing of progress and performance by HEIs with regards to widening participation indicators through a community of practice should be considered.

Outcomes and Impacts

Most apprentices engaged through the evaluation remain in the midst of their degree apprenticeship and therefore it is perhaps too soon for tangible outcomes to be realised, that said:

• 45% feel more confident and better at performing their role
• 28% have been promoted and 29% have gained additional responsibilities

Two thirds of employers reported that participation in the programme was increasing rates of productivity amongst degree apprentices whilst, in the absence of the degree apprenticeship, 36% of apprentices would not have participated in an alternative course (rising to 41% amongst those on the digital framework).

Collectively, these initial indicators suggest strong net additional impact emerging for the programme and illustrate the importance of a more robust impact framework being established for the programme.

Recommendation

• That the impact framework is taken forward at the earliest possibility to enable a robust assessment of the net additional impact of the degree apprenticeship programme, relative to other provision is undertaken.

The future development of Degree Apprenticeships

HEIs and employers expressed a desire for an increase in the range of degree apprenticeships available and an expansion in the level of degree apprenticeships to NQF Level 7 (the equivalent of a Master’s).

Stakeholders are concerned that a failure to expand the degree apprenticeship offer in Wales will lead to increasing numbers of employers sending their employees to England to access that provision.

Any expansion of the degree apprenticeship programme requires additional funding. If additional public sector funding is unavailable to facilitate this expansion this would likely need to be drawn from either additional contribution from employers or from the employees participating in a degree apprenticeship.

Report authors: Oliver Allies, Chloe Maughan, Declan Turner

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Heledd Jenkins
Email: kasemployabilityandskillsresearch@gov.wales

Social research number: 7/2023
Digital ISBN 978-1-80364-835-4

Image