Evaluation of the Working Wales service: summary paper 1

Introduced in 2019 and delivered by Careers Wales, Working Wales is a Welsh Government-funded service designed to provide streamlined and efficient employability support that is responsive to an individual’s needs.

An evaluation is being undertaken by the social and economic research company, Wavehill, focusing on the design, delivery, performance, and impact of the Working Wales service.

This is the first of a series of summary papers that will be produced as part of the evaluation and follows the first evaluation report, published in January 2022. It sets out the key findings of consultations undertaken in the latter part of 2021:

• An online survey of Working Wales customers/participants (n=344).
• In-depth telephone interviews with Working Wales customers/participants (n=44).
• Interviews with Working Wales management staff (n=20).
• Interviews with external/partner organisations that refer customers to Working Wales and to which Working Wales refers customers (n=18).

The consultation with Working Wales participants included both an online survey of all those supported by the service and more in-depth interviews with a smaller group of participants.

The number of responses to the online survey was smaller than had been hoped. It was also challenging to engage with participants for the in-depth interviews. It is important to take this into account when considering the findings of the survey.

A likely reason for this is that the support that Working Wales provides can be relatively minor or ‘light-touch’. This is the nature of the Working Wales service which is responsive to the needs of the individual. There may also be a general lack of desire to participate in the research, which was not incentivised. It is also possible that a proportion of the email addresses provided are not used or monitored by the individuals in question having been originally provided when, for example, the individual in question was at college.

These issues need to be considered as the evaluation progresses. Nevertheless, the data that has been collected still provides an important insight into the experience of customers of the Working Wales service and the impact that it can generate.

Relationship with Jobcentre Plus

The relationship between Working Wales and Jobcentre Plus (JCP) is crucial. Most participants reported that they had heard about the service through JCP. They are also identified in the survey as the main source of referrals to Working Wales.

Marketing and promotion

Marketing activities are important in terms of engaging with a broad range of potential beneficiaries and there were positive comments about those activities from stakeholders. Whilst the limitations of the sample must be considered here, the proportion of survey respondents saying that they had heard about Working Wales via a marketing or promotion route was relatively low with the JCP a more prominent point of information about the service.

Comments from stakeholders about the importance of local promotion of the service, which considers differing local characteristics and local priorities, need to be noted. Such an approach would potentially prioritise issues identified in, for example, regional labour market reports and employment action plans developed by Regional Skills Partnerships (currently under development).

Access to the service

The consultations did not identify any concerns about access to the service, although issues caused by the COVID-19 pandemic were raised by the Working Wales team. The value of in-person interactions between advisors and participants – not possible when restrictions as a result of COVID-19 were in place - was emphasised.

The importance of the outreach activities undertaken at a local level, in communities, including the attendance of staff at jobs fairs, also not possible during restrictions because of the pandemic, was also emphasised.

The value of a ‘local approach’ was also, again, highlighted. It was recognised that this inevitably leads to an inconsistent approach across Wales, but the need for a different approach in rural and urban areas was also recognised.

A key finding of the in-depth interviews with participants was that many did not appear to have had a comprehensive understanding of the support Working Wales could offer ahead of their engagement. There is a link here to the points on the marketing and promotion of the service discussed earlier. This is possibly also linked to the broadness of the support that can be provided and the fact that the initial contact is to discuss the needs of the individual and the support that can be provided.

Assessing the needs of the individual

Both internal and external stakeholders identified the ability to effectively assess the needs of participants as being crucial and the strength of the Working Wales service. Indeed, the independence of the advice and guidance provided by the advisors (assured by the level of qualification that all advisors held) was identified as a key ‘USP’ and critical element of the service.

The value of having specific officers/teams focusing on ensuring that the service is up to date with the latest labour market information and effectively engaging with employers was generally described by the Working Wales team as being essential to the service. There was general support for the pilot Job Match Service although the ‘opportunity cost’ needs to be considered. 

Referrals to and from the service

Only a relatively small proportion (25%) of respondents to the online survey and just under half of the in-depth interviewees, said that they had been referred to, or provided with information about, another service or further support by Working Wales. Our sample for exploring this element of Working Wales is, therefore, very small and this is an area where more focused evaluation activities may be needed in the future. The feedback from that small group was however positive. In particular, most (10/14) of the in-depth interviewees in question said that they would not/probably not have accessed the service without the referral/information from Working Wales.

The Working Wales team were generally very positive about the referral process with the development of a standard referral form being identified as an important development. Where there was a concern, the issues raised were a perception that some providers considered Working Wales as their competition.

There was also a concern about the introduction of new support by the Department for Work and Pensions, in response to the COVID-19 pandemic, which was perceived to have led to an increase in ‘internal referrals’ within DWP at the expense of referrals to Working Wales.

Some partner organisations’ concerns about the number of referrals being made by Working Wales were also acknowledged. However, the fact that referrals were dictated by the needs of the participant in question was stressed, as was the importance of the independent nature of the advice being provided and/or referral being made.

Discussions with external stakeholders about the referral process were unanimously positive. This included the extent to which participants being referred were ‘matched’ to the support that they could provide. Whilst the impact of the COVID-19 pandemic was recognised, the concerns about referral numbers recognised by the Working Wales team were also expressed by some of the external interviewees.

Stakeholders from within and outside Working Wales were generally positive about the impact that the support was having on participants, making comments in particular about the impact of support on individuals’ confidence and other ‘soft outcomes’ which were echoed by participants engaged by the research.

Bearing in mind the limitations of the sample, the information provided by participants via both the online survey and in-depth interviews was generally positive in respect of the outcomes of the support. Most respondents to the online survey agreed that Working Wales had helped them become more:

• confident in their approach to job hunting and/or work
• motivated to consider employment opportunities and training
• knowledgeable and aware of the training and employment available to them.

Comments made by in-depth interviewees were consistent with this, with the impact on the confidence and well-being of participants being particularly apparent from those discussions. Indeed, most in-depth interviewees were very positive about the benefits they had derived from their involvement with Working Wales.

This suggests that Working Wales can have a positive impact in terms of what may be described as ‘soft outcomes’. This also chimes with the outcomes described by stakeholders when discussing their perceptions of the impact of the service.

The comparison of the situation of the online interviewees before and after their involvement with Working Wales is interesting. The proportion of respondents reporting that they were employed increased by over 30%, whilst those reporting that they were unemployed decreased by over 35%.  These figures are positive although the small sample again needs to be emphasised. The prevailing economic conditions (falling levels of unemployment) also need to be considered.

The extent to which these changes can be attributed to Working Wales is unclear. Indeed, the survey also found that most (61%) of those reporting a change in their employment situation did not attribute the change to the support they received from Working Wales. The prevailing economic conditions as noted earlier may influence this, as may the ‘light-touch’ nature of the support that the survey respondents generally reported. The finding is, however, consistent with the views of some stakeholders who questioned whether employment outcomes could be completely attributed to support provided by Working Wales. The key to this is perhaps that Working Wales should be seen as a stage in a process rather than the support mechanism itself - it helps participants to move closer to work even if it is not always the final step in that process.

As the evaluation progresses, alternative means of consulting with Working Wales participants should be considered. This includes potentially being more focused going forward - targeting specific groups of participants/customers to discuss specific issues to improve the engagement with them and the robustness of the data collected.

The potential pros and cons of undertaking more local or regional marketing and promotion of the Working Wales service should be considered. This should include the consideration of the ‘opportunity cost’ of national marketing and promotional activities, compared to a more localised and targeted approach potentially led by the local Working Wales team.

The evaluation will explore these issues further as it progresses. The emerging finding, small sample noted, is, however, that Working Wales has an important positive impact on the participants, but that the impact is not, in the main, considered by the participants to be directly responsible for their employment outcome.

Report authors: Endaf Griffiths, Dr Nicola Vousden, Sam Grunhut (all Wavehill) and Calvin Lees (Learning & Work Institute)

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Sean Homer
Email: kasemployabilityandskillsresearch@gov.wales

Social research number: 15/2023
Digital ISBN 978-1-80535-073-6

Image