Evaluation of the Social Services and Well-being (Wales) Act 2014: process evaluation (research briefing)

The Welsh Government commissioned a partnership between academics across four universities in Wales and expert advisers to deliver the evaluation of the Social Services and Well-being (Wales) Act 2014. The study is being led by Professor Mark Llewellyn (University of South Wales) in partnership with Professor Fiona Verity (Swansea University).

This report present findings from the Process Evaluation phase of the study. The evaluation, known as the IMPACT study, will examine the implementation and outcomes of the Act through its five principles (and the financial implications of each) and five domains as determined by the study team:

1. service users
2. carers
3. families and communities
4. workforce
5. organisations

The Process Evaluation focuses on domains 4 and 5, with a focus on the others to follow during 2021 during the Impact Evaluation phase (during 2021).

The aim is understand how the legislation has been implemented at a national, regional and local level, looking particularly at the role that the wide range of organisations that are impacted by the Act have had in this implementation.

The objectives of the process evaluation are to:

• consider what planning was undertaken by key partners for the implementation of the Act and whether this was sufficient (This includes planning by Welsh Government, Local Health Boards, Local Authorities, the third sector, the independent sector and other key partners that are considered to be relevant)
• assess whether all components of implementation to date have been completed as intended
• whether the components of the Act have permeated into practice
• assess interpretation of the Act at a national, regional and local level
• consider the experience of those involved in implementation, with particular focus on integration, co-production, leadership, management, interaction, training and provision of services in Welsh

The approach taken in the evaluation of the Act is that of Principles-Focused Evaluation.^[1] This approach is particularly useful in evaluating interventions that are complex with many components, and which will be variously interpreted and implemented in different environments and settings.

The data collection process used mixed methods via two distinct strands:^[2] a Wales-wide on-line pro forma (which yielded 30 responses); and qualitative data collection, via telephone interviews, face-to-face interviews and discussion groups.

In more detail, the core elements of the approach used for data collection were threefold:

1. Wales-wide survey of key stakeholder organisations and networks across Wales
   An online pro forma of 8 to 10 questions was developed based on the objectives of the process evaluation and sent to key stakeholder organisations. 30 responses were received
2. Stratified case studies on four local authority ‘footprints’
   Four local authority areas of Wales (Localities 1 to 4) were approached to take part in the process evaluation as representative of Wales’ communities: one predominantly rural, one predominantly urban, one predominately valleys, and one predominantly Welsh-speaking. Three different ‘strata’ of the workforce were engaged in those areas, and different organisations within the four footprint areas were included. In total, 140 interviewees took part across the four localities.
3. Interviews with key stakeholder organisations
   Qualitative research was carried out in addition to the ‘census’ in order to provide additional depth to the issues raised in the pro forma. 12 such interviews were undertaken. A list of key organisations was drawn from within the membership of the Study Expert Reference Group: ADSS Cymru, Social Care Wales, Care Inspectorate Wales, WLGA, WCVA, NHS Confederation, Older People’s Commissioner, Children’s Commissioner, Care Forum Wales, among others.

It is important to note that the data collection took place prior to the COVID-19 pandemic, between January and March 2020.^[3]

[1] Patton, M. (2018) Principles-Focused Evaluation, The Guilford Press, London.

[2] In addition to this, the study team are working on analysis of the published data – from Welsh Government, Social Care Wales, Data Cymru and other official sources.

[3] Given that the Welsh Government has now extended the overall end date for the study by 12 months to October 2022, this will allow for a second process evaluation phase in Spring 2022 which will permit the study team to follow up on the issues raised in this first phase.

Principles of the Act

• There was a recurring view that the principles of the Act form an important values-based framework for action.
• Implementing the principles of the Act was seen to need time on an ongoing and continuous ‘journey’ of change.

Prevention

• There are some positive examples of prevention models and practices, but this is seen by some as patchy.
• There are mixed accounts of the investment in prevention, with reports of underinvestment as well as some allocations to prevention.

Co-production

• There were positive examples of co-production in the development of individual and community interventions for care and support.
• Challenges were noted in securing greater leadership support for co-productive ways of working, continuing to shift professional expert paradigms, and responding to the intrinsic complexities of co-production processes.

Well-being

• Well-being seen as integral to social care, but as a concept is contested and subject of much discussion.
• Enabling well-being requires the implementation of all the principles of the Act.

Voice and control

• Examples given of positive interpretations of the impact resulting from a focus on ‘voice and control’ and more awareness of how supporting voice and control can be better undertaken.
• Fragmentation and overlap of advocacy services was reported, along with the need to keep raising awareness of the importance of advocacy.

Multi-agency working

• Strong commitment to and positive examples of multi-agency working.
• However, there is fragility, gaps and inconsistencies in multi-agency working.
• Multi-agency work based on trust, relationships, communication and organisational capacities to support this work.

Act Implementation

• The Act has enabled new ways of working including practice change, and developing and strengthening partnerships).
• Preparatory work and planning in readiness for implementation (for example, service remodelling, information gathering, workforce training and ensuring compliancy with the Act) was broadly effective.
• Numerous descriptions of how the Act had supported change were offered. For example, the Act was referred to as offering validation and legitimation, as a catalyst to drive and deliver change and as an enabler.
• Implementation and the shift to a new way of working is an ongoing process with acknowledgement that implementation is a journey.
• The naming of the Act was considered problematic, leading to misconceptions about other organisations duties and responsibilities, and in particular, health.
• Lack of public awareness and understanding of the Act has created challenges to asset-based way of working (for example, Citizen, service user, carer expectation management, lack of knowledge and understanding of what it means)

Local authority and social services functions

Assessments

• New approaches which embodied the emphasis on strengths- and asset-based assessment under the Act in understanding people’s eligible need around well-being were evident.
• Participants reflected on the disconnect between legislative rhetoric and operational reality, especially when faced with the tensions between local flexibility and interpretation versus centralised control.

‘What matters’ conversations

• ‘What matters’ conversations were reflected on by participants in largely positive terms, seen as a return to good practice.

Outcomes

• The challenge for the workforce is that outcomes are subjective and contested, and not fixed or standardised in how they are assessed or collected.
• There was an overall sense of the move towards outcomes being ‘work in progress’.

Commissioning

• There was a sense that practice had evolved such that commissioning for the principles and outcomes of the Act had been realised, but there was considerable progress still needed.
• The main issue identified with commissioning during the implementation of the Act was the lack of co-ordination between authorities leading to 22 distinct and different ways of doing things.

Safeguarding

• In respect of safeguarding, the Act was perceived to have brought in useful changes.
• Respondents noted that the new Wales Safeguarding Procedures provide, to date, positive challenges to established ways of working and much needed clarity and continuity.

Operational relationships with partners

• The importance of leadership to initiate and sustain change is clear.
• There is great value placed on positive, reciprocal working relationships with partners.
• The Act is a driver and lever for developing partnerships with health.
• The Act has, to an extent, enabled the integration of social care and health to develop in respect of collaborative regional approaches, commitment and buy-in from leaders, integrated working spaces, mutual respect and trust, and consistent messages to both organisations.
• Time and resource are required to build effective partnerships.
• The voluntary sector is an excellent partner on the whole, but concerns over capacity, funding and sustainability persist.
• Competing ‘cultures’ of different organisations, especially social care and health, need to be further reconciled.

Strategic relationships with partners

• Boards and structures have been a key aspect enabling the formalising and strengthening of partnerships between social care, health, and other agencies.
• Regional Safeguarding Boards were especially viewed as positive developments to enable regional working.
• Work is required to continue to develop the structure of RPBs, and to improve relationships between the RPB and the PSB.
• In more general terms, the size of the regions presents challenges to in-depth discussions about health and social care integration.
• Applying ‘a one size fits all’ regional approach is problematic in responding to sub-regional and locality issues.

Workforce

• Managing and developing social workers is an ongoing process.
• The provision of support from service and team managers is crucial in the further implementation of the Act.
• The change to an asset-based way of working has had a positive impact on the workforce. Benefits included an opportunity to work differently, job satisfaction and motivation, and the value of working with individuals to achieve outcomes.
• Increasing caseload, decreasing capacity and pressures of demand are rate-limiting factors on the continued ‘journey’ of implementation, impacting the ability and extent to which the workforce can initiate and sustain change, specifically in respect of:
  • increasing demand and complexity
  • workforce capacity exacerbated by recruitment and retention issues

Data

• Capturing ‘softer’ qualitative outcomes is seen as a welcome shift from solely quantitative data collection and measurement.
• The Act has helped prioritise the focus on individual outcomes.
• There are identified advantages, and frustrations, associated with the functionality of WCCIS, the Welsh Community Care Information System.
• Challenges to capturing and evidencing data include:
  • uncertainty of how best to do it, how to best to report it, a continued emphases on quantitative data, and time/capacity to do it meaningfully
  • difficulty attributing positive outcomes to a particular type of support or intervention
  • learning to use WCCIS effectively, especially in respect of data extraction, analysis and reporting

Financial and economic implications

• Participants noted that the implementation phase for the Act came about at a difficult moment for public services, in the middle of a period of austerity.
• Negative impacts of austerity and financial pressures were identified and linked to:
  • overspending within social services and a reduction to the statutory provision of services
  • controlling demand and lowering expectations whilst trying to manage competing demands within local government
  • consequences of funding arrangements, and questions about the overall effectiveness of such short-term funding arrangements
• There is an identified disconnect between the rhetoric and reality of pooled budgets and questions about their role.
• Several difficulties associated with short-term funding were identified, primarily the lack of stability and sustainability in such arrangements, like the opportunity costs of servicing grants, and time lost in managing staff contracts.
• The Integrated Care Fund (ICF) was perceived to have helped facilitate work in line with the principles of the Act, but could have potentially had greater impact if it had been better planned.
• Savings were also linked to the positive impact from working alongside the Act, with an emphasis on partnerships and sharing the burden across different organisations.

There is considerable complexity (and a degree of contradiction to an extent) in the findings presented in the report. The comments provide very many perspectives across the whole of social services and partner organisations.

Given this, Figure 1 provides an attempt to rationalise the complexity we have seen. It offers a conceptualisation of the feedback received into two principal domains: a transformation modality, and a continuation modality (which has three forms itself).

Change is taking place in two ways. The first is within which forms of activity and practice continue on, where things to a greater or lesser extent had already been established prior to the Act’s implementation. The second concerns forms of practice that in order to meet the duties and requirements of the Act, required an element of transformation.

Image

It suggests that due to the differential starting points of all of the localities in Wales, and especially the four localities researched in this phase of the study, four different forms of practice have been embodied which have require different amounts of change:

• ‘Continuation | (Pre) Aligned’ echoes those professing that they were already doing what the Act outlined ahead of implementation
• ‘Continuation | Acquiescent’ describes those (reluctantly) moving to a new form of practice under the Act
• ‘Continuation | Absorptive’ reflects continuity with extant practice simultaneous with those keen to adopt new forms of practice
• ‘Transformation’ presupposes the greatest amount of change, reflecting new aspects of practice or infrastructure under the Act

Priorities for further implementation and next steps for Welsh Government

Priorities for further implementation of the Act included the continuation and development of integration and partnerships, monitoring and evidencing outcomes, and the infrastructure to facilitate integrated working.

To realise the full potential of the Act, the importance of an open dialogue between Welsh Government and local authorities, and reciprocal working relationships was highlighted. Data monitoring and capture to evidence outcomes and impact were also seen as a priority.

Messages for what the Welsh Government should do next included a move from short-term to long-term approaches to develop/sustain community models and resilience, and longer-term sustainability including funding, consistency of practice, and whole system buy-in and transformation.

Comparisons were made between health and social care with a call for parity of funding between the two. Sustainability was highlighted in the context of funding but also a lack of consistency in practice across Wales.

There was an emphasis on Welsh Government helping to ensure the Act is embedded and to promote recognition that its duties apply to all organisations providing care and support.

Funding was only one element seen as enabling long-term sustainability. Other important aspects included consistency in practice and embedding the Act across Wales, and a ‘whole system transformation’, driven by leadership that focusses on valuing the workforce.

Closing thoughts

The next phase of the study (January 2021 onwards) will hear extensively from all service users, carers, families and communities to ensure that we provide balance against the perspectives from the workforce provided through this process evaluation report. Ahead of that, the following represents the start of a developing list of questions to be carried forward into the impact evaluation phase of the study.

• How well are the expectations of service users and carers being met in their experiences of social care and social services?
• To what extent is co-production embedded in social services practice and assessment?
• How have service users experienced care assessments and case management since the Act was introduced?
• To what extent are service users given voice and control in their interactions with social services?
• To what extent are financial pressures and austerity impacting frontline services, practice and care provisions?

The Act clearly has legitimised change, and has been a catalysing force in the development of social services, and local authorities’ relationships with key partners in health, the voluntary sector and the independent sector. Four years after the Act came into force, there is considerable evidence of the difference made, but also in respect of the difference still to be made. It’s important to recognise that whilst implementing the Act is an ongoing journey, it is also the law.

Mark Llewellyn, Fiona Verity, Sarah Wallace and Sion Tetlow

Llewellyn M., Verity F., Wallace S. and Tetlow S. (2020) Evaluation of the Social Services and Well-being (Wales) Act 2014: Process Evaluation. Cardiff. Welsh Government, GSR report number 2/2021.

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government

For further information please contact:

Rebecca Cox
Social Services and Integration Division
Welsh Government
Cathays Park
Cardiff
CF10 3NQ

Email: rebecca.cox@gov.wales

Social research number: 2/2021

Image

Digital ISBN 978-1-80082-717-2