Evaluation of the Healthy and Active Fund: final report (summary)

This paper reports on the Programme Evaluation of the Healthy and Active Fund (HAF), a £5.8m+ investment in 17 Projects from 2019 to 2023 that enabled voluntary sector organisations and public bodies to explore how to support those with inactive lifestyles to become more active. It was designed and delivered through a four-way partnership between two Welsh Government policy teams (Health Inequalities and Healthy Communities Branch in Health and Social Services, and Sports Policy Branch in the Culture and Sport Division), Public Health Wales and Sport Wales. The HAF focused on those facing significant barriers to leading physically active lives. It aimed to sustainably increase their level of physical activity and improve their mental wellbeing. The activities through which the aims were to be achieved were not prescribed, thus making the HAF accessible for a wide range of organisations and enabling innovative approaches. 

A range of methods were used for the Programme Evaluation, including documentary reviews and semi-structured interviews with Programme and Project representatives.

By most measures the HAF was a success.  It worked effectively as a Programme, founded on an unusual/unique collaboration and budget sharing between two Government Departments, Public Health Wales, and Sport Wales. It was set up in exemplary fashion and created a strong and clear framework for inviting and approving bids It provided an operating framework using innovative arrangements, including designated Case Officers. It also demonstrated flexibility in supporting adjustments to funding and delivery models when the pandemic hit. Only one Project of the 17 ended prematurely, and that was for unavoidable Covid-related reasons. The HAF Team effectively put in place most of the necessary enablers for a successful Programme. It also succeeded in producing the intermediate outcomes of having a good range of effective Projects.

A wide geographical target audience and activity range was delivered.

HAF process (Chapter 2)

The HAF process was exemplary across a wide range of criteria. The national partners created effective governance and put into practice the conditions for developing an effective Programme in terms of accountability, strategy and approach, guidance and assessment, and resources and support, albeit this latter at some cost to the actors involved.

HAF theory of change (Chapter 3)

Our work on the HAF revealed that the right combination of people, with clarity of purpose, may not need an explicit theory of change to succeed in creating a credible intervention,although such a theory would strengthen clarity in linking Programme process to Programme purpose. This work identified that the HAF had a clear underlying theory of change. It highlighted the key enablers which underpinned the successful design and launch of the HAF, including an effective grant strategy and process; appropriate resources and support; clear guidance and assessment criteria; strong accountability; positive leadership and governance; and monitoring and evaluation processes. Our work also highlighted the critical role of collaboration. Ministerial priorities provided a strong impetus to collaborate, for example in budget sharing across Welsh Government and even with external partners. An explicit theory of change in a HAF-type programme, with multiple Projects funded on an application-response basis, can provide a clear and relatively simple test of alignment between the logic of intervention at project and programme level.

Delivery and Covid Adaptations (Chapter 4)

Despite the huge challenges of the pandemic, there is a legacy of learning and innovations in response to Covid-19, and many positive outcomes in the adaptations made by Projects and supported by the Programme. Several Projects retained changes they made due to their success and innovations. These included a mix of online and face to face opportunities, extending the reach of Projects and helping to remove geographical and confidence barriers for some individuals. Training approaches were adapted, and digital literacy was strengthened, and many Projects left a legacy of virtual resources. 

The pandemic strengthened partnership working, and the significance of community engagement to enable Projects to access their target groups emerged as a major theme. Projects experienced the necessity to invest time and resources in understanding their target audience to craft bespoke and tailored approaches to meet different needs. 

Tailoring monitoring and data collection methods to the particular activities and characteristics of each Project was critical. Standardised methods only suited some participants and activities. Supplementary methods were chosen by Projects in collaboration with their own appointed evaluators and/or internal advisers, and as a result varied widely.

Main Themes (Chapter 5)

Main themes explored as part of the Programme Evaluation throughout the main delivery years were (i) community engagement; (ii) sustainability; and (iii) the learning process. 

(i) Community engagement

The needs and demands of communities were sometimes very different from what Projects had envisaged at the outset, especially during the pandemic. Community engagement work was key to adjusting successfully as well as the ability to offer flexible, tailored approaches. The flexibility offered at Programme level supported Projects to adapt their plans. Effective partnership working was found to be central to community engagement and implementing community-centred approaches, along with the significance of developing relationships as early as possible, to provide an effective foundation for delivery and to help shape the kinds and modalities of activities and engagement offered by the projects. The pandemic stimulated stronger community leadership and promoted greater cross-agency working and signposting between services.

A face to face presence remained vital for developing trust and providing opportunities for one to one conversations with new/prospective participants. Having an on-the-ground presence was crucial for engagement with new target beneficiaries, especially when working with Black, Asian and Minority Ethnic communities. Those partners facilitated ‘introductions’ to potential beneficiaries. Project staff needed sufficient capacity to coordinate and develop partnerships with organisations that were connected to participants. Projects found that it was possible to offer more holistic support to communities and individuals through connections and inter-referral. 

Projects that had planned to work in partnership with the health sector and develop formal prescribing routes via GP surgeries, had to adapt their approaches because the NHS workforce was focused on responding to the pandemic. Nevertheless, some made significant progress and a wide range of referral systems were developed. 

(ii) Sustainability

Sustainability was an early and important consideration for the Programme and designed into the process. The tapering of funding in the final year was designed to encourage early consideration of exit routes and Project closure. 

(iii) Project level learning process

Both our literature review and evidence from Projects suggested that sustainability may not require a project to continue in its present form. It may be possible to achieve lasting impacts by instilling new norms that become ‘mainstream’ or to effect irreversible change. There can be a legacy of learning to inform policy and practice at strategic level, but also sustainability at the participant level, and sustaining activities through volunteer involvement. Sustainability issues will be further considered in an extension to the Evaluation that will take place in 2024.

The Programme focused on learning throughout. Online events for Projects were well-received. Formal opportunities for cross project learning and networking, organised centrally, were fewer in the last funding year. Projects could raise any evaluation-related questions with the Evaluation Group and support via Case Officers continued throughout the programme. This was welcomed, but it did not support cross Project learning. Projects felt that the HAF would have lasting impact on their own organisations’ thinking, policies and practices. Collaboration with partners enabled ongoing learning and reflection on optimal design and implementation to meet the needs of target beneficiaries.

Projects which established systematic internal learning arrangements benefited from them considerably. Projects’ final reports have many examples of transferring knowledge. Projects have been feeding learning ‘upwards’ but are unsure where and if this is being made use of. There have not been any clear mechanisms for sharing learning. 

Overall project achievements (Chapter 6)

Monitoring data submitted by Projects shows that 12,000+ people are recorded as having taken part in HAF Projects. Almost two thirds of participants where gender was recorded identified as female, partly related to the target beneficiaries of some Projects. Around nine percent of participants where ethnicity was recorded identified as non-White. Just under a quarter of participants who provided a response to this question reported a health condition or disability. Almost a third of HAF participants lived in Welsh index of Multiple Deprivation (WIMD) decile 1 and 2 areas, which represent the most deprived areas, and almost three quarters lived in WIMD decile 1 to 5 areas.

10,000+ survey tools were completed at baseline and 5,000+ at follow up. Of the Projects using the designated HAF tools, four showed increased physical activity and seven had inconclusive data, and five showed increased mental well-being and six had inconclusive data. Eight of the sixteen Projects that continued operating for three or more years met all the numerical targets they had set, and six met most or part of them. In many cases, Projects exceeded at least some of their targets. All Projects reported outcomes of improvements in physical health and mental well-being for beneficiaries and also suggested that HAF had considerable social benefits. Two Projects analysed the social return on the investments made and identified positive multiples of 3 to 5.

The HAF has fostered learning about working with target groups and has both reinforced existing knowledge and gained new insight.  Projects gained from the experience of working with groups they had not previously engaged with. Projects also reported economic benefits for partners and providers linked to the HAF. Project organisations overwhelmingly demonstrated their ability to flex to unexpected circumstances and adapt their processes to achieve their desired outcomes and demonstrated great resilience.

Overall programme achievements (Chapter 7)

Programme-level actors’ hopes and expectations

The key actors came to the HAF with a range of perspectives. They recognised that the HAF would not produce population level change. It had a more limited but still valuable role in exploring ways to engage hard to reach groups in supporting and improving their physical and mental wellbeing.

Programme and project management

Some key programme and project management principles were evident in the design and delivery of the HAF. Others, such as capturing the learning, were aspirations of the HAF Programme but realised to a lesser degree. 

Learning and policy transfer

Learning and the associated policy transfer to other interventions and policy domains was central to the HAF. The HAF generated useful learning at programme level. It is uncertain whether that is yet being systematically transferred/used, although this Evaluation may help to achieve that in part. 

Ways of working

The HAF demonstrated the effective ‘ways of working’ of the Well-being of Future Generations (Wales) Act 2015. The HAF initially succeeded in instilling the ‘five ways of working’ across the Projects through the HAF process. At Programme level, the five ways of working were an important reference point rather than a direct guide to action, and especially in the formative stages. The position in terms of both exhibiting the ways of working and also generating value from them, is not yet closed. There are Project level evaluations with significant learning to draw on, as well as this Evaluation. 

The principal area for strengthening the value of the HAF was in harvesting, codifying and amplifying both the Programme and the Project level learning, and ensuring that the asset was exploited effectively. That it did not was partly a result of pandemic conditions, but also structural weaknesses in governance and leadership, which were independent of the commitment and talent which many senior actors in the partner organisations brought to the HAF.

Welsh Government should continue to use a HAF style of funding programme where it is testing new approaches and seeking innovative solutions, but only where the key enablers to a successful programme have been put in place, and where such an approach is suited to the intended outcomes. 

Such programmes should observe core project and programme management principles, including providing for governance, leadership, and policy transfer during and at the end of the programme as well as at initiation. 

Collaboration should be actively encouraged involving partners from across Government and outside of it where appropriate, and should include budget, risk and governance sharing. 

There should be a dynamic communication and engagement strategy as part of project management. This should relate to all phases. 

An explicit part of the programme design should be thought-through arrangements for policy learning and transfer, preferably through a comprehensive and integrated Monitoring, Evaluation, Accountability and Learning Plan. 

Value for money considerations, in their widest sense, should be built into programme design and the project application process, recognising the inherent challenges of measuring the value for money of non-monetary benefits, especially in innovative and pilot projects.   

Theory of Change approaches should be part of the repertoire of tools available to those designing and implementing such programmes.  

A Case Officer role can add value in connecting Project to Programme levels in both directions. The role needs ongoing attention from Programme leaders.

Specific arrangements should be made to draw on learning at Project and Programme level and ensure that it is brought to the attention of relevant operational and governance actors, and actually applied.

Report authors: UK Research and Consultancy Services Ltd

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Dr Eleri Jones
Health and Social Services Research
Email: Research.HealthAndSocialServices@gov.wales

Social research number: 32/2024
Digital ISBN 978-1-83625-001-2

Image