Evaluation of Enabling Natural Resources and Well-being Grant: interim report (summary)

Aims and objectives of review

OB3 Research, has been appointed by the Welsh Government to undertake an evaluation of the ENRaW scheme.

The aim of the evaluation is to assess if ENRaW funded projects, delivered between April 2019 and March 2023, deliver against scheme aims and objectives. The evaluation is expected to review:

• the administration and delivery of the grant scheme by Welsh Government, including application processes, monitoring requirements and grant management with a view to identifying what has worked well and what could be improved in the future
• the direct impact of grants and whether the grants awarded achieved their original aims and objectives, including achievement of targets and outcomes set out in applications and delivery plans
• wider and unexpected benefits and achievements, including any wider multiple benefits over and above direct benefits expected.

The evaluation is being undertaken between October 2021 and October 2023. It has already involved the preparation of a Theory of Change (ToC) and Evaluation Framework report (unpublished, January 2022) and an Interim Report which considers the administration and delivery of the grant scheme and provides early findings on the impact of the scheme. It will also involve the preparation of a final evaluation report by October 2023 which will consider the impact and achievements of the scheme in greater detail.

This interim evaluation has involved:

• a desk-based review of recent policy and strategic documents, ENRaW scheme documentation and monitoring data, and project level information
• interviewing a total of 13 Welsh Government officials involved in the design and development of ENRaW and facilitating a ToC workshop with these officials
• developing and distributing a bilingual web survey to all ENRaW applicants and receiving 45 survey responses, a response rate of 47 per cent
• selecting a sample of 30 out of a possible 57 funded projects and undertaking in-depth interviews with their project leads. Further fieldwork was undertaken with partner organisations, beneficiaries, community representatives and Welsh Government policy officers across eight of these selected projects; in total, 91 individuals contributed to the fieldwork across the 30 selected projects
• interviewing one unsuccessful ENRaW grant applicant
• synthesising the findings of the fieldwork and desk-review and preparing an interim report.

The evaluation found that ENRaW has closely reflected Welsh Government policy and strategies and has been an excellent opportunity to support cross-policy projects spanning environmental, community, economic, health and wellbeing across Wales at scale and pace.

In terms of scheme administration and delivery, the evaluation found that:

• ENRaW has been promoted effectively, with energy and positivity on the part of Welsh Government staff
• demand for funding has been high, not least because the scheme has been designed to support 100% of project costs over a medium-term period of three years, and this has been an attractive offer to partnerships
• processing a high number of funding applications placed significant demands upon staff resources within Welsh Government
• Window 1 grant application, assessment and administration arrangements were found to be reasonable and appropriate, and a clear strength of this funding window has been the applicant’s ability to deal directly with a Welsh Government officer
• the transfer of the scheme into RDP mechanisms and the need to satisfy RDP funding requirements has been disruptive and detrimental to the smooth administration of the scheme; these changes took place prior to the Window 2 and 3 grant application process and coincided with the COVID-19 pandemic; the change in administration process was found to be causing significant stress for projects and is impacting negatively on their ability to deliver successful outcomes, not least because of the loss of staff, community, and partner goodwill to their projects

The evaluation found that in terms of alignment of funded projects with the scheme aims and objectives:

• projects are of good quality and delivering worthwhile environmental and community work
• projects are delivering activities in line with the RDP Focus Areas and Cross Cutting Objectives, as well as the Welsh Government cross-cutting themes, although as yet there is very little evidence in place to demonstrate how they are embracing gender mainstreaming
• projects are adopting very good, often creative, methods of using and promoting the Welsh language in a positive way
• ENRaW’s focus on collaborative working has encouraged and supported significant cross sector working which is unlikely to have been achieved otherwise
• there is good evidence that ENRaW has supported the establishment and development of new partnerships as well as enabled others to expand and strengthen, both geographically and in terms of the range of partners involved

In terms of funded projects achieving their aims and objectives:

• there is good evidence that Window 1 completed projects have delivered what they intended to deliver, and these projects have delivered outcomes which cut across different policies and sectors
• an appropriate set of scheme indicators is in place which is helpful to demonstrate the outputs being achieved; the scheme is performing well against its original targets and to date, projects have far exceeded the targets they set themselves; however, further work is required to capture and report more consistent metrics against these indicators to allow for the reporting of more accurate achievements
• completed projects have generated a wide range of community and social; health and wellbeing; and environmental benefits; to date, few completed projects can evidence that they have generated positive economic outcomes, other than using local contractors to carry out works, although there is some good evidence that projects have secured further sources of income
• the significant issues associated with Windows 2 and 3 means that these funded projects are unlikely to achieve all of their intended aims and objectives, mostly due to their shortened window of delivery but also because grant administration issues, such as securing prior approval for any spend over £500, is hindering progress and pace of delivery
• there is good evidence that projects and partnerships are being sustained post funding or are likely to be sustained post funding in the case of ongoing projects; projects and partnerships are being sustained in a variety of ways, including via other sources of grant funding, existing organisation funds or via community and voluntary ownership

In terms of lessons learnt from the implementation of the RDP funded windows, the evaluation found that:

• timescales and processes for approving scheme expenditure needs to be simplified and shortened
• funded projects should be afforded greater flexibility to accommodate changes to their budgets
• the current payment and claims system is unsuited to the administration of a large, complex project such as ENRaW

The evaluation offers three recommendations for the remaining ENRaW delivery period.

1. ENRaW should support funded projects to adopt consistent metrics when reporting the outputs achieved against the core programme indicators such as green spaces improved and hedgerows planted.
2. The Welsh Government should facilitate the sharing of experiences between funded projects and in particular the sharing of good practice approaches in relation to the cross-cutting themes, including the Welsh language, and RDP Focus Areas in order to encourage others to adopt similar practices and to address weaker areas such as gender mainstreaming.
3. The Welsh Government should allocate a dedicated funding officer from within RPW to ngoing RDP funded projects to help resolve financial claim and grant payment issues.

Post ENRaW, the evaluation recommends that the Welsh Government recognises, and builds upon, the strengths of ENRaW’s grant funding design, notably:

• the consultation process and the input of stakeholders into its co-design
• its intended long-term approach to provide funding over a three-year period
• its offer of both revenue and capital funding
• its full cost recovery funding model
• its focus on sustainable partnership and collaborative working across multi policy areas
• its focus on regional and landscape scale delivery

The Welsh Government reflects on the feedback from applicants and funded projects about the grant application, assessment, and claims processes to inform future approaches; given that it strongly suggests that Window 1 processes have been better suited to a funding scheme such as ENRaW. In particular, the evaluatorswould recommend that:

• a two-stage application process consisting of an EoI and full application stage be retained but that the EoI stage requirements be simplified and provide a greater opportunity for applicants to outline the intended purpose of their project
• timescales for assessing and approving applications be agreed in advance and adhered to
• a dedicated funding officer be allocated to each funded project to monitor progress and resolve any claims issues
• arrangements for submitting and approving claims be modelled on Window 1 processes which are better suited, and provide greater flexibility, to complex cross-policy projects 

Report authors: Bryer, N and Bebb, H (2023), OB3 Research

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Hannah Browne Gott
Email: climateandenvironmentresearch@gov.wales

Social research number: 22/2023
Digital ISBN 978-1-80535-537-3

Image