Pan Wales Rural Tourism evaluation (summary)

The Welsh Government’s Pan Wales Rural Tourism schemes have been delivered under the Rural Communities-Rural Development Programme (RC-RDP) 2014-20 (page 154) which aims to ‘promote competitiveness and create sustainable growth and jobs for the people who live and work in rural Wales’.

Four funding schemes were delivered via Pan Wales Rural Tourism over a four-year period between 2017/18 and 2020/21:

1. Micro and Small Business Fund (MSBF): a capital fund of between £25,000 and £500,000 for private and third sector enterprises and which could support up to 40% of eligible costs.
2. Tourism Amenity Investment Scheme (TAIS): a capital fund of between £25,000 and £128,000 available to public and not for profit organisations to fund small-scale tourism infrastructure improvements and which could support up to 80% of eligible costs
3. Tourism Product Innovation Fund (TPIF): a revenue fund of between £30,000 and £150,000 to support joint working between tourism partners in the public, private and third sectors and which could support up to 90% of eligible costs.
4. Regional Tourism Engagement Fund (RTEF): a revenue fund of between £30,000 and £150,000, open to any organisation who would lead a collaborative proposal to promote and develop high-quality visitor destinations and which could support up to 90% of eligible costs.

MSBF was administered on a rolling application basis whilst two rounds of funding were administered for the other three schemes. Each scheme adopted a two-stage application process which involved an Expression of Interest ^{[footnote 1]} and a full application.

A total of 140 projects were funded across the four grant schemes between 2017/18 and 2020/21: 50 were MSBF projects, 43 were TAIS projects, 25 were RTEF projects and 22 were TPIF projects. In total, £13.1 million grant funding was awarded to these projects. The total value of the funded projects’ costs amounted to £28 million, with £15 million provided in match funding ^{[footnote 2]}.

In addition to the £13.1 million project level grant funding, a further £5.6 million, via the RTEF and TPIF schemes, was made available to fund four Visit Wales ‘Year of’ thematic campaigns.

OB3 Research was appointed by the Welsh Government to undertake an evaluation of Pan Wales Rural Tourism.  

The aim of the evaluation was to review the four tourism schemes and provide an independent assessment of scheme implementation and delivery, including the outcomes and impact of the schemes. The evaluation was expected to:

• consider the alignment of the projects with the RDP and Welsh Government strategic policy objectives
• review the effectiveness of scheme management – including the claims process, monitoring systems, communications, and the availability of support post award
• consider the delivery and implementation of the schemes
• assess the extent to which supported projects were able to achieve the objectives set out in their plans
• review outcomes and impact, where possible, of the schemes
• assess the responsiveness of the various schemes to the perceived needs of the tourism industry, including how support was offered to different sizes of businesses, and larger or smaller scale projects
• assess value for money (VfM) where possible
• provide recommendations and lessons learnt for similar grants in Wales

The evaluation was undertaken between September 2022 and July 2023. It involved the preparation of a Theory of Change (ToC) report (unpublished, November 2022) and a final evaluation report. 

The evaluation involved:

• a desk-based review of relevant policy and strategic documents, scheme documentation, and tourism and visitor data
• interviewing eight Welsh Government officials involved in the design and development of Pan Wales Rural Tourism and facilitating a Theory of Change workshop with these officials
• developing and distributing a bilingual web survey to grant funded recipients and receiving 61 survey responses
• selecting a representative sample and undertaking in-depth interviews with 24 funded projects; further fieldwork was undertaken with users and visitors across four of these projects
• interviewing three other key stakeholders and receiving a written submission from a fourth organisation
• synthesising the findings of the desk research, fieldwork and survey data and preparing a final evaluation report

In terms of alignment of projects with RDP and Welsh Government strategic policy objectives, the evaluation found that:

• there has been a long-standing supportive policy context for Pan Wales Rural Tourism and the Welsh Government’s commitment to the tourism sector was reinforced following the COVID-19 pandemic
• Pan Wales Rural Tourism supported the delivery of Welsh Government policies as capital and revenue investment was made in projects which enhanced the tourism product, encouraged greater collaboration, promoted the tourism offer, attracted more visitors, improved visitor satisfaction, and grew the sector
• funded projects were well aligned with Visit Wales’ priorities; there was also strong alignment with Visit Wales’ most recent annual marketing themes of ‘Year of Discovery’ and ‘Year of Outdoors’, but perhaps fewer funded projects supported earlier themes of ‘Year of Legends’ and ‘Year of the Sea’
• it was not possible to extrapolate the impact of Pan Wales Rural Tourism upon the tourism sector in Wales because tourism data is not available for rural areas only and the data would have been impacted by external factors such as the COVID-19 pandemic and current cost of living crisis.

The evaluation found that in terms of the effectiveness of scheme management:

• the four schemes were managed appropriately and effectively; grant application processes were rigorous, and funding was allocated in an equitable and transparent manner
• the adoption of a two-stage application process was appropriate and application guidance was fit for purpose
• the time taken to assess applications was reasonable and acceptable to the sector
• there is scope in the future to simplify the application and administration processes for those accessing smaller amounts of grant funding
• a key strength of the grant management process was the role played by Visit Wales officers who supported applicants and grant holders, and this should continue in the future
• scheme managers were able to incorporate lessons learnt into delivery over time, and there is benefit in adopting a flexible approach to grant management in any future scheme
• preparing financial claims was the most onerous element of the scheme administration for grant holders and future financial claims processes could be simplified, particularly for smaller grant funds
• reporting requirements and evidencing outputs was reasonable and proportionate although there is some suggestion that scheme level outputs were under-reported due to the lack of follow up monitoring visits

In terms of the delivery and implementation of the schemes

• a wide geographical range of projects were supported, and it was logical that north Wales accounted for a higher proportion of projects and funding compared to other regions due to the high number of local authorities which accessed TAIS funding and the importance of tourism across this region
• performance against scheme-level targets was very mixed, with cases of significant over and under-achievement alike despite some targets being revised downwards; some targets were set at too low a level and others were not achieved because of external factors such as the pandemic; it is difficult to offer a view on how well the schemes performed against their funded targets, as the targets set were not realistic
• the outputs reported for those targets associated with Visit Wales marketing activities were calculated using appropriate standard industry methods and show that the campaign helped to maintain awareness of Wales as a potential destination amongst UK residents thereby influencing visitor spend which in turn supported the visitor economy and employment; this approach is limited however as it does not account for the impact of other factors
• future funding schemes would benefit from adopting targets which are more closely aligned with the direct outcomes of funded activities

In terms of the extent to which supported projects were able to achieve their objectives:  

• funded projects performed well against their aims and objectives
• the main enablers to strong performance were detailed up-front planning, effective collaborative working, skilled and experienced delivery teams, and careful procurement; flexibility on the part of the funder and grant recipient was also a vital component of successful implementation
• the COVID-19 pandemic generated many of the delivery challenges experienced by grant holders from 2020 onwards and accounted for much of the under-performance against funded targets; other delivery challenges experienced by grant holders included increased costs, staffing issues, and partnership working
• in terms of cross-cutting themes there was good evidence that funded projects contributed positively towards innovation, equality of opportunity, sustainable development, addressing climate change and the Welsh language; there was less evidence that funded projects contributed towards tackling poverty and social exclusion in rural areas

In terms of the outcomes and impact of the schemes:

• MSBF funding was used appropriately to extend and improve the quality of facilities for visitors, and in the absence of the support many developments would have been less ambitious in their scale and quality. There is convincing evidence that large-scale capital projects would not have progressed at all in the absence of grant funding. Changes in visitor demand for MSBF funded projects, whilst positive, cannot be disentangled from the wider effects of external factors such as the pandemic and current increased living costs. Where demand increased, businesses believe that project funding positively contributed towards the change. More modest changes in visitor demand during off-peak and shoulder seasons were reported by MSBF businesses, suggesting that the investment had less of an impact on addressing this policy objective
• TAIS funding was used effectively to implement basic infrastructural improvements across popular tourism areas which are expected to result in better visitor experiences at key destination sites. In the absence of the fund, these improvements would not have taken place. There is some evidence available to show that there has been increased use of TAIS funded facilities, but similar future schemes would benefit from adopting and capturing more comprehensive data on visitor numbers and experiences
• RTEF projects helped to strengthen collaborative working across the tourism sector and these collaborative arrangements are being sustained post funding. There is less evidence of TPIF funded collaborations being sustained
• a mixed picture emerged in terms of whether RTEF and TPIF promotional and marketing resources were being used post funding, with less evidence of TPIF projects doing so
• a mixed picture also emerged in terms of the difference made by RTEF and TPIF projects, and a key message is that there was very little data available to evidence anecdotal increases observed by grant holders.

In terms of the responsiveness of the various schemes to the perceived needs of the tourism industry:

• the four funding schemes were well received by the tourism sector and a key strength was the co-ordinated approach to investment and addressing funding gaps
• the schemes focused on improving the quality of tourism facilities within rural Wales thereby contributing towards Welsh Government policies and ambitions for the sector

In terms of a value for money assessment, across MSBF:

• the cost per job created or safeguarded was higher than anticipated (£13,468 compared to the target of £10,000 per job created or safeguarded)
• grant funding of £4.5 million contributed to a turnover growth of circa £12.7 million per annum across supported businesses
• value for money would have been higher had the pandemic not impacted negatively upon business performance

In terms of lessons learnt for similar future grant schemes:

• a key gap in investment support is small-scale capital grant funding for the private and third sector. If introduced, this should be used to invest in regenerative tourism developments which support local communities; environmental sustainability initiatives and support businesses to become net zero; and improving accessibility within private sector providers

The evaluation offers the following recommendations for the Welsh Government to consider.

1. We recommend that the Welsh Government explores the need for a small-scale capital grant funding for the private and third sector as a successor fund to MSBF.
2. The criteria adopted to award funding via the MSBF should be retained on the basis that it is well aligned with Welsh Government policy objectives.
3. Should a successor scheme to the MSBF be introduced, it should be used to support the three key priorities of regenerative tourism, environmental sustainability and improving accessibility.
4. The Welsh Government should explore whether there is a need for continued revenue funding to support collaborative tourism marketing activities as a successor fund to RTEF and TPIF.
5. Should a successor revenue fund be introduced, the criteria adopted to award funding should be reviewed to ensure that it supports (i) meaningful collaborative activities which contribute towards regional and local destination management plans and (ii) resources which can be sustained and utilised post funding. A future revenue fund should not be geographically restricted to rural areas.
6. Future funding schemes should continue to adopt a two-stage application process, using an initial Expression of Interest to check eligibility and a full application for those considered eligible to progress further.
7. Application and grant administration processes, to include financial claims processes, for those wishing to access small amounts of funding should be simplified.
8. Future funding schemes should retain the role of a dedicated contact officer to support grant applicants and grant holders.
9. Future funding schemes should be designed in a flexible enough way to respond to the needs of grant holders, particularly in terms of the delivery period available for projects.
10. Future funding schemes should adopt more realistic and directly achievable targets. Greater time and consideration need to be allocated upfront when developing targets to ensure that they better reflect the direct outcomes of funded activities and can be monitored and reported upon.
11. Greater consideration should be given to the approach adopted for monitoring and reporting of any targets set for future funding schemes. The Welsh Government should provide detailed guidance to fund holders on monitoring indicators and data collection methods which they expect them to adopt, to better record the impact of funded projects in terms of changes to visitor numbers and experiences.
12. The Welsh Government should ensure that follow up monitoring visits to supported projects are undertaken to capture all outputs achieved within any future funding schemes.

[1] The term Introductory Questionnaire (IQ) was also used by the Welsh Government to describe this initial form, and both terms were used interchangeably across the funding schemes. For the most part, the evaluation report refers to Expression of Interest.

[2] The data supplied by the Welsh Government shows that no match funding was provided for 21 of the projects funded during the second round of RTEF and TPIF.

Authors: Bryer, N and Bebb, H

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Siân Hughes
Email: tourismresearch@gov.wales

Social research number: 81/2023
Digital ISBN 978-1-83504-572-5

Image