Timber Business Investment Scheme (TBIS) evaluation (summary)

The Timber Business Investment Scheme (2014 to 2020) (TBIS) was a £9m capital grant scheme available to private forest and tree nursery owners, local authorities, or other public sector forest owners and small to medium-sized enterprises (SME) that were engaged in forest planting and management activities, timber harvesting, and/or timber processing. The scheme was funded through the Rural Development Programme (RDP) 2014 to 2020 and was delivered between early 2016 to June 2023.

The rationale behind the scheme was to use targeted investment to provide opportunities for timber businesses to grow, to improve the condition of woodlands in Wales, and to bring inaccessible woodlands into management. A sizeable proportion of woodlands in Wales was unmanaged or undermanaged, often due to its inaccessibility, and thus required different types of machinery for sustainable production.

The principal aim of the evaluation was to provide an independent assessment of the implementation and impact of the scheme within Wales. This will be used to form the evidence base with which to inform future decisions relating to investments supporting the timber industry in Wales. Eight key evaluation questions were assigned for this study, as follows.

1. How effectively was the Expression of Interest (EOI), application and appraisal process implemented?
2. What was the level of engagement with beneficiaries and potential applicants? How effective were these processes?
3. How effectively was the scheme designed, administered and coordinated, including the claims process, monitoring systems, communications, and the availability of support post-award?
4. To what extent did TBIS meet the targets set for the scheme?
5. What was the overall impact of the TBIS (with particular consideration of any economic and environmental impacts of the scheme)?
6. What has been the overall value for money of the scheme?
7. To what extent did the scheme align with other support services and Welsh Government and European Union (EU) strategic objectives?
8. What lessons learnt and recommendations for the current scheme can be provided from the assessment and evaluation of the TBIS?

A mixed-method research approach was used for the final evaluation, drawing on qualitative and quantitative data. The evaluation was based on the following activities

• Two rounds of in-depth interviews with Welsh Government officials and stakeholders.
• A survey of 31 businesses that had received the grant support (i.e. the beneficiaries), equating to a 48% response rate.
• A survey of 39 businesses that had submitted an EOI but did not receive support (i.e. non-beneficiaries), equating to a 33% response rate.
• A review of the management information (MI) data collected by Welsh Government.

There were four notable limitations to the evaluation. Firstly, a large proportion of projects (26%) were still active when our research concluded in February 2023, with some beneficiaries still being in the process of purchasing the equipment, thus limiting our ability to establish the full, eventual impacts (although this was mitigated by exploring future potential). Secondly, the sample lacked statistical robustness with the beneficiary and non-beneficiary groups proving hard to reach, although we note that the sample sizes do provide suitable, indicative findings. Thirdly, the economic impact assessment lacked robustness with a lack of detailed supply chain mapping (which would have required greater resource), although it was sufficient to provide a strong indication of the impact generated by the scheme. Fourthly and linked to the last two points, only 65% of beneficiaries were happy to share turnover data, thereby further limiting the economic impact analysis.

Effectiveness of the EOI, application and appraisal process

There was a mixed response from beneficiaries with regards to their satisfaction with the application process, with 42% reporting dissatisfaction while a similar proportion were satisfied. The issues most commonly cited by beneficiaries concerned the onerous and time-consuming nature of the application process, while many spoke about the general complexity entailed with submitting an application. More than a third of beneficiaries felt a need to hire a consultant to help develop the application for them, which raises questions as to the equitable nature of the scheme and the effectiveness of the selection process (i.e. that it was not necessarily the best projects that get funded, but also partly about which businesses were large enough to acquire support from the private sector).

Delivery team members concurred that there had been issues with the process, noting lengthy delays and particular issues with the stage 2 full application where there was an insufficient uptake rate from EOI through to the full application. These delays were compounded by external events such as Brexit, the COVID-19 pandemic, and the war in Ukraine (with its associated inflationary effects). To compound matters further, there was no flexibility within the scheme to increase the grant level in order to account for the rising costs of the equipment, meaning that beneficiaries were left to deal with the brunt of the costs associated with inflation. There was evidence of a disconnect between the two application stages, with two separate teams working with two processes, thereby making it more difficult to ensure that applicants had a smooth transition through the application process.

The delivery team believed that the appraisal process was effective, highlighting that they had an appropriate mix of personnel assessing the applications and succeeded to support a good and suitable group of beneficiaries. Their one reservation was that the scoring criteria favoured larger applicants due to the value for money aspect within the criteria, which favoured applicants with larger areas of woodland. An adjustment was made in the final application window to address this issue.

Effectiveness of engagement with beneficiaries and potential applicants

The scheme was promoted through a modest marketing campaign which primarily utilised industry channels. The team felt that this was appropriate, given the small and close-knit nature of the sector. That being said, applicants generally felt that there was not a particularly high level of awareness of the scheme within the sector.

The facilitation support through the application process and thereafter seems to have been limited, with Welsh Government only allowed to advise on policy and scheme rules – they were not allowed to advise businesses on their projects. Where support had been received, beneficiaries provided a mixed view regarding its usefulness, which is partly related to a lack of resource within Rural Payment Wales (who were responsible for delivery) to respond to queries promptly and in-person. The vast majority of applicants would have valued receiving other support to help navigate the application process.

Beyond the application process, very few beneficiaries received any ongoing support. Generally, beneficiaries were satisfied with the claims and monitoring processes, however a sizeable minority (23%) expressed dissatisfaction with beneficiaries citing time delays and the bureaucratic nature of the process. There was a lack of flexibility, largely driven by RDP regulations, with beneficiaries often being penalised for factors that were outside of their, and the delivery team’s control. The delivery team acknowledged that the lack of flexibility and resources had been an issue throughout the scheme and that “every stage of the process has taken too long.”

Effectiveness of scheme design

The decisions made with regard to the design of the scheme were generally well received by applicants, with there being broad support for the flexibility and appropriateness of the grant size, the intervention rate, and the eligible costs and activities. On a more strategic point, however, there perhaps ought to have been greater clarity as to what the scheme was seeking to achieve and the types of investments that it should make. For example, there was initially an expectation that the projects would be predominantly environmentally focused, but the bulk of activity appears to have been more economically focused. Additionally, the way the scheme has been designed did not lend itself to meeting some of the other objectives such as supporting the smaller organisations (including nurseries) and retaining supply chain spend in Wales (although a change was made in the final application window to address this).

Achievement against targets

The data reveals that just 71% of the £9m grant budget was successfully claimed by beneficiaries. Whilst the demand was plentiful, a combination of internal and external issues led to attrition in the amount claimed. Perhaps the main limitation was that the budgets were allocated on a yearly basis, with any underspend returning to the RDP ‘pot’ and being used for other priorities. Accordingly, they were not able to reallocate funding shortfalls from one window to the next.

As for the key performance indicators (KPIs), the scheme supported fewer operations than anticipated (74% of the target), although it performed well against all other KPI targets. It exceeded the target for jobs created and significantly outperformed the target for jobs safeguarded. In addition, it achieved well beyond the target for area of woodlands supported, which was more than 45 times the target. We note, however, that the target figures do appear to have been very modest and, thus, provide limited insights into the value for money generated. Accordingly, we supplement this review with a more detailed assessment of the impacts and value for money.

Impacts

Most beneficiaries believe that they had achieved what they had set out to achieve whilst Welsh Government inspectors came to the same conclusion in their in-situ reports. Beneficiaries cited a wide range of results with the main themes comprising increased production, accessing more difficult terrains, and accessing new markets. Linked to this, 30% cited an impact on their sales whilst others cited increased efficiencies with most reporting that the project had led to a reduction in their costs. Accordingly, the evidence certainly suggests that the scheme has had an impact with regard to improving the capabilities of the timber sector in Wales.

Our economic impact assessment estimated that TBIS created or safeguarded 71 jobs in total, equivalent to ca. £110k in investment for each job created or safeguarded, although we note that this estimate is limited by the small sample size and other constraints within the methodology. Most beneficiaries also reported that their project had led to an improvement in the quality of employment they provide, including 77% citing increased job security (e.g. avoiding zero hours, temporary contracts, hours of operation), whilst 55% reported that it had led to improvement in the skill composition of their workforce.

The analysis of turnover data further suggests that TBIS has generated growth for beneficiaries and an estimated £2.5m in net additional Gross Value Added (GVA). Beyond this, there is strong future potential for further economic returns, particularly when considering that 30% of projects were still active at the time of our survey while many beneficiaries felt that it would take more time to see the economic impact. Based on a tentative assessment of turnover projections and the potential impact from TBIS on future turnover, our estimate suggests that TBIS has the potential to generate a further ca. £5.8m in GVA impacts over the next year (at the time of this report). We would caution, however, that these estimates are more speculative and more susceptible to optimism bias.

While the drivers were more economically related, and beneficiaries’ responses suggest that this has also been the main impact, we also found evidence of environmental outcomes. Most beneficiaries had developed less intensive and environmentally damaging processes, most were able to reduce the dangerous tree population or improve woodlands conditions, whilst around half had brought new woodlands into sustainable management (mostly from farm woodlands).

Alongside the value generated from increased turnover and employment, there is also economic value associated with the environmental impacts. For instance, bringing new areas of woodlands into sustainable management is key in growing the value of the sector in Wales, with the economy being able to benefit from areas of woodlands that were previously left idle. Additionally, the impact from safeguarding woods and their associated biodiversity for future generations alongside climate change mitigation has an economic impact. Accordingly, whilst not quantifiable, the economic impact generated is likely to be higher than the direct returns for beneficiaries.

The scheme does appear to have represented strong additionality. We found that none of the beneficiaries would have been able to make the same investment at the same time without the support, and around one quarter would not have been able to make an investment at all. 65% reported that they would have made a different or smaller investment had the grant support not been available, while 42% would have waited longer to make the investment. Beneficiaries further reported that 75% of the investment made would not have occurred without TBIS support. On this basis, we estimate that the scheme has leveraged £6.4m of investment from businesses in their capabilities, which would not have occurred without the support.

Strategic alignment

There has been strong strategic alignment between TBIS and broader policy objectives in some respect. For instance, there was certainly strong alignment with several of the Woodlands for Wales strategic targets regarding developing greater sustainable management of woodlands. Equally, there was perhaps insufficient clarity with regards to the scheme’s strategic vision as we have already discussed. For example, the investment was not effectively targeted in a way that would meet Welsh Government’s objectives around woodland creation (a primary objective) as the scoring criteria made it difficult for nurseries to access the finance. Additionally, whilst part of the ambition of the scheme was to retain more supply chain value in Wales through taking up a market share from non-Welsh suppliers, there is very little survey evidence of this taking place. Beneficiaries’ sales were almost exclusively to Welsh customers and the scheme had next to no impact on that.

Value for money

Generating strong value for money relates back to delivering the vision and impacts within the theory of change developed for the scheme. These included ambitions to: provide opportunities for timber businesses to grow; improve the condition of woodlands in Wales; bring inaccessible woodlands into management; increase the production of timber in Wales; generate a positive return on investment; retain more of the supply chain value within Wales; and generate environmental benefits. The scheme has successfully delivered against most of these objectives, with the exception of generating a positive return on investment and retaining more of the supply chain value within Wales.

If we were to judge the value for money purely based on the turnover-related economic returns generated to date, there would be some doubt as to whether the scheme could be described as a success. However, alongside the economic value already generated for beneficiaries, the survey evidence suggests that the scheme will continue to generate economic returns over the next year, which will potentially outstrip what has been spent through grant funding. Additionally, the scheme has leveraged £6.4m in investment from businesses and, undoubtedly, developed the capability within the sector. There is also strong evidence that it has generated important environmental impacts, which have significant economic value attached to them. Put together, the evidence indicates that the scheme has demonstrated good value for money.

This evaluation has demonstrated that, whilst there have been several challenges and issues within the delivery of the TBIS, it has largely been able to deliver its remit. Overall, the scheme can point towards several achievements, including the increase in sustainable management of woodlands in Wales, growth within the businesses supported, and environmental impacts. Stakeholders felt that there is a need for this type of support going forward, although future interventions should have a clearer identity and remit.

Based on these findings, we make the following recommendations.

Recommendation 1

Any future intervention should establish a clear role and remit from the outset in alignment with the policy objectives for the sector.

Recommendation 2

Future schemes should incorporate mechanisms to better compel businesses to explore markets outside of Wales to maximise the GVA impacts if that is the intention.

Recommendation 3

There should be a clearer understanding of the target audience for future interventions (e.g. large/small businesses), with the processes designed accordingly.

Recommendation 4

Future schemes should be better resourced to avoid time delays and disruption, either through establishing a larger delivery team or reducing the size of the programme.

Recommendation 5

Linked to the above, future schemes should either provide more support to help applicants through the process (e.g. a named individual with a direct line) or become less risk-averse internally to streamline the process so that applications can be processed more quickly.

Recommendation 6

A tiered system could also be considered to make the process more proportionate to the smaller grant applications.

Recommendation 7

There should be a single body responsible for overseeing applications through the entire process to provide a smoother transition.

Recommendation 8

Future schemes should incorporate some flexibility in the system to allow for inflationary changes in costs where processing time is lengthy.

Authors: Ioan Teifi, Endaf Griffiths

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Richard Self
Email: research.evaluation@gov.wales

Social research number: 99/2023
Digital ISBN: 978-1-83504-915-0

Image