Evaluation of the knowledge transfer, innovation and advisory services programme: final report (summary)

Farming Connect aims to improve the profitability, competitiveness, resilience and sustainability of farm, forestry and food businesses across Wales, supporting the sector through a period of significant change as it moves away from direct payments via the Common Agricultural Policy.

The 2014-20 programme period was funded through the Rural Communities–Rural Development Programme (RDP), a seven-year European Agricultural Fund for Rural Development (EAFRD) programme funded by the European Union and Welsh Government. Farming Connect comprises a wide range of support, including knowledge transfer activities, continuous professional development and training, and one-to-one and group advice. The programme was delivered by Menter a Busnes (MaB) and Lantra and overseen by the Welsh Government.

SQW, with Arad and our agricultural expert Martin Collison, was commissioned to undertake an evaluation of the Knowledge Transfer, Innovation and Advisory Services Programme (2014-2020) known as Farming Connect. The evaluation sought to assess the effectiveness of implementation, gather evidence on outcomes achieved, and learn what works (and why) to inform ongoing delivery and the design of future programmes.

The evaluation was undertaken in two phases: first, in 2018 through to early 2019; and second, in late 2019 to mid-2020. This report draws together findings from both phases.  Throughout, the focus of the evaluation was predominantly on the 2014-20 programme period, which formally ended (in terms of delivery) in August 2019.  However, the fieldwork for Phase 2 took place shortly after the launch of the Farming Connect refresh (an extension to August 2022); whilst some consultees were able to reflect on the potential benefits arising from the refresh, it was too early to comment on this, or to evidence effects in practice.

In line with the Specification for the study from the Welsh Government, the evaluation adopted a theory-based and largely qualitative approach[1]. This included a series of in-depth longitudinal case studies covering 13 strands of Farming Connect activity (involving consultations with beneficiaries and delivery staff) to evidence outcomes and test the theory of change, regional focus groups with beneficiaries to check and refine emerging findings, and consultations with governance, management and delivery staff and with stakeholders.  In addition, SQW undertook a desk-review of programme documentation and monitoring, and of social media data.  Evaluation findings were presented to the Welsh Government and Farming Connect Strategic Advisory Board.

[1] Quantitative data gathering (for example, via a telephone survey of beneficiaries) and counterfactual impact evaluation techniques were not within the scope of this assignment, as set out in the Welsh Government’s Specification. Alongside this evaluation, the Welsh Government had planned to include Farming Connect beneficiaries in the wider sample for the Farm Practices Survey to allow for comparisons to be made between beneficiaries and non-beneficiaries and inform the qualitative research.  However, this survey work has not yet been undertaken by the Welsh Government.

What activities have been delivered to date, compared to expectations?

Total programme expenditure over the 2014-20 contract period (to August 2019) was £25.72m, very close to budgeted spend of £25.73m. The programme performed well against target outputs for that contract, and offered a broad range of support, reflecting the differing needs, stages of development and preferred learning styles found across the farming sector. The programme is 'well known' and 'trusted' across Wales, and the longevity, stability and continuity of Farming Connect has been important. 

How intensively do farmers engage with the programme and progress through the offer, and what drives this?

Significant effort went into registering new farmers into Farming Connect during the 2014-20 programme period to widen the programme’s reach. However, there is now debate around whether the programme should continue to strive for wider reach, or now focus efforts on farmers who want to change; in Phase 2 of the evaluation, the weight of argument amongst consultees was towards the latter. There is also concern that for many, engagement with the Farming Connect offer can be relatively narrow and/or light touch: one quarter of those registered had not engaged[2] with any Farming Connect support, and around half of those who had engaged received support from only one of Farming Connect’s three strands.

Navigating the Farming Connect offer has been a challenge[3]. Development Officers play a critical role to encourage take-up and facilitate some farmers’ journeys through Farming Connect, but beneficiaries expressed frustration with the lack of a single point of contact within the programme. The broadness of Farming Connect is a strength, but more could be done to package, integrate and communicate the offer more effectively. Both farmers and external stakeholders suggested there was scope to provide more 'personalised' and 'holistic' support, with better integration of Farming Connect activities, informed by an effective baseline, alongside a more consistent approach to facilitation and 'nudging' through the customer journey. This was necessary, in order to deliver 'real impact' and an essential 'step change' in the sector[4]. 

[2] i.e. engaged with one or more of Farming Connect’s activities, rather than solely in receipt of Farming Connect materials (all registered farmers receive general information emails etc)

[3] The Farming Connect refresh introduced measures designed to improve the ability of farmers to navigate the support, such as setting out clearer route(s) to follow through the offer and redesigning the website.

[4] More recent efforts to create 'progressive packages' of Farming Connect support, with campaigns running alongside inter-linked elements of support, are a welcome step in the right direction.

How effectively and efficiently is the programme being delivered, managed and governed?

A number of aspects of Farming Connect were highlighted as good practice across both phases of the work. Crucially, the offer focuses on what needs to change, and why and how this can be achieved in a practical and cost-effective way. Features that have worked well include self-help and action learning approaches, encouraging beneficiaries to self-define goals 'bottom up' to ensure a close fit with their needs and buy-in to the process, practical, farm-based learning and peer-to-peer support, and flexibility to adapt the focus of an activity in response to changing conditions and to suit the working patterns of those involved. Facilitation is important to provide structure, challenge and momentum, alongside input from high quality and 'trusted' advisors/speakers to provide inspiration and expertise. Benchmarking has been demonstrated as a key driver of change in behaviour, particularly where this is undertaken in discussion with peers or facilitators.

There have been some delivery challenges, including reported variability in the quality and consistency of facilitators and advisers, the capacity of Development Officers, the use of training application windows, and managing the use of 'time limited' support. Consultees also suggested there may be missed opportunities for farmers who want to push ahead; clearer pathways could be put in place for the most progressive farmers. There was also concern that farmer engagement with Farming Connect lacks clear focus and purpose for some participants, and assessment of farmer needs at the outset has been ad hoc and inconsistent[5].

Programme management was found to have worked well throughout the evaluation, with a highly experienced and knowledgeable team in place. A strong emphasis on feedback and continuous improvement maximised the effectiveness of Farming Connect in real time, and the programme was actively managed in response to changing needs and conditions (as illustrated in the response to COVID-19). A substantial and positively received shift in governance arrangements over the last year included more private sector representation on the Strategic Advisory Board, in order to provide more strategic industry-led challenge. Partnership working at an operational level has improved considerably over recent years, but there was scope to strengthen partnership working at a strategic level in terms of long-term prioritisation and planning. 

[5] The programme has sought to address these issues in the refresh, by encouraging all beneficiaries to complete baselines (including business plans, benchmarking, animal health and nutrient management plans). This was perceived as a positive step forward by consultees.     

To what extent are changes implemented on farms, leading to intended outcomes and impacts?

The evaluation evidence demonstrated how Farming Connect had played a crucial role in creating the 'foundations for change', with a substantial impact on personal outcomes such as changes in mindsets, attitudes, confidence and ambition, alongside improved skills and knowledge (both in terms of business and technical skills). These benefits led to changed management practices, and crucially to more informed and confident decision-making processes within businesses. Farming Connect also has a (recognised but often under-appreciated) impact on the mental health of farmers and in strengthening networks in the farming community. 

For many farmers, the support is leading to small scale, incremental changes over a long period of time. This reflects the fact that many very small businesses are capacity and resource constrained, so changes need to be affordable and manageable. It may also reflect the way in which engagement with Farming Connect (and navigation of the offer) and the definition of goals within many of the activities are farmer-led. We found that these marginal gains across many aspects of the business are helping to create more viable and sustainable enterprises in the longer term. There are also widespread benefits in terms of environmental impacts and animal health, notably in terms of reduced antibiotic and fertiliser use, sustainable resource management and biodiversity. In addition, for some of those involved, Farming Connect has had a more transformative impact on the business, for example, through significantly reducing costs, improving productivity/yields or diversification. 

There was a consistent message from delivery staff, external stakeholders and beneficiaries that it is the combination of interventions, alongside the support of a Development Officer or mentor that really makes the difference in realising impact. Beneficiaries have taken a multitude of routes through the offering, making it difficult to observe patterns or comment on which routes are most effective. However, the evidence suggests that a combination of both peer-to-peer group and one-to-one support is critical for many, as well as the more intensive aspects of Farming Connect (such as Agrisgôp and Agri Academy) that can make a real difference. This re-emphasises the importance of having a broad offer and being able to navigate it.

Development finance was highlighted as one of the most important barriers to implementation; and whilst this was not included in the original Specification for Farming Connect, it is an important factor hindering the delivery of the programme’s goals. Other barriers to implementing change were time, capacity and firefighting issues within very small businesses, succession issues, and external factors such as markets, broadband provision and weather conditions. Some external stakeholders also argued that, whilst Farming Connect had led to behavioural and attitudinal changes, it is 'trying to be all things to all people' and spreading the support too wide and thin, there is a concern that this dilutes the programme’s impact. 

Overall, Farming Connect appears to be delivering outcomes that would not have been achieved at all, or which would have taken longer, been lower quality or less sustainable, in the absence of the programme. Moreover, the contribution of Farming Connect compared to other internal or external factors is substantial. Few beneficiaries consulted had made other changes in their business alongside Farming Connect support, and where they did so, this had made marginal or no contribution to the outcomes observed. The exception was grants, which were important in enabling farmers to implement change.

What are the key lessons to inform ongoing delivery and design of future interventions?

There was consensus across the various stakeholders and beneficiaries that support to help the farming sector adapt and remain competitive will be even more critical in the immediate future and beyond, and that the current programme provides a strong, and widely appreciated, platform for this. The findings of this evaluation do, however, raise some questions that we believe the Welsh Government and partners should consider in the design of future programmes.

First, it would be helpful to clarify and focus Farming Connect’s strategic priorities[6], particularly in the current context of Brexit and the speed at which the farming sector needs to adapt. This includes clarifying Farming Connect’s role within Wales’ agricultural innovation ecosystem, and how it is integrated with other public/private sector actors and interventions.  

Second, the programme should consider more formal and consistent partnership working and networking with private sector intermediaries[7] to facilitate more effective ‘supply-side’ networking, reinforce Farming Connect’s key aims, and encourage greater knowledge transfer across the wider farming community (notably those who engage with vets, banks, accountants etc, but not with Farming Connect directly).  This should build on collaborative working with large animal vet practices which deliver Animal Health Clinics for Farming Connect, to ensure that engagement with intermediaries more broadly is planned and implemented strategically.

Third, assess the feasibility of shifting the Farming Connect model towards a more personalised approach, with some degree of facilitated access for farmers who are willing to change. This is not to say Farming Connect should cease to be universally available or focus only on the more “progressive” farmers, but the programme should consider more economical mechanisms where possible (informed by the apparently positive experience during COVID-19 of online delivery) for the “broad and shallow” aspects of support.

Fourth, there is scope - and evident demand from some farmers - to introduce more inspirational ideas and external challenge into the programme to really drive new thinking. In this context, the programme should consider the balance between supporting the “traditional” farming sector to become more productive/resilient and enabling opportunities for new and innovative food production systems[8].

Fifth, as Wales transitions out of the EU, a greater emphasis will be placed on “public good” outcomes, particularly in relation to the environment. The programme should consider strengthening and promoting its offer in this respect (in conjunction with an industry wide push).

Sixth, strengthening the underpinning evidence base for Farming Connect, including: targeted research to characterise non-participants and understand why they are not engaging with Farming Connect; strengthened data gathering within programme, particularly in terms of consistent and comprehensive use of baselines and monitoring progress against this; and undertaking programme-level quantitative impact analysis, via surveys and/or allowing for data-linking into national datasets to compare the performance of beneficiaries and non-participants over time.

[6] To note, since the research was undertaken for this evaluation and the programme refresh, a new SAB and Chairman has been appointed and a long-term strategy for Farming Connect (taking into account Brexit and COVID-19) is under development. 

[7] In addition to breakfast meetings held with intermediaries and flyers sent to feed merchants

[8] To note, since the research was undertaken there is evidence of the programme promoting innovation, for example, through the Diversification and Innovation Show in October 2019 which attracted 1,500 attendees. Whilst planning of the event took place during the evaluation period, the event itself was held after the evaluation period.

Authors: Hindle, R., Pates, R., and Barber, J (SQW)

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Tom Carwright
Email: RDPMandE@gov.wales

Social research number: 41/2021
Digital ISBN: 978-1-80195-483-9

Image