The Third Sector European Team (3-SET) evaluation (summary)

The Third Sector European Team (3-SET) is a ‘function’ or team ‘owned’, run and populated by the Wales Council for Voluntary Action (WCVA). Staff are employed to carry out specific functions to provide assistance to the third sector across Wales in relation to the 2014 to 2020 European Structural and Investment Funds (ESIF) programmes. What is more, the team represents the third sector in the 2014 to 2020 ESIF programmes and provides signposting links to other European funds such as Erasmus+ and Horizon 2020. The service is funded through the ‘Technical Assistance’ stream included in each of the four European Social Fund (ESF) and European Regional Development Fund (ERDF) operational programmes. The total cost of the service across its eight year lifespan is projected to be £2,526,984 with £1,779,368 committed from EU Structural Funds.

In its current form, 3-SET 2014 to 2020 has three objectives.

1. To increase awareness amongst third sector organisations of the European Structural and Investment Funds (ESIF) and other European programmes available in Wales.
2. To increase third sector engagement in delivering the ESIF programmes in the 2014–2020 funding period.
3. To improve compliance rates and reduce the risks in relation to third sector project delivery.

These objectives are pursued through four strands of activity.

1. Information and advice.
2. Training and events.
3. Facilitating networks and networking.
4. Representing and promoting the third sector.

The Welsh Government commissioned Wavehill to undertake an evaluation of the Third Sector European Team (3-SET) in 2020. The overarching aim of this review was to assess the performance of the 3-SET, with a further objective being to help inform the post Brexit approach of the 3-SET and its activity for the remainder of the 2014 to 2020 programming period and beyond [1].

[1] The 3-SET continues to operate under the Welsh national eligibility rules and eligible actions as defined in the relevant specific objective contained in each of the 2014-2020 operational programmes. Please see EU Structural Funds programmes 2014 to 2020: operational programmes | GOV.WALES. EU Structural Funds 2014 to 2020: compliance and eligibility guidance.

The coronavirus pandemic has impacted on both the operation of the 3-SET and the evaluation. The 3-SET’s activities had to be delivered remotely, involving a switch to online training and networking meetings. Similarly, the evaluation data also had to be collected remotely. Due to the impact of the pandemic upon the capacity of staff within the organisations to engage in the evaluation, the response rates from organisations were slightly lower than anticipated.

The evaluation framework (co-developed with the management and key stakeholders of the 3-SET) identified the following as key research questions in order to explore the impact achieved.

Has the 3-SET:

• Successfully encouraged third sector organisations to apply for Structural Funds when they otherwise might not have?
• Provided clear and effective guidance that has supported third sector organisations in successfully applying for Structural Funds?
• Provided clear and effective guidance that has supported third sector organisations in successfully delivering Structural Funds projects?
• Facilitated networking and collaboration in the third sector?
• Successfully brokered relationships between the third sector, the Welsh European Funding Office (WEFO), regional partners and/or local authorities?
• Supported organisations in delivering the Cross Cutting Themes?

Data was collected through qualitative interviews with delivery and management personnel (n=7) and wider stakeholders (n=10); qualitative, semi-structured interviews with beneficiaries (n=52); an online survey of 59 other organisations benefitting or who had not engaged with the 3-SET; a Workshop with key stakeholders; a review of project monitoring and evaluation data and the relevant project delivery plans and policies.

The 3-SET is set to exceed its target of training events held in order to meet its dissemination and press release targets and achieve 90 per cent of its target of newsletters issued. Management personnel noted that whilst targets were reconfigured following Brexit so as to account for a reduced focus on training, there was also a knock-on impact on the utility and consequent need for as many newsletters. In light of this, a decision was made to provide fewer newsletters in agreement with the WEFO.

The 3-SET have also engaged directly with the formal policymaking structures in Wales and beyond, largely through responses to consultations, the presentation of evidence to Welsh and UK committee enquiries, and conducting and publishing research and evidence reports. In total, to date, the 3-SET have responded to 15 consultations or enquiries in Wales or the UK, contributed to the Secretary of State’s Brexit Advisory Group, made representations to the Comprehensive Spending Review 2020, and published an ‘Empowering Communities in the Context of Brexit’ report (WCVA, 2019). Furthermore, the 3-SET have attended and contributed to 16 formal networks ranging from transnational, Ireland–Wales networks to the Equality and Human Rights Coalition and to networks involved in the development of replacement programmes and future planning.

With regards to the impact of this policy engagement work, the evaluators comment that, in Wales at least, there is reason to believe that the 3-SET’s messages are being heard and considered. However, stakeholders were less confident with regards to the effectiveness of engagement with UK policy structures, suggesting less familiarity with the Welsh third sector in Westminster and Whitehall as potential challenges.

A significant majority of interviewees suggested that at least one area of their operation or organisation has been improved or developed because of the support from the 3-SET. Beneficiaries also spoke of softer but valuable impacts such as improved confidence in engaging with and meeting the requirements of European funding. The majority of interviewees also suggested that the changes were permanent or that they would remain in place until (at least) the end of the project’s lifespan.

The data strongly suggests that the 3-SET encourages organisations to apply for funds when appropriate. Very few interviewees noted that the 3-SET had not done so. However, there has been a deliberate attempt on behalf of the team to instil realistic expectations regarding the compliance requirements of European funding; in other words, whilst (ideally) organisations would apply, the 3-SET believe that some organisations are not always well placed and that the correct decision would be not to apply.

Beneficiaries suggested that the 3-SET encouraged their applications in a number of ways, most notably by offering and providing constant support throughout the application process, by raising awareness of the funding and their requirements, and by increasing the confidence of potential applicants. This was primarily done through the newsletters and events held by the 3-SET.

Beneficiaries were also of the view that the 3-SET had provided effective guidance with which to support them in successfully applying for funding. Indeed, nobody suggested that the 3-SET had not supported them to some extent, with the majority believing that they had done so to a great extent. Ongoing, sometimes informal advice and review of forms was the most common way in which this was done.

In regards to delivery, beneficiaries were overwhelmingly of the view that the 3-SET had provided effective guidance to support the delivery of their projects. Organisations have been supported most commonly through support to improve monitoring and compliance procedures. Furthermore, the team facilitated and supported direct engagement with the WEFO.

A significant majority of organisations suggested that the team had facilitated networking and collaboration within the third sector. The 3-SET and events were seen to be the main ways in which this was facilitated. However, many beneficiaries also noted the potential networking opportunities that training events offered.

Over half of the interviewees felt that the 3-SET had brokered relationships with WEFO, regional partners and/or local authorities to some or a greater extent. Though events were the main, formal way in which this was done, many reported having developed networks and relationships through the delivery of projects. This was particularly the case for organisations delivering on behalf of larger programmes. The data also suggests that ambassadors and the sharing of information and good practice also fostered trust and understanding that supported the brokering of relationships less directly.

Although relationships with local authorities were most commonly established, organisations, nonetheless, expressed a desire for more links to be developed with councils. Moreover, organisations expressed a desire to engage with partners outside of the third sector, as well as to engage in more collaborative work in the future. Some beneficiaries also identified future funding opportunities such as the Shared Prosperity Fund, and a desire to engage in collaboration with other organisations, or even to create consortia through which to deliver larger-scale projects.

Finally, the work being delivered by the third sector often makes a vertical contribution to the Cross Cutting Themes (CCT), i.e. the projects and work being delivered directly address the themes. This was felt, on the one hand, to be a particular strength of third sector delivery, but also an area that the sector has historically found to be challenging to evidence. The data from beneficiaries suggests, however, that the 3-SET have worked to address these shortcomings. The interview data strongly suggests that the 3-SET have supported organisations in a range of ways in relation to the CCTs. Much of the support amounts to a process of demystifying the CCTs and demonstrating or illustrating where and how the organisations could effectively and efficiently evidence their contributions.

The management team were widely credited with the effective and impactful delivery of support to the sector. What is more, they are held in high regard by organisations and stakeholders for their practical and technical expertise, professionalism, and experience. The good and personal relationships developed over the years emerge as a key factor that engenders trust in the team.

The sector faces a range of challenges over the coming years. COVID-19 is thought to have hampered or delayed organisations’ efforts to engage with the challenges posed by the end of European funding.  They were, nonetheless, aware that the challenge posed by the end of EU funding was a significant one, albeit something that has been understood to be a longer-term challenge, to address after the pandemic. Stakeholders, meanwhile, suggested that the challenges of sustaining funding and delivery were more immediate, whilst organisations were also likely to see an increase in demand for services, coupled with diminishing revenue from fundraising activities.

There appears to be a risk, therefore, that third sector organisations, due largely to the challenges of the COVID-19 pandemic, do not have the capacity to engage in legacy and future planning effectively, and may require additional support over the short to medium term. The 3-SET may make a valuable contribution over the final few years of delivery in this respect.

The data suggests therefore, that there will be a need and demand for a similar support platform providing technical assistance akin to the 3-SET after the end of European funding. The exact form and structure of this platform should be designed with the support needs of the sector and the experience of the 3-SET in mind. It is likely that a platform will be most needed to support engagement with large, structural investment programmes that may replace European funding in the future. Its most valuable contribution will likely lie in a) supporting the third sector in accessing the funding, and b) continuing with the impactful technical assistance that has enabled the third sector to successfully apply, deliver and add value to strategic investment programmes. However, enabling third sector access to such funds will be a priority. A precursory challenge, therefore, will be to make the case for the Welsh third sector as deliverers of strategic investment, as well as to impress upon policymakers the need to design any funding structures with the Welsh third sector and its capacity and needs in mind.

In summary the report recommends:

• exploring the feasibility and possible methods of promoting the service more effectively
• investigating opportunities to bring organisations into contact with local authorities more frequently and potential partners together, including those from outside the sector, to collaborate (for example, on bids for funding)
• investigating the ways in which the 3-SET’s work in relation to replacement funding can be of benefit to organisations beyond those in receipt of ESIF funding this round
• assessing the progress made within the sector in preparing for the end of European funding, and the additional support needs that organisations may now have
• viewing the promotion of the Welsh third sector, its needs and requirements, its capacity to deliver projects related to strategic investment programmes, and the added value and reach it offers, as a priority for its remaining years; emerging replacement funding structures should be considered a priority 'target' for this promotion.
• depending on the form that structures such as the Shared Prosperity Fund take, the 3-SET should consider its role in preparing the ground for such a support platform
• Welsh Government consider the 3-SET team and its expertise/experience, as a means of adding value to any new platform offering technical assistance and promoting the Welsh third sector, given the widespread view that such a platform will be required in the future

The evidence and data examined for this report suggest that the 3-SET have been, and are likely to continue to prove to be, an effective, impactful, valued and needed service for the third sector in Wales. What is more, there is a case to be made that its contribution in the final few years may prove to be most valuable in relation to the future, in preparing organisations to access replacement funding structures and to lobby for the sector and its access to replacement investment structures on a strategic and policy level.

Authors: Dyfan Powel, Anna Burgess and Sam Grunhut (Wavehill)

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information, please contact:
Charlotte Guinee
Email: rme.mailbox@gov.wales

Social research number: 51/2021
Digital ISBN: 978-1-80195-659-8

Image

Image

Image