European Social Fund (ESF): evaluation of supporting the NHS and social care through coronavirus (COVID-19) (summary)

In early 2020 the European Commission (EC) notified Managing Authorities that Structural Funds money could be used to respond to the COVID-19 Pandemic. The Welsh European Funding Office (WEFO) reviewed the available options set out by the EC and proposed a new Priority Axis - ‘Containing Covid through Capacity’ - which was proposed to and cleared by the Wales European Structural and Investment Funds Programme Monitoring Committee (PMC) ^{[footnote 1]}. The primary function of the PMC is to monitor the implementation of the 2014-2020 ESI Programmes operating in Wales in accordance with European Union (EU) regulatory requirements ^{[footnote 2]}.

The primary priority of the operation was to purchase PPE with a secondary element subject to funding availability of the recruitment of temporary staff.

In July 2023, Opinion Research Services (ORS), was appointed by WEFO to undertake an evaluation of its ‘Supporting the NHS and Social Care through Covid’ operations. The aim was to produce a methodologically robust and independent evaluation, assessing the delivery of, and the outcomes achieved by, the operation.

This summary report provides an overview of the evaluation and its findings.

The evaluation fieldwork was conducted between August and November 2023 and comprised of scoping interviews, a data and documentation review, and virtual in-depth interviews with stakeholders conducted via Microsoft Teams. A theory-based approach was applied to the evaluation design, to review the operation’s theory of change model designed by WEFO.

Overall, three scoping interviews were completed, and a further seven stakeholders took part in the in-depth interviews. It is important to note that these were recruited from a small initial list of 20 stakeholders. The operation was retrospective in nature, meaning that stakeholders with a working knowledge of it were few and far between.

There were also sensitivities around the ongoing UK COVID-19 inquiry, which seems to have led to a reticence to take part among stakeholders. Some stakeholders invited to take part sought legal advice on whether they should participate due to concerns related to the Covid Inquiry, and its high public profile at the time the evaluation was taking place.

Rationale and design of the operation

Due to the retrospective nature of the funding award, stakeholders participating in the in-depth interviews considered the overarching rationale of the operation to be simple: “to supply money to the NHS for PPE equipment and increased staffing levels during the pandemic” (WEFO stakeholder). Targets were therefore largely numerical, focussed on the number of individuals recruited and the volume of PPE procured.

While stakeholders are now aware that ESF funding has been retrospectively awarded to help cover the extra cost of PPE and staff during the pandemic, knowledge among health board staff and higher education providers on the availability of funding was limited in the earlier days of the pandemic. However, there was some understanding that support would likely be available, probably from Welsh Government.  

The retrospective nature of the operation meant that additional data collection burdens at the most challenging time of the pandemic could be avoided. 

Internal stakeholder participants felt that the approach to planning the operation was a pragmatic one, which supported the whole government approach in a positive way. This was enabled by the EC’s quick issuing of guidance and clarity in explaining what was within the parameters of the funding, especially considering that ESF funding had not been used to support the health sector in Wales previously. 

Outputs

The operation was successful in delivering its key outputs: more PPE was purchased and staff capacity within the health sector was increased. The numeric targets also largely reflected the level of purchasing or recruitment that was eventually funded, which strategic stakeholders said is not always seen in ESF funding, where underspends are common.

Outputs: PPE procurement

At the start of the pandemic, it was quickly recognised that the PPE supplies and stock levels held in Wales were inadequate for the response needed against COVID-19. 

The Welsh Government provided funding to the NWSSP to purchase PPE. The Welsh Government also worked with other nations of the UK, including on the provision of mutual aid for PPE. The emergency work carried out in response to the pandemic allowed for robust and secure supplies of PPE across Wales and data shows that stocks did not run out.

In terms of the procurement process in Wales, while it was accelerated, good governance measures were adhered to and this is evidenced by the Audit Wales findings in its report ^{[footnote 3]} on procurement process for the PPE in Wales, which commends the approach taken.

According to the Audit Wales report 630 million items of PPE were issued by NHS Wales Shared Services Partnership (NWSSP) between 9 March 2020 and 7 February 2021, and that the Welsh Government expected to spend over £300 million on PPE for Wales during 2020-21. This compares to a typical pre-pandemic spend on PPE in Wales of around £8 million ^{[footnote 4]}

Figures from NWSSP show that the ESF budget purchased 352,486,650 items of PPE, which represents more than half of the total purchased across Wales, and accounted for a spend of £110,242,777 million, or over a third of the total spend. 

Outputs: staff recruitment

Priority-level targets were set based on projected cost estimates for PPE and staff recruitment, with 60% of the Specific Objective’s financial allocation expected to be allocated to staff recruitment. Priority-level targets for additional temporary staff to be recruited were set at 2,400 in the West Wales Valleys (WWV) region and 1,800 in East Wales (EW). 

The changes introduced by UK healthcare professional regulators on the emergency registration of health professionals included changes to education standards to enable the deployment of healthcare students and retired staff. 

This would enable student nurses and midwives in their final year to complete the last six months of their programme in appropriate placements settings and former staff who had left or retired in the previous three years were asked to re-register with their professional bodies.

In total, £64,753,095 was spent through ESF funding on NHS staffing. This amount covered the gross salaries of the staff hired but did not cover a wider set of costs such as advertising the positions, the hiring process, and any other administrative costs.

Stakeholders mentioned several challenges in recruiting staff, including managing, and responding to the significant interest from members of the public, and managing the differing recruitment streams in place. Ultimately, the complex recruitment process was considered unavoidable, but was managed as best as possible under the circumstances, according to a health board stakeholder. 

Cross-cutting themes

European funded operations include a set of indicators known as Cross Cutting Themes (CCTs), which are intended to promote activity on: Equal Opportunities; Gender Mainstreaming and the Welsh language; Sustainable Development; and Tackling Poverty and Social Exclusion. As this was an emergency response to the pandemic, and funds were awarded retrospectively, the decision was made not to include ‘formal’ indicators for CCTs in the business plan.

However, in considering CCTs, it was felt there had been a wider response in Wales to the impact of the pandemic on Black, Asian and ethnic minority communities and people working in the health and social care sector, with a Welsh Government task force established, leading to a report on the impact of Covid on ethnic minorities ^{[footnote 5]} and a staff assessment tool being developed. At a later stage, the Race Disparity Evidence Unit was established. It is, however, important to note that these developments were in motion before the funding was paid retrospectively.

Although PPE products are single use, and therefore procurement was not able to contribute to the Welsh Government targets for waste and recycling in its ‘Towards Zero Waste Policy’, NWSSP have subsequently gone to market to establish a Framework Contract for any future requirements which may come through, which should ensure that issues such as Modern Slavery and Carbon Footprint along with opportunities for local suppliers are addressed.  In doing so they have helped to make the supply chain more secure and reliable.

Short- and long-term outcomes

Fully evaluating the outcomes and long-term socio-economic impact of this operation would be very complex and beyond the scope of this evaluation. Ultimately, this project supported the NHS and Social Care through providing additional capacity for PPE and staff, which helped contain the spread of the virus, ensure the delivery of safe and sufficient services, and save lives. In so doing, it helped reduce the long-term negative health, economic and social impacts of the pandemic 

All operational stakeholders were in agreement that although money was spent on staff and PPE without as much of the usual consideration of affordability at the beginning of the pandemic, receiving ESF funds retrospectively as well as other sources of funding has helped cover these unexpected costs. They felt they would have to make difficult decisions about what compromises would have to be taken now, to cover the costs of the pandemic.

The use of ESF funds for health purposes was not something that had been envisaged before the pandemic and the ‘newly formed’ relationship this forged between Welsh Government/WEFO and health boards is something that is hoped to open doors in future.

Lessons learned, good practice and long-term sustainability

For the WEFO team, key lessons learnt included confirmation that there can be flexibility in longer-term funding programmes; and the process that led to this operation is seen as a positive example of joined up working across organisations.

For many stakeholders, there have been key lessons learnt around the importance of effective collaboration and co-production as well as utilising and leaning on well established relationships.

Although the retrospective nature of the programme was thought to have worked well for most stakeholders interviewed, a Welsh Government stakeholder suggested that in another pandemic situation, it would be better to have funds available to be distributed from the outset, with a strong governance process built around it. It was felt that this would demonstrate that lessons have been learnt from the COVID-19 pandemic.

As a result of the work undertaken associated with these operations, there has been best practice identified and implemented. The Welsh Workforce Risk Assessment Tool has been developed and rolled out across all Health Boards in Wales. Other good practice identified came around the speed of procurement of PPE.

The ongoing legacy of the operations will come from the lessons learnt and incorporated into the wider NHS service. This links to the Welsh Workforce Risk Assessment Tool and the guides produced to help staff with PPE fit. However, it should be noted that these are not direct results of this specific operation, more results of the overall response which this operation formed part of.

Although difficult to fully evaluate the outcomes of the operation in terms of lives saved and long-term health, economic and social impacts, the evaluation findings show that this was a carefully planned and executed operation, with a well-thought-out rationale.

There is also evidence of positive working relationships between WEFO, Welsh Government, NWSSP, and other strategic partners being utilised to provide impetus to the operation and ensure programme targets were met.

The most unique aspect of the operation was the emergency nature of the response, which necessitated speedy operational decision making. This was aided by retrospectivity, another important factor in considering the operation’s success.

Although  specific objective targets and outcomes were purposefully narrow, stakeholders felt they had been met: enough PPE was available for the health sector; and staff capacity had been sufficiently boosted, with the data supporting this.

A number of main lessons have been observed by stakeholders, which relate to Wales’s response to the pandemic more widely. As an overview:

• There is a renewed appreciation of how flexibility within long-term funding programmes can be used to respond to a crisis.
• Effective cross-organisational working has been a key enabler for driving this kind of operation forward as part of an emergency response.
• Developing relationships is also key to successfully navigating crises at an operational level.
• There are areas to explore and progress in terms of manufacturing PPE within Wales.
• The COVID-19 response has highlighted the need to further support staff from an ethnic minority background and with other protected characteristics in the healthcare workforce ^{[footnote 6]}.
• The COVID-19 pandemic has also highlighted how pandemics can have differential impacts on different demographic groups within the population. Therefore future health responses need to consider the risk of a differential impact when designing that response.
• Improvements are needed in the availability and quality of data in the NHS and in social care around protected characteristics including ethnicity, disability, and long-term health. This would lead to better identification of socio-economic factors impacting health inequality and help address them more effectively ^{[footnote 7]}.
• When financial limits are relaxed, change can happen at pace, and it is likely that the changes made will lead to financial efficiency at a later date.
• Capitalising on technological advances to improve health systems and care models is important, including a shift toward online recruitment practices.

[1] The Wales PMC was established by Welsh Government at the outset of the 2014-2020 European Structural and Investment Fund Programmes (ESI Programmes). At inception, the PMC comprised 27 Members derived from the public, private, higher education, and third sectors; civil society; and environmental and equality bodies. It is chaired by a Member of the Senedd.

[2] Regulation EU 1303/2013 of the European Parliament and of the Council (‘Common Provisions Regulation’ or ‘CPR’), in particular articles 47,48, 49 and 110; regulation EU 1304/2013 of the European Parliament and the Council (‘ESF Regulation’), in particular article 13(3) and 19; regulation EU 1305/2013 of the European Parliament and the Council (‘EAFRD Regulation’), in particular article 74. Of particular importance to this evaluation is the regulatory requirement for the PMC to examine the evaluation prior to its submission to the European Commission.

[3] Procuring and Supplying PPE for the COVID-19 Pandemic (Audit Wales)

[4] No specific date is given as a comparator.

[5] COVID-19 Black, Asian and Minority Ethnic Advisory Group: potential impact of COVID-19

[6] This issue is being considered and addressed through the introduction of the workforce race equality standard - Workforce Race Equality Standard (NHS Wales)

[7] This issue is being considered and addressed through the introduction of the workforce race equality standard - Workforce Race Equality Standard (NHS Wales)

Report authors: Lock, K., Moore, N., Davies, A. (Opinion Research Services)

Views expressed in this report are those of the researchers and not necessarily those of the Welsh Government.

For further information please contact:
Emily Rowlands
Email: research.evaluation@gov.wales

Social research number: 40/2024
Digital ISBN 978-1-83625-190-3