Supporting People data linking project: year three progress report

The Welsh Government Supporting People Programme (SPP) provides housing-related support. The programme helps vulnerable people avoid homelessness and live as independently as possible.

The SPP exists to help people find and keep a home. That home should meets their needs and encourage independence in a healthy and safe environment.

In 2017-18, the SPP budget was £124.4 million. The annual budget and the total grant available changes with budgetary decisions and constraints. The programme supports more than 57,000 people each year and aims to prevent problems by providing help as early as possible.

The programme provides two main types of support.

• Long-term maintenance support helps people keep or gain independence. This avoids the need for more costly interventions such as entering care
• Short-term preventative services help people avoid homelessness, often during a time of crisis.

There is one further distinction around the nature of support – that between fixed and floating support. The programmes ties fixed or accommodation-based support to particular accommodation. Fixed support can include homeless hostels, refuges or other supported housing schemes. Floating support’s flexibility makes it possible to provide it in many places, including supporting a person in their own home. The programme is largely preventative in nature and this is in keeping with the aims of The Housing (Wales) Act 2014.

The Supporting People Research and Evaluation Group (SPR&EG) was set up to help develop research to measure the impact, effectiveness and value for money of the SPP. 

The group comprises:  

• two members of the Supporting People National Advisory Board
• representatives from local authorities
• service providers
• the SAIL (Secure Anonymised Information Linkage) Databank/Administrative Data Research Centre Wales (ADR Wales)
• Welsh Government.

The Supporting People Data Linking Project (SPDLP) followed on from the Welsh Government Supporting People Data Linking Feasibility study (SPDLFS), which reported in March 2016. The SPDLFS showed that a more robust and large-scale study of the impact of the SPP in Wales using data linked in the SAIL Databank was deliverable. The Welsh Government Cabinet Secretary for Communities and Children agreed funding for a four-year project. Reporting of the project will take place at the end of each financial year up to March 2020.

The project also formed part of the Welsh Government Programme to Maximise the Use of Existing Data (Data Max). The Data Max programme ran from 2011 to 2018 and was part-funded by the UK Economic and Social Research Council (ESRC). From 2018, the programme was incorporated into the ESRC-Funded Administrative Data Research Wales, part of Administrative Data Research UK. The research will inform Welsh Government policymaking around six priority themes. Working as part of ADR UK, ADR Wales uses cutting-edge techniques to produce world-leading research. For its research, ADR Wales uses anonymised data from across government departments.

Welsh Government developed the SPP to prevent homelessness and help people stay independent. This would allow people to continue to live in their own home rather than entering long-term care. As part of its primary purpose the Programme would also expect to reduce demand on the NHS and other services.

The SPDLFS looked at the impact of the SPP on the use of health services. This involved looking at two pathfinder local authorities (see SPDLFS report). The SPDLP aims to build on this by linking a Wales-wide sample of SPP data with existing routine health and other administrative datasets in the SAIL Databank. This will allow the project to assess the impact of the SPP on indicators relating to the wider well-being of service users.

The broad research question which this project aims to address is “Does the receipt of Supporting People services have an impact on patterns of public service use?”.

The specific indicators of well-being on which this project aims to focus over its lifespan are interactions with:

• health services (primary and secondary care)
• social care services
• housing options services
• pre-16 education.

The project aims to look at interactions with the criminal justice system and labour market participation. This work will only be done if resources allow.
 

As a result of the work completed during Year Two, the project proposed objectives for Year Three. The objectives below are those proposed for Year Three in the Year Two Progress Report. Some objectives were carried over from Year Two. The objectives changed during Year Three to reflect progress with this and other Welsh Government data linking projects. 

The objectives are divided into four sub-sections:

• Supporting People data

      Objective 1: sourcing post-2016 data
      Objective 2: assess the current state of the analytical work
      Objective 3: comparing the methods of the SPDLFS and the main stage
                          SPDLP
      Objective 4: reviewing the Supporting People data quality
      Objective 5: agreeing research questions
      Objective 6: continuing to expand on the previous analytical work

• Cost-offset model

      Objective 7: continuing to explore the option of creating a cost-offset model

• Non-Supporting People data

      Objective 8: sourcing datasets for further indicators

• Analysis and reporting

     Objective 9: reporting emerging findings of the analysis of the Supporting
                         People data.

Until February 2019, a Swansea University analyst was delivering the SPDLP. The analyst reported jointly to the Welsh Government Housing Researcher and the Data Max Programme lead.

As noted above, the ESRC has funded ADR Wales to produce research using administrative data. This gave the opportunity to deliver the analysis more inexpensively using the ADR Wales Team in Welsh Government. ADR Wales initially agreed to recruit a secondee to deliver the work. Whilst the recruitment was ongoing, the ADR Wales Team would continue the analysis, but due to resourcing it would be at a reduced pace. The recruitment for the vacancy is currently ongoing.

The handover of analysis has been completed as of July, and work began in August to meet objective two, as listed below. The project team previously hoped to publish further findings in 2019. Delays in the analysis timetable mean this is now unlikely to happen before late 2019.

Objective 1: source post-2016 data

Three local authorities were unable to provide data to the Project in Year Two. For these local authorities, the project will try to get SAIL data disclosure agreements in place and add post-2016 data to SAIL.

For all 22 Local Authorities, the project will seek to add further Post-2016 data to SAIL.

Progress in year 3

The project has completed data sharing agreements with all 22 Local Authorities in Wales. A total of 21 of the 22 Local Authorities have supplied data.

The ADR Wales Team aims to continue this work by bringing further years’ data into the SAIL Databank for all 22 Local Authorities. The Team will build on the good relationships the Swansea University analyst developed with the SPP teams. Where possible, the Team will streamline the data sourcing process to improve efficacy, reliability and efficiency. This will include working with projects like the Dataflow Development Project (DDP).

The DDP intends to create the capability within Local Authorities to automate the split-file process. The split-file process is a process for depositing anonymised individual-level data into the SAIL Databank for research and statistical purposes.

Objective 2: assess the current state of the analytical work

The ADR Wales analyst will assess the completed analytical work to find areas to add value.

Progress in year 3

The Swansea University analyst progressed the analytical objectives from Year Two. This included data cleaning, production of code and analysis. The ADR Wales Team will familiarise themselves with the completed work. The Team will assess the completeness and quality of the data and the methods used to identify any potential to add value.

After understanding the data and the methods used, the Team will build on completed work to deliver further analysis. With access to both the previous code and datasets, a review of prior work is ongoing.

Objective 3: compare analysis methods between the SPDLFS and the main stage SPDLP

This will identify any explanations for observed differences. This is a new objective.

Patterns of health service use differed between the SPDLFS and the main stage SPDLP ‘Emerging findings’ report. The project aims to investigate the reasons for these differences. To do this, the project will secure access to the SPDLFS analysis and compare methods.

Progress in year 3

The SPDLFS analysis suggested a quantitative evaluation based on linked routine administrative data would be likely to produce statistically robust substantive findings. By comparing the characteristics of SPP service users in the local authorities for which data can be linked with service users in the remaining local authorities, a strong demonstration could be provided of the generalisability of the findings for the whole of Wales.

To compare the methods used in the two projects, the ADR Wales Team is obtaining access to the work completed and data used in the SPDLFS. The Team will use the method from the SPDLFS to analyse data for the main stage SPDLP to compare the results and seek to identify the reasons for any differences.

Access to the SPDLFS has been sought from SAIL, along with an IGRP extension.

The Information Governance Review Panel (IGRP) provides independent guidance and advice on Information Governance policies, procedures and processes for SAIL Databank. The Panel reviews all proposals to use SAIL Databank to ensure that they are appropriate and in the public interest, and it comprises representatives from various organisations and sectors.

Objective 4: review Supporting People data quality

Continue to review the quality of SPP data provided to the project. This includes linking rates and the number of missing data items. The project will give feedback to local authorities to improve up-stream data collection. The project will investigate further cases where no ‘lead need’ was recorded.

Progress in year 3

There has been improvement in the comprehensiveness of the data provided for the project over the course of years two and three. However, some issues remain, including the completeness of datasets.

Objective 5: agree research questions

Agree research questions with the SPR&EG, prepare an analysis plan and carry out agreed analysis.

Progress in year 3

To ensure further analytical work provided the best value, the SPR&EG decided to focus the analysis on a small number of analytical tasks. The Swansea University analyst had developed a list of potential analytical tasks in consultation with the Welsh Government Housing Researcher and Data Max lead. This list was presented to the SPR&EG. There was agreement that the Swansea University analyst would deliver analysis of a single ‘lead need’ group of recipients. This would show whether the proposed methods provided useful, actionable findings. The project chose the Domestic Abuse group as the demonstration ‘lead need’.

The project asked the SPR&EG to recommend which further ‘lead need’ groups of recipients would be the greatest priority. The group decided the focus of further analysis would be Mental Health and Substance Misuse. The SPR&EG prioritised these subgroups mostly because of their likely priority to policy and the high number of relevant health events available for analysis. ‘Lead needs’ were also prioritised if there was a greater likelihood of creating a robust control group. Please see below for discussion of a potential control group for Substance misuse.

Objective 6: continue and expand on the previous analytical work

This includes seeking to identify suitable control groups  for analysis purposes. A robust control group would provide the best demonstration of the efficacy of the SPP. It would be the most credible evidence it is making a difference in its user's lives.

A control group is a group of individuals who do not receive an intervention. It is selected to resemble the individuals who do receive the intervention. The project would compare the intervention group to the control group to allow the project to attribute any observed impact to the SPP.

Progress in year 3

The ADR Wales Team will build on the analysis of the Domestic Abuse ‘lead need’ to analyse Mental Health and Substance Misuse. 
The ADR Wales Team will continue to explore options for providing control groups. No perfect control group is available to the Project. This is because the vast majority of people requesting support receive services. Obtaining data to develop robust control or comparison groups was thus always expected to be challenging. Initially, and in discussion with the SPR&EG, the project envisaged creating various control groups of different kinds. Each group would be used to draw different inferences about the likely impact of the SPP. The inferences would differ based on the basis of their comparability with SPP users.

Both the feasibility and main stage phases of the Data Linking Project aimed to identify suitable control groups. The project has discussed constructing control groups with the SPR&EG at length. This includes discussion of potential methods and sources of data. Progress continues towards creating these groups.

Below is a description of the potential control groups with an update on progress.

Control group A: 'Matched controls method'

This method uses the anonymised routine records held in the SAIL Databank to select cases from the general population of Wales to match the socio-demographic profile of SPP service users. Controls are matched on age, gender, deprivation area and geographical location.

The Swansea University analyst completed work to construct this control group during Year Two. Because it was not known whether the individuals selected for this control group were experiencing a housing-related crisis – were theorised, in fact, to be significantly less likely to be doing so – it was felt this was unlikely to provide a useful comparison.

However, both the Welsh Government policy team and the SPR&EG have become interested in comparative analyses with the general population. In particular, in replicating studies from elsewhere where groups like the SPP recipients were found to be ‘chronologically’ older. This means the constellation of health and functional problems of older homeless adults resemble those of geriatric people in the general population. In particular, the SPR&EG were keen to see replicated analysis undertaken in other studies using similar control groups. Work is complete reproducing this analysis for the exemplar ‘lead need’ of Domestic Abuse. This work will be completed, as part of the broader analysis mentioned above, for the ‘lead needs’ of Mental Health and Substance misuse.

Control group B: 'Within Supporting People data derived method'

This method uses the anonymised routine records for individuals referred to the SPP but who did not receive support. The SPDLFS made use of this method. The SPDLFS found that SPP service users had on average around one more GP event per month than this comparison group in the months before SPP support began. The explanation for the difference in the level of GP use may include many factors. For example, the comparison group may have failed to engage because they were experiencing crises that were relatively less severe than those who did engage. Equally, their crises may have been so severe they were withdrawing not only from the SPP but also from primary care services. Hence, although potentially informative, this comparison group has limitations.

More work is needed to explore the data for this comparison group to learn more about the extent to which the above explanations are likely, in practice, to explain the differences observed. The project will complete further exploratory analysis of this control group. This will be part of the review to compare the methods used in the SPDLFS with those of the 'Emerging findings' report.

Control group C: 'Housing options data derived method'

Homelessness duties are discharged by Housing Options Services. Housing Options Teams have a duty to assess needs and take reasonable steps to prevent and help to relieve homelessness. The service allows people in need of housing to explore their housing and further support options. This includes those who are homeless or at risk of homelessness. These options include both social housing and alternatives to social housing as well as homelessness prevention/alleviation and SPP services. If Housing Options data can be obtained, a control group could be constructed by selecting the anonymised routine records for individuals who were referred to Housing Options, but did not receive support through the SPP.

It is unlikely that this will be an all Wales dataset in the short term. Individuals receiving Housing Options support are likely to be experiencing a housing-related crisis. Thus, this group will potentially make an informative comparison group for the analysis. However, those individuals who needed the further support provided by the SPP would have been referred to the SPP.

Work has begun by an academic research project to canvas all Welsh Local Authorities on their ability and willingness to provide Housing Options data. A pilot project for Swansea Local Authority is underway and Housing Options data has been sourced for SAIL.

Some local authorities also capture the recorded housing status at the end of SPP support.

The SPDLP can thus explore ways to construct this control group during Year 4.

Control group D: 'Segmentation on historical health care pattern utilisation'

This method aimed to identify individuals within the SAIL Databank with similar patterns of healthcare use to SPP service users. The project completed work to create this control group. However, there was an insufficient number of hospital admissions to create a statistically robust control groups. The project decided not to use this method going forward.

Developing further control groups

The ADR Wales Team will continue work to identify robust control groups. This may involve examining the feasibility of using further statistical methods such as cluster analysis. This might then be used to identify people in the general population with similar characteristics to SPP recipients.

As noted in the Year Two Progress Report, the Substance Misuse dataset held by NWIS (the NHS Wales Informatics Service) has now been imported into SAIL and made available to the project. The dataset captures data on all individuals presenting for Substance Misuse treatment in Wales. This includes both young people and adults. Welsh providers delivering Substance Misuse treatment and who are in receipt of Welsh Government Substance Misuse revenue funding are required to submit data. Exploratory analysis began during Year Three of the project. Analysis will continue as part of the Substance Misuse analysis mentioned above.

Social care data would provide the ability to examine the impact of the SPP on interactions with social care services, as discussed in the Feasibility report. Local Authority data would also help in developing control or comparison groups. Both the advantage and limitation of using social care data for comparison purposes would be similar to those for Housing Options. Individuals receiving support from social care would be likely to be experiencing a crisis, and may have been referred for a reason similar to SPP ‘lead needs’. The limitation would be that if those individuals were also at risk of homelessness, they should already have been receiving support from the SPP. Nevertheless, as one of a range of comparison groups, the project continues to pursue opportunities to source social care data, including through the ESRC-Welsh Government DDP. Some DDP pilot local authorities have offered to provide a range of datasets to the SAIL Databank, for some of which social care data remains under discussion.

The ESRC funding for ADR Wales has supported the creation of Strategic Impact Programmes (SIPs), programmes of work designed around the Welsh Government’s strategy Prosperity for All and the Well-being of Future Generations Act. Prosperity for All is our national strategy designed to identify the major societal challenges faced by the nation and drive integration and collaboration across the Welsh public sector. The 5 priority areas from Prosperity for All form the basis for 5 of the SIPs. The Social Care SIP is seeking to drive social care data from Local Authorities in Wales into the SAIL Databank.

Furthermore, Social Care Wales has convened a ‘Social care data access and use working group’. This forms part of Social Care Wales' research and development strategy for Wales 2017-2022. The group aims to bring together key players in the social care arena who have an interest in or role to play in this area. This group includes the ADR Wales lead for its Social Care Strategic Impact Programme. The group is working with the Association of Directors of Social Services Cymru (ADSS Cymru) to develop research questions of interest to ADSS. The group hopes working with ADSS Cymru will help drive data from local authorities into the SAIL Databank.

The SAIL Databank is also developing a dynamic care home register. This might provide an opportunity to look at movements into care homes for the Older People ‘lead need’. It would have the potential to provide a control group for Older People.

Objective 7: continue to explore the option of creating a cost-offset model using linked administrative data

Work with a Welsh Government economist to explore further the feasibility of creating a cost-offset model. The model would make use of a range of linked routine datasets.

Progress in year 3

During Year Two, the opportunity arose to work with an economist attached to the Government Social Research profession within Welsh Government to further explore the feasibility of developing of a cost-offset model using the anonymised individual-level data for the SPP in combination with the routine records already held in the SAIL Databank. The economist completed some initial work to develop a cost-offset model, and indicated that around 6 to 8 weeks further work was necessary to complete the model.

The availability of further time from an economist to complete the model is currently under discussion within Welsh Government.

Objective 8: source datasets for further indicators

Pursue the sourcing of datasets relating to further indicators. The project will prioritise data for social care, housing options and educational attainment. If resources allow, the project will pursue data for criminal justice and labour market participation. The project will agree timelines for sourcing with SPR&EG.

Progress in year 3

Datasets are being sourced through ADR UK to add to each ADR Centre in the UK. The partnership is targeting multiple GB and UK administrative and survey datasets. Datasets include ONS data, such as the Census 2011 and Annual Population Survey, and data from other UK Departments such as the Department for Education, DWP, and HMRC. Individual projects are also underway involving joint working between ADR Wales and UK government departments. This includes the Nuffield Family Justice Observatory, which is seeking to source data from, among others, the Ministry of Justice.

The ESRC and Welsh Government are jointly funding the DDP. It is envisaged the DDP will take on some of the data sourcing work for the SPDLP, in particular for Social Care and Housing Options.

The work involving the DDP is ongoing. It aims to bring additional Local Authority data into the SAIL Databank in the near future.

The addition of the above datasets will greatly expand the range of questions that can be answered about the SPP. They will help analysts to determine the efficacy of its interventions, and explore new potential impacts of the policy. This could include an assessment of the impact of the SPP on recipients’ interactions with the justice system. Examining interactions with the criminal and family justice systems could provide the potential for new policy interventions, for example by changing the approach the courts take to individuals either referred to or with the potential to be referred to the SPP. In a subset of SPP service users, it might be possible to plan interventions that would occur before interactions with the SPP based on interactions with the Police.

As mentioned above, the inclusion of Housing Options and Social Care data also has the potential to provide a robust control group for the SPDLP. This would enhance the ability to make robust inferences about the impact of SPP.

Objective 9: Report emerging findings of analysis of Supporting People data

Complete data cleaning and reconciliation across Local Authorities. The project will begin exploratory analysis and initial substantive analysis of SPP data linked to routine health records.

Progress in year 3

A noted above, there have been significant delays on the analysis work on the project. Staffing availability issues, the move to deliver the analysis within ADR Wales and the recruitment process for the proposed secondee have contributed.

As noted above, decisions were made during the year to focus on delivering analysis for Domestic Abuse as a demonstrator and, if the analysis proved both robust and informative, to deliver analysis of Mental Health and Substance Misuse. The Swansea University analyst completed analysis on the Domestic Abuse’ demonstrator. This analysis is being quality assured and reviewed before sharing with the SPR&EG.

As noted above, the Substance Misuse dataset held by NWIS has now been imported into SAIL and made available to the project. Exploratory analysis began during Year Three of the project. This work will continue as part of the ‘Substance Misuse’ analysis mentioned above. Work will continue to explore options for creating additional control groups.

Statistician: Craiger Solomons
Telephone: 0300 025 6960
Email: ADRUWales@gov.wales

Media: 0300 025 8099

Research report number: 37/2019