Measures of poverty: April 2020 to March 2021
Information about issues affecting the Households Below Average Income dataset for Wales for the period up to March 2021.
This file may not be fully accessible.
In this page
Summary
The Family Resources Survey (FRS) is an annual household survey managed by the Department for Work and Pensions (DWP). It forms the basis for the Households Below Average Income (HBAI) datasets, from which official statistics on low income, including relative income poverty, are produced on an annual basis.
- On 31 March 2022, the DWP released new FRS, HBAI and other related statistics including the first official poverty statistics for the period after the beginning of the coronavirus (COVID-19) pandemic, financial year 2020-21.
- Fieldwork operations for the 2020-21 FRS were rapidly changed in response to COVID-19 and the introduction of public health measures. Several factors impacted on response rates and characteristics of respondents to the survey.
- Analysis of the HBAI data below UK level is not recommended using this year’s data as the combination of smaller sample sizes and additional bias means it is not possible to make meaningful statistical assessments of trends and changes in the most recent year compared to the pre-coronavirus level.
- As a result, unlike previous years, the DWP have not included any Welsh poverty statistics in their latest HBAI release.
Due to the issues described above, we will not publish the usual range of additional Welsh Government analysis of the poverty data this year. Instead, this article describes the data quality issues, and presents Welsh poverty-related figures using the 2020-21 HBAI data alongside confidence intervals. We are doing this to ensure full transparency, but we advise against use of the unreliable 2020-21 dataset for Wales. Please refer to Relative income poverty: April 2019 to March 2020 for the latest National Statistics for Wales.
This article also signposts some alternative sources of poverty-related information, where there are Wales level data available for a period after the beginning of the coronavirus pandemic in 2020.
The changes to our publication are to be viewed as temporary and will be reviewed in collaboration with DWP as they develop the 2021-2022 dataset. It is too early to confirm whether the pandemic and changes to survey fieldwork will have any impact on the 2022 FRS and HBAI release.
Impact of the coronavirus pandemic on the Family Resources Survey
In the 2020-21 survey year, several factors impacted on response rates and the distribution of characteristics among Family Resources Survey (FRS) survey respondents, including:
- change in the mode of interviewing from face to face to telephone;
- changes in the methods used to elicit responses from survey participants as the year progressed; and
- changes in people's behaviours and circumstances during the coronavirus (COVID-19) pandemic which may have made them more or less likely to respond to a household survey.
The established face-to-face interviewing approach employed on the FRS was suspended and replaced with telephone interviewing for the whole of the 2020-21 survey year. This change impacted on both the size and quality of the achieved sample.
The move to telephone interviewing resulted in a reduction in the survey response rate and a smaller achieved FRS UK sample size of just over 10,000 households compared with 19,000 - 20,000 in previous years. The sample size was most affected in the first half of the survey year, particularly April 2020, due to the challenges involved in having to rapidly adapt the FRS fieldwork approach in response to the government restrictions on household mixing.
In addition, the sample composition (characteristics of respondents) was different to previous years. Whilst statistical methodologies (grossing regimes) in place bring selected sample profiles, including age and tenure, in line with the UK population, there may remain unobservable residual bias that cannot be corrected by the grossing regime. This means that the 2020-21 data is likely to be a less representative picture of poverty in Wales this year.
Impact on the Households Below Average Income dataset
The discontinuities and biases introduced by changes to FRS data collection during the pandemic mean that it is not possible to make meaningful statistical assessments of trends and changes for many breakdowns of HBAI estimates in the most recent year compared to the pre-coronavirus level.
DWP have identified that in 2020-21, low income rates were more variable across the year compared with previous years, largely affecting non-pensioner groups. To an extent this might be expected due to household incomes changing over the course of the pandemic in response to the wider context of legal restrictions, and associated changes to employment and unemployment. However, the patterns seen in the data are not always in the direction expected, at the times expected. It is difficult to quantify the impact of changes introduced to the FRS fieldwork on observed changes in low income rates.
The changes in in-year low income rates are more prevalent when looking at children, particularly when disaggregated further by UK country/region, which means country/regional estimates carry a greater degree of uncertainty.
Changes to Department for Work and Pensions publications
Following detailed quality assurance on the HBAI dataset, DWP have published the main, UK-level HBAI report, including confidence intervals and guidance to users on where to use caution. Alongside this they have published a Technical Report providing an assessment of the impact of the coronavirus pandemic on HBAI statistics.
The detailed tables which are usually published alongside the DWP HBAI report include some tables broken down by UK country and region. DWP have taken the decision not to publish these additional breakdowns for the 2020-21 data. They have issued a more limited range of UK-level data tables to support the HBAI report.
The source data behind these statistics will still be made available for download and further analysis via the UK Data Archive. DWP recommend that expert users exercise additional caution when using the data for 2020-21, particularly when making comparisons with previous years and interpreting changes in smaller subgroups.
Changes to Welsh Government publications
Following independent investigation of the HBAI data for 2020-21, Welsh Government statisticians have also found that the figures are not sufficiently robust to indicate poverty trends for Wales for the period ending March 2021.
The 2020-21 HBAI dataset includes fewer than 380 households in Wales, less than half usual annual levels. Even when combining three years’ worth of data (as is usual for regional/country estimates) the small sample and the additional bias in the latest year of data is enough to significantly compromise the quality of the statistics.
Due to the issues described above, we will not publish the usual range of additional Welsh Government analysis of the poverty data this year. Instead, below we present key Welsh poverty-related figures based on 2020-21 HBAI data alongside confidence intervals to reflect additional uncertainty. We are doing this to ensure full transparency, but we advise against use of the unreliable 2020-21 dataset for Wales.
We have published a brief update on new DWP statistics on persistent poverty for Wales for the period ending December 2020. These figures come from the Understanding Society study, which was not affected by the same data quality issues described above.
Analysis
In this section we present Wales and UK poverty-related figures based on the 2020-21 HBAI dataset. However, due to the reasons outlined above, the latest estimates for the period ending 2020-21 are not robust enough to accurately describe changes to poverty in Wales since the pandemic began, and we do not recommend using this year’s data. In the tables and charts below, we have labelled some data as “unreliable” to reflect this.
A Technical Report assessing the impact of the coronavirus pandemic on HBAI estimates has been published by DWP and should be considered alongside interpretation of any analysis in this article. We are not satisfied that the sample which informs the Welsh estimates is representative and meets the standards required of National Statistics output.
For that reason, Welsh poverty-related statistics published last year (data for the period ending 2019-20) should be those used to understand poverty in Wales. Relative income poverty and material deprivation statistics for the period ending 2019-20 can be found on the poverty statistics pages, whilst absolute income poverty statistics for Wales can be found on the gov.uk HBAI pages.
In terms of the distribution of household incomes and poverty, 2020-21 is likely to have been an outlier due to the level of disruption brought about by COVID-19 and the public health measures introduced to reduce transmission, the impact of the furlough scheme and the temporary uplift to Universal Credit. It will be several years until we fully understand the long-term impact of the COVID-19 pandemic on people’s financial situations. In the meantime, for data beyond 2019-20, users can refer to DWP’s 2020-21 reports on household incomes and poverty for the UK more broadly, and to additional sources of information related to poverty in Wales described in the section on “alternative measures” below.
Measures of uncertainty
The smaller sample size achieved this year means we cannot say with confidence that any changes to Wales level poverty figures in the short term are meaningful. In other words, changes are not statistically significant and may well have only arisen by chance due to variation in the sampling.
The uncertainty in the figures is illustrated in the charts below, which show rates of poverty in the UK and Wales up to the 2020-21 financial year, alongside 95% confidence intervals. Confidence intervals give a range within which the ‘true’ value for the population is likely to fall. There is a 1 in 20 chance that the 95% Confidence Interval does not contain the true value. The larger the confidence interval, the less precise an estimate is.
Please note that when generating confidence intervals there is no statistical method of further widening them to capture the impact of unquantifiable sampling error or bias, as outlined above and in DWP’s Technical Report.
DWP have assessed that high level UK figures for 2020-21 are sufficiently reliable to publish, therefore we have included latest UK statistics alongside the (unreliable) Wales figures below to provide some insight into what has happened to poverty across the UK during the pandemic. We have also included UK estimates in the charts where possible, based on 3 financial-year averages to match the reference periods for Wales figures.
Relative income poverty
We define a person to be living in relative income poverty if they live in a household where the total household income from all sources is less than 60% of the average UK household income (as given by the median). The figures below relate to relative poverty in Wales after housing costs such as mortgage interest payments/rent, water rates and structural house insurance were paid.
Below is a table of relative income poverty estimates for Wales and the UK, followed by charts illustrating via confidence intervals the level of uncertainty around the poverty rate for Wales, for each age group separately. Please note that as well as sampling variation, as described previously there are additional biases in the 2020-21 dataset which further reduce the accuracy of the latest figures for Wales.
| Measure of poverty | Wales, 2017-18 to 2019-20 | Wales, 2018-19 to 2020-21, This estimate is unreliable |
UK, 2019-20 | UK, 2020-21 |
|---|---|---|---|---|
| People living in relative income poverty | 23% | 24% | 22% | 20% |
| Children living in relative income poverty | 31% | 34% | 31% | 27% |
| Working-age adults living in relative income poverty | 22% | 22% | 20% | 20% |
| Pensioners living in relative income poverty | 18% | 19% | 18% | 15% |
Source: Households Below Average Income, Department for Work and Pensions
Absolute income poverty
Relative income poverty (relative to the contemporary UK median income) is the key measure we use to describe poverty in Wales. However, DWP also produce a measure of absolute income poverty, which can add to our understanding of trends over time.
The absolute measure is designed to assess how low incomes are faring with reference to inflation. A person is in absolute low income if they live in a household where the total household income is below 60% of the 2010 to 2011 median income adjusted for inflation. The percentage of individuals in absolute low income will decrease if individuals with lower incomes see their incomes rise by more than inflation. This makes it a useful measure to use alongside relative measures, in periods when there are significant changes to the income distribution over time, such as during a recession or a pandemic.
Absolute income poverty (after housing costs) data are shown below to provide more context for this poverty analysis. However, both relative and absolute measures are considered unreliable for Wales for the most recent period ending 2020-21.
Below is a table of absolute income poverty estimates for Wales and the UK, followed by charts illustrating via confidence intervals the level of uncertainty around the poverty rate for Wales, for each age group separately. Please note that as well as sampling variation, as described previously there are additional biases in the 2020-21 dataset which further reduce the accuracy of the latest figures for Wales.
| Measure of poverty | Wales, 2017-18 to 2019-20 | Wales, 2018-19 to 2020-21, This estimate is unreliable |
UK, 2019-20 | UK, 2020-21 |
|---|---|---|---|---|
| People living in absolute income poverty | 19% | 18% | 18% | 17% |
| Children living in absolute income poverty | 24% | 25% | 25% | 23% |
| Working-age adults living in absolute income poverty | 19% | 18% | 17% | 16% |
| Pensioners living in absolute income poverty | 14% | 13% | 13% | 11% |
Source: Households Below Average Income, Department for Work and Pensions
Material Deprivation
In the HBAI publication, DWP publish estimates of the percentage of households with children who are materially deprived, including in combination with a series of low-income thresholds, and also estimate of the percentage of pensioner households who are materially deprived.
Material deprivation is a measure of living standards. We define a person to be living in material deprivation if he or she is not able to access a certain number of goods and services. In 2020-21, several of the questions asked as part of the measure were affected by legal restrictions introduced in response to the coronavirus pandemic. This meant that it was not possible for those sampled to access the social opportunities or services during periods of lockdown, regardless of deprivation.
Because of this, 2020-21 estimates of material deprivation are not comparable with previous years, and we cannot say that any change in recorded material deprivation during 2020-21 represents a genuine change in household circumstances. This is explained further in DWP’s Technical Report. In addition, for figures based on data from 2020-21 the same quality issues relating to the sample and biases as described above apply for material deprivation analysis, but we show the figures to ensure full transparency.
Further details on the measures are provided on our Material deprivation and low income webpage.
Below is a table of material deprivation estimates for Wales and the UK, followed by charts illustrating via confidence intervals the level of uncertainty around the material deprivation estimates for Wales, for children and pensioners separately. Please note that as well as sampling variation, as described previously there are additional biases in the 2020-21 dataset which further reduce the accuracy of the latest figures for Wales.
| Measure of material deprivation | Wales, 2017-18 to 2019-20 | Wales, 2018-19 to 2020-21, This estimate is unreliable |
UK, 2019-20 | UK, 2020-21 |
|---|---|---|---|---|
| Children living in material deprivation and low income | 14% | 14% | 12% | 10% |
| Pensioners living in material deprivation | 7% | 6% | 6% | 5% |
Source: Households Below Average Income, Department for Work and Pensions
Alternative measures
This section signposts some alternative sources of poverty-related information, where there are Wales level data available for a period after the beginning of the coronavirus pandemic in 2020. The latest available figures (as at March 2022) are shown, however other data referring to earlier, post-pandemic periods may be available from the underlying sources.
Results from the National Survey for Wales telephone survey 2020-21 show that 79% of adults said they were keeping up with all their bills and commitments without any difficulty, compared with 68% in 2019-20. However, given that the National Survey switched from face-to-face to telephone mode in April 2020, care should be taken in making direct comparisons.
The Survey of public views on COVID-19 is an online study by Ipsos MORI of adults aged 16 to 74 in Wales. It shows that:
- The proportion of people in Wales concerned that they will not be able to pay their bills in a month’s time remained broadly similar since the start of the pandemic, at around 25%.
- Latest figures suggest a recent increase up to 35% in March 2022, although differences between waves should be interpreted with some caution due to the credibility intervals involved.
Public Health Wales are conducting a public engagement telephone survey to ask members of the public in Wales how coronavirus and related control measures are affecting their health and wellbeing. The survey began in April 2020 and currently operates on a monthly basis. The latest report (Public Health Wales) from February 2022 finds that 16% of adults 18+ worried ‘a lot’ about their finances over the past week, and 28% of adults 18+ worried ‘a little’ about their finances over the past week.
DWP have released new “Children in low income families” local area data for 2020-21, based on administrative datasets and calibrated to HBAI regional figures. Comparisons across countries or years is not advised, and statistics show local area variations within each country for 2020-21 only.
Although not official statistics, users may wish to refer to this report for additional context: A snapshot of poverty in Winter 2021 (Bevan Foundation).
Notes on the use of statistical articles
Statistical articles generally relate to one-off analyses for which there are no updates planned, at least in the short-term, and serve to make such analyses available to a wider audience than might otherwise be the case. They are mainly used to publish analyses that are exploratory in some way, for example:
- introducing a new experimental series of data
- a partial analysis of an issue which provides a useful starting point for further research but that nevertheless is a useful analysis in its own right
- drawing attention to research undertaken by other organisations, either commissioned by the Welsh Government or otherwise, where it is useful to highlight the conclusions, or to build further upon the research
- an analysis where the results may not be of as high quality as those in our routine statistical releases and bulletins, but where meaningful conclusions can still be drawn from the results.
Where quality is an issue, this may arise in one or more of the following ways:
- being unable to accurately specify the timeframe used (as can be the case when using an administrative source)
- the quality of the data source or data used
- other specified reasons.
However, the level of quality will be such that it does not significantly impact upon the conclusions. For example, the exact timeframe may not be central to the conclusions that can be drawn, or it is the order of magnitude of the results, rather than the exact results, that are of interest to the audience.
The analysis presented does not constitute a National Statistic, but may be based on National Statistics outputs and will nevertheless have been subject to careful consideration and detailed checking before publication. An assessment of the strengths and weaknesses in the analysis will be included in the article, for example comparisons with other sources, along with guidance on how the analysis might be used, and a description of the methodology applied.
Articles are subject to the release practices as defined by the release practices protocol, and so, for example, are published on a pre‑announced date in the same way as other statistical outputs.
Contact details
Statistician: Nia Jones
Telephone: 0300 025 4088
Email: stats.inclusion@gov.wales
Media: 0300 025 8099
SFR 107/2022