Mark Drakeford, Minister for Health and Social Services
Today I publish the review undertaken by Professor Stephen Palmer, the Mansel Talbot Professor of Epidemiology and Public Health at Cardiff University, into the use of risk-adjusted mortality data in NHS Wales.
The Welsh Government and NHS Wales are committed to transparency of patient outcome and have been publishing an increasing number of quality and performance indicators for all to see over the last 18 months.
One measure which has caused particular public confusion is the risk adjusted mortality index (RAMI), which in Wales is calculated by an external company for individual district general hospitals. To be specific, there has been – and continues to be - widespread uncertainty among experts and the public, about the extent to which a single figure can indicate the quality of care delivered by the wide and complex range of services within a hospital.
To address this important question, I asked Professor Palmer, a distinguished epidemiologist, to undertake a review to advise me about the following questions:
- To what extent does risk adjusted mortality data provide valid information?
- How can risk adjusted mortality measures be interpreted for NHS Wales?
- Do hospitals with risk adjusted mortality indicators above 100 require further work on data or clinical quality?
- How are NHS Wales boards using clinical data for quality improvement?
- What is the quality of clinical data used by NHS Wales for improvement purposes
- Are NHS organisations using clinical data effectively for quality improvement?
- What do the clinical data tell us about the quality of care in NHS Wales?
I would like to thank Professor Palmer for his considered report. He has undertaken significant stakeholder engagement and applied the wealth of his personal knowledge and experience to this task. His report provides a valuable expert overview of the ways in which quality and safety data are collected, processed and interpreted across the NHS in Wales and I hope it will be widely read.
Professor Palmer concludes RAMI is not a meaningful measure of quality, indeed he says it is misleading.
He does, however, make recommendations for Welsh Government and NHS Wales to ensure it has meaningful and useful information to measure and describe quality care in hospitals.
Professor Palmer supports the mortality case review process, which is in place for all deaths in Welsh hospitals – this is an area in which Wales leads the UK. He advises ways in which this review process can be further strengthened by improved consistency and clinical engagement. Professor Palmer also makes recommendations to improve clinical coding, participation in national clinical audits and data interpretation for Boards.
The Chief Medical Officer for Wales is today writing to all clinicians in NHS Wales to reinforce their responsibilities about medical records, which is, after all a patient safety matter, but which is also important to ensure clinical coding is optimised.
When I commissioned this review I also asked Professor Palmer to look at six hospitals, which had a Welsh RAMI score of more than 100 in March 2014. I asked him to let me know if, in these cases, the RAMI figure represented the sounding of a "smoke alarm". In five of the six hospitals, Professor Palmer concludes, clearly, that it is not.
As far as Prince Charles Hospital, in Merthyr Tydfil and Royal Glamorgan Hospital, in Llantrisant, are concerned Professor Palmer concludes the RAMI scores of more than 100 are expected from the way in which the measure is calculated. He also says Cwm Taf University Health Board has an impressive and highly visible clinical case note mortality review process in place, which provides considerable reassurance to the board that high RAMI scores are not indicators of poor care.
At Neath Port Talbot Hospital and Singleton Hospital, in Swansea, Professor Palmer concludes that the RAMI scores are probably the result of coding and classification changes and not a reflection of deficiencies in quality of care. He does however recommend that Abertawe Bro Morgannwg University Health Board takes steps to fully embed the mortality review process.
In relation to Wrexham Maelor Hospital, Professor Palmer does not consider that the report of the RAMI to require any additional action. Furthermore, he says a rigorous and systematic mortality review process, which covers all hospital deaths, is in place with four out of 10 deaths referred for an in depth review.
As far as Glangwili Hospital, in Carmarthen, is concerned, the review concludes that the high RAMI score appears to be a consequence of changes to the way the 2013 RAMI is calculated.
However, Professor Palmer believes further work is needed to understand the ways in which changes to the classification of palliative care cases and other matters may have had an impact on the RAMI score. He supports the additional work the medical director at Hywel Dda University Health Board is currently carrying out.
There are many similarities between Professor Palmer’s conclusions about RAMI and what value it can provide to the NHS and the Welsh public’s understanding of the quality of hospital care provided across Wales and a report published by the Deputy Chief Medical Officer’s transparency and mortality taskforce in March 2014.
The taskforce will therefore reconvene to consider Professor Palmer’s report and his recommendations in detail. It will provide further advice in the autumn, as to how the Welsh Government and NHS Wales should respond.
I commend Professor Palmer’s report to all Members. It confirms my own view – and that of many other experts and academics – that RAMI is largely an unhelpful set of numbers. However, I share Professor Palmer’s belief that there are more helpful measures which we must record and review with consistency and rigour.
It provides an authoritative basis on which debate about the meaning and usefulness of RAMI data can be moved forward. I share Professor Palmer’s conclusion that mortality case note reviews provide a more helpful measure, and one which can be further improved through consistent and rigorous recordings and review. As previously indicated to Members, I intend to report more extensively on the use of mortality case note reviews in September.