CQC is in the process of developing a new statistical surveillance tool which is intended to help guide its inspection programme towards those providers who are most at risk of providing poor care. The tool will be used to focus inspections of NHS trusts and will be based on its ‘Intelligent Monitoring’ (IM) tool that has been utilised to monitor risk over the past few years. The original IM statistical surveillance tool was designed to collate 150 different scores across hospitals such as whistleblower reports and mortality in order to identify those providers most at risk. An analysis of the effectiveness of the original IM statistical surveillance tool was recently published in BMJ Quality and Safety Journal. The paper concluded that the tool was not able to achieve its purpose of predicting the outcome of NHS hospital trust inspections and stated it cannot be used for the prioritisation of inspections. It therefore recommended that a new approach to inspection planning was required.
Research conducted by Alex Griffiths, London School of Economics also raises concerns about the accuracy of this type of tool: –
“The process the CQC had, and elements of it are being used in their new system, wasn’t up to the job. It cannot effectively work and the risk is that poor care can continue longer than it should do and good quality providers will be unfairly burdened. There is certainly a danger that the revised system will not effectively predict the risk outcomes, which is what is necessary if you are becoming ever more reliant on it as funding is reduced.”
“The risk estimates were wrong more often than they were right. The tool was, in that sense, actively misleading. The danger of that is that if you’re missing poor quality care then people can go on suffering. There is also the opportunity cost – that if you’re burdening high quality providers then you’re distracting them from administering the good quality care at the same time, so there is a double cost of getting it wrong.”
CQC’s response to the research stated: –
“Our intelligent monitoring tool served a clear purpose, which was to help prioritise our comprehensive inspection programme of every NHS trust in this country. It was a planning tool, helping with our initial scheduling – it was not a formal judgment of risk or a tool to predict ratings… This will consider more intelligence in real time and will place greater emphasis on qualitative data, such as the views of individual healthcare professionals, patients and local partners. Ultimately, this will allow us to be even more responsive to risk and to carry out more focused and targeted inspections of NHS services.”
Any system which is overly reliant on data to predict areas of risk may hold within it an element which is capable of being misread or distorted. The level of distortion based on the IM statistical surveillance tool is 52% which suggests that risk would have been missed more often than being captured when based on that tool alone. CQC has confirmed that a new tool is being developed but the existence of a tool which, although used in the initial planning stages, could be found to be so greatly inaccurate does not inspire confidence.
CQC’s intention to move to a more risk-based model of regulation across the sector could be questioned if the processes it has to target those risks prove to be at fault.
