The three-year ECHOUTCOME project found that the key assumptions underlying NICE’s QALY evaluations are false, and decisions based on them are dangerously unreliable.
The report recommends that prescribing decisions be made relative to specific conditions and patient groups, rather than relying on a general algorithm.
NICE’s evaluation system is being considered by other European health technology assessment bodies, but ECHOUTCOME advises against its use.
The QALY system estimates the number of years of healthy life gained by the patient, with adjustments for the likely impact of the patient’s symptoms, to reach a figure for the number of Quality Adjusted Life Years (QALYs).
If the cost per QALY for each patient is estimated to be below £30,000, NICE will usually recommend the treatment.
But according to the European researchers, NICE is basing its quantitative assessments on factors that are both qualitative and ill-conceived.
The ECHOUTCOME study, conducted by six universities and several research agencies, analysed data from 1,300 respondents in Belgium, France, Italy and the UK and concluded that QALY cannot capture medical outcomes accurately.
QALY relies on four key assumptions: time and quality of life can be measured in consistent intervals; life years and quality of life are linked; patients have a neutral attitude towards risk; and willingness to sacrifice life years for quality of life is constant over time.
According to the study, these assumptions are all false, and so QALY has no valid empirical basis.
Gerard Duru, Emeritus Research Director in Mathematics at the French National Centre of Scientific Research, commented: “The underlying assumptions of the QALY outcome are very theoretical and are not verified in a real population. It is impossible to know what we are measuring, and therefore impossible to base a formula upon it.”
According to ECHOUTCOME’s Project Leader, Ariel Beresniak, a different approach is needed: “Each case is different and each should use adequate evaluation tools. There should be a list of adequate validated tools to make these decisions with guidance on when to use each.”
For example, the report recommends using a cost per remission approach to evaluate drugs treating rheumatoid arthritis, and a cost-benefit approach to evaluate drugs for wide-scale vaccinations.