3 Committee discussion

The appraisal committee considered evidence submitted by Janssen, a review of this submission by the evidence review group (ERG), and responses from stakeholders. See the committee papers for full details of the evidence.

This review looks at data collected as part of the Cancer Drugs Fund to address uncertainties identified during the original appraisal for ibrutinib. Further information about the original appraisal can be found in the committee papers. As a condition of the Cancer Drugs Fund funding and the managed access arrangement, the company was required to collect updated efficacy data from study 1118E. In addition, data was collected on ibrutinib for people with Waldenstrom's macroglobulinaemia who had had at least 1 previous therapy in the NHS through the Cancer Drugs Fund using the Systemic Anti-Cancer Therapy (SACT) dataset.

The managed access arrangement for ibrutinib to be used in the Cancer Drugs Fund did not cover the whole population included in the marketing authorisation for ibrutinib and was limited to people who had had at least 1 previous therapy.

Treatment pathway and clinical need

There is high clinical need for alternative treatments to rituximab and chemotherapy

3.1 Waldenstrom's macroglobulinaemia is an incurable, rare type of non-Hodgkin lymphoma with limited treatment options. There is no established standard care. Treatment options include chemoimmunotherapy, such as rituximab combined with a range of chemotherapy regimens including alkylating agents (such as cyclophosphamide) or nucleoside analogues (cladribine or fludarabine). When chemoimmunotherapy is unsuitable, treatment options include monotherapy with rituximab or chlorambucil. The marketing authorisation for ibrutinib includes people for whom chemoimmunotherapy is unsuitable but this population was not included in the managed access arrangement for use of ibrutinib in the Cancer Drugs Fund. The clinical experts explained that generally, the disease responds well to chemotherapy but there are a restricted number of lines of chemotherapy that can be used because of cumulative toxicity. Chemotherapy options can rapidly become exhausted, leaving no effective therapies available. Some people may also have intolerance to rituximab. The patient expert highlighted that the constant risk of relapse has a significant effect on the mental wellbeing of patients and families. The committee noted that Waldenstrom's macroglobulinaemia is associated with major disease-related symptoms such as neutropenia which can cause infections, weakness, extreme fatigue and breathlessness. The patient expert explained that symptoms like fatigue can be intense, disabling and significantly impair day-to-day life. The committee concluded that there is no standard care for treating Waldenstrom's macroglobulinaemia and there is high unmet clinical need for new and effective therapies.

Ibrutinib is a step-change in managing Waldenstrom's macroglobulinaemia

3.2 The clinical experts explained that ibrutinib is a novel and effective non-chemoimmunotherapy treatment option for disease that is refractory to first-line treatment or that has relapsed after successful first-line therapy. Both the patient and clinical experts emphasised that ibrutinib is highly effective compared with existing treatments, and very well tolerated. The convenience of an oral therapy is also greatly valued by patients because it allows them to take the treatment at home, with no need for hospital visits for infusions. The patient expert explained that he had been having ibrutinib for several years. He found it to be a life-transforming drug that had dramatically improved his quality of life, allowing him to take part in general day-to-day activities and very quickly return to normal life. The clinical experts noted that although the condition often responds to chemoimmunotherapy, the speed and durability of response is better with ibrutinib, meaning that people having ibrutinib "feel better" a lot more quickly than with chemoimmunotherapy. The clinical experts attributed the highly effective, immediate, and durable disease control with ibrutinib to be a step-change in the management of Waldenstrom's macroglobulinaemia. The committee agreed and concluded that the availability of an effective and well-tolerated oral treatment option as an alternative to chemoimmunotherapy is highly valued by both patients and clinicians and ibrutinib is a step-change in managing Waldenstrom's macroglobulinaemia.

Clinical evidence

SACT data is generalisable to NHS clinical practice and more relevant than updated trial data from study 1118E and iNNOVATE arm C

3.3 In the original appraisal, clinical-effectiveness evidence for ibrutinib came from 2 sources:

  • a single-arm, open-label study without a control group (study 1118E) for 63 adults with Waldenstrom's macroglobulinaemia who had had at least 1 previous therapy

  • an open-label sub-study of 1 arm of a randomised controlled trial (iNNOVATE arm C) with no comparator data for 31 people with Waldenstrom's macroglobulinaemia whose disease relapsed within 12 months of having treatment containing rituximab.

    The committee noted that the original appraisal concluded that further data collection on progression and overall survival was needed in the Cancer Drugs Fund because of uncertainties about long-term survival. The updated data submitted by the company included:

  • additional 22 months clinical trial evidence from study 1118E (median age 63 years)

  • additional 40.9 months clinical trial evidence from iNNOVATE arm C (median age 67 years)

  • new observational data for people in the Cancer Drugs Fund obtained from the SACT dataset for 823 people with Waldenstrom's macroglobulinaemia that had at least 1 previous therapy before ibrutinib (median age 75 years)

  • new observational data from a UK clinical registry (Rory Morrison Registry) for 112 people with Waldenstrom's macroglobulinaemia that had ibrutinib as second or subsequent-line treatment; the company and clinical experts noted that this data was a subset of people from the SACT dataset; the committee noted that the company considered the SACT data the most generalisable to how ibrutinib would be used in clinical practice.

    The committee understood that the SACT database did not collect data on disease progression. It also noted that the cost-effectiveness estimates are dependent on the progression-free survival of people having ibrutinib compared with standard care. The committee noted that the proportion of patients alive at 24 months in study 1118E and the SACT was 95% and 73% respectively. People who had ibrutinib through the Cancer Drugs Fund (and who were included in the SACT dataset) were on average older than people in the trials. So, it acknowledged that real world evidence could be associated with lower overall survival rates than trial evidence because of potential differences in patient baseline characteristics. The committee also noted that people having ibrutinib in the SACT dataset may have had multiple previous treatments before ibrutinib, and that it was expected to be used primarily as a second- or third-line treatment in the future. The Cancer Drugs Fund clinical lead explained that real world evidence from 823 patients for a rare disease like Waldenstrom's macroglobulinaemia offers best available data and noted that 84% of people in the SACT cohort had only 1 or 2 lines of previous therapies in keeping with expected use. The clinical experts also agreed that the SACT cohort was reflective of current and future use of ibrutinib in the NHS. The committee concluded that SACT data is generalisable to NHS clinical practice and is more relevant than updated trial data from study 1118E and iNNOVATE arm C.

The agreed approach for estimating ibrutinib progression-free survival indirectly from SACT data is reasonable but uncertain

3.4 Because progression-free survival data was not collected in SACT, the company indirectly estimated progression-free survival for ibrutinib using time to treatment discontinuation data from SACT. In the original appraisal for ibrutinib, time to treatment discontinuation had been assumed to be equal to progression-free survival. For the current appraisal, the company did not consider time to treatment discontinuation a good proxy for progression-free survival because 67% of people stopped treatment before disease progression in the SACT dataset. At technical engagement, the company accepted the ERG preferred approach of estimating progression-free survival for the ibrutinib group by assuming a proportional relationship between time to treatment discontinuation and progression-free survival in the Rory Morrison Registry, based on a comparison of exponential survival models fitted to this data. The hazard ratio between time to treatment discontinuation and progression-free survival was then applied to the time to treatment discontinuation SACT data. The ERG noted that this approach assumes that the hazards for time to treatment discontinuation compared with progression-free survival in the Rory Morrison Registry are proportional and that the same relationship is found in other populations such as people in the whole SACT cohort. The clinical experts stated that people generally stay on treatment until disease progression and may even stay on treatment after disease progression because ibrutinib may slow subsequent disease progression. However, some people will stop treatment before progression, and progression happens soon after treatment discontinuation. The committee noted that time to treatment discontinuation in SACT was lower than in the Rory Morrison Registry. A clinical expert considered this may have been because of variation in clinical practice. They explained that some people would stay on treatment with ibrutinib because of clinical benefit despite clinical assessment of disease progression. The committee concluded that in the absence of progression-free survival data from the SACT database, the agreed approach to indirectly estimate it was reasonable but subject to uncertainty.

Based on an indirect comparison the extent to which ibrutinib improves progression-free survival compared with standard therapies is uncertain

3.5 The company did not update its estimate of the hazard ratio for progression-free survival on ibrutinib compared with standard therapies from the original appraisal of ibrutinib. But it updated its estimate of progression-free survival on ibrutinib to be applicable to the SACT cohort. In the original appraisal for ibrutinib, the company presented an indirect comparison of progression-free survival with ibrutinib compared with existing treatments for Waldenstrom's macroglobulinaemia. This was referred to as 'physician's choice of standard therapies' (a blend of alternative second-line or more rituximab plus chemotherapy options) and will be referred to as standard therapies going forward in this document. The data for standard therapies came from a European chart review; a retrospective observational study that generated data on epidemiology, treatment and efficacy outcomes for untreated and relapsed Waldenstrom's macroglobulinaemia over 10 years. The committee recalled that in the original appraisal, the company created a matched cohort to the study 1118E population by selecting a subset of the European chart review cohort who had had similar lines of therapy to the people in study 1118E. The indirect comparison estimated a hazard ratio for progression-free survival for ibrutinib compared with standard therapies of 0.25 (95% confidence interval [CI] 0.11 to 0.57, p=0.001). This suggested a substantial reduction in the risk of disease progression with ibrutinib compared with standard therapies. The committee noted that the indirect comparison was of trial data for ibrutinib compared with data from a non-trial setting (real world evidence) for standard therapies which made the hazard ratio highly uncertain. The ERG explained it had concerns about methods used to select people in the matched cohort. In the submission for the current appraisal, the company stated that it considered its original estimate, based on 24 months data from study 1118E to be relevant and best available estimate of the hazard ratio for progression-free survival with ibrutinib compared with standard therapies. On the ERG's request, the company updated the indirect treatment comparison with additional study 1118E long-term data using an unanchored matching-adjusted indirect comparison (MAIC) with the full dataset from the European chart review after technical engagement. The hazard ratio from this analysis is 0.28 (95% CI 0.10 to 0.49). The ERG explained that the MAIC is useful in providing supporting evidence of the relative treatment effect on progression-free survival for ibrutinib compared with standard therapies but is an unanchored comparison meaning that data from single-arm studies, rather than studies with a common comparator, had been included. This approach assumes that prognostic variables and effect modifiers have been accounted for. Given limited data on prognostic variables available with most trial data needing to be imputed, the results from the MAIC based on this assumption are highly uncertain. The committee discussed the size of the estimated treatment effect of ibrutinib compared with standard care which showed a large benefit of ibrutinib in delaying disease progression. The clinical experts and Cancer Drugs Fund clinical lead stated that a large benefit was plausible, and the committee noted the statements from the clinical and patient experts that ibrutinib was a step-change in managing Waldenstrom's macroglobulinaemia (see section 3.2). The committee recalled its conclusion in the original appraisal that the hazard ratio was highly uncertain. The committee accepted, based on the results of the indirect comparison and the testimonies from patients and clinical experts, that ibrutinib appears to be more clinically effective than existing treatments but concluded that there remains significant uncertainty around the extent to which ibrutinib improves progression-free survival.

Cost effectiveness

The company's updated model is suitable for decision making

3.6 The company used the same modelling approach as in its original appraisal for ibrutinib, that is a Markov state transition model comparing ibrutinib with standard therapies. The health states included progression-free survival and having up to 4 follow-on treatments after progression. It also modelled the chance of dying in each health state. The modelled cohort was updated to reflect the population for whom data was collected in the SACT database rather than study 1118E, that is a population with an average age of 70. Updated data and assumptions used in the model included:

  • adverse event frequencies from additional long-term clinical outcomes data from study 1118E

  • time to treatment discontinuation for ibrutinib from the SACT dataset

  • an updated approach to modelling pre-progression mortality with ibrutinib based on SACT data, with a calibration so that overall modelled survival with ibrutinib reflected overall survival observed in the SACT dataset

  • progression-free survival for ibrutinib was based on the indirect estimates of progression-free survival in the SACT database based on the time to treatment discontinuation seen in this cohort.

    To model progression-free survival on standard therapies, the updated model applied the hazard ratio from the original appraisal for progression-free survival with ibrutinib compared with standard therapies. However, because the modelled progression-free survival was updated for ibrutinib in the current appraisal, this resulted in a different modelled progression-free survival for standard therapies than the original appraisal. The modelled time on treatment for standard therapies was assumed to be the same as progression-free survival. This meant that the modelled time on treatment for standard therapies was also different to that in the original appraisal. The committee noted that the company had aligned its modelling with the ERG's preferred assumptions for estimating progression-free survival for the SACT dataset (see section 3.4) and the ERG's approach for modelling pre-progression mortality with ibrutinib and calibrating modelled overall survival for ibrutinib to reflect the observed data from the SACT dataset. The committee considered the company's approach to use the SACT data is reasonable but noted that the evidence available to estimate progression-free survival for ibrutinib and any outcome in the standard therapies group, which is dependent on the progression-free survival hazard ratio for ibrutinib compared with standard therapies, is subject to considerable uncertainty. The committee concluded that the company's updated model is suitable for decision making, but the modelling uncertainties should be considered.

The model predicted survival outcomes from the company's revised base-case model do not correlate with clinical experience and are highly uncertain

3.7 The committee discussed the modelled clinical outcomes from the company's revised base case. There were 3 modelled outcomes which did not reflect outcomes seen in clinical practice:

  • The modelled delay between stopping treatment and disease progression. The revised model predicted a 6‑month delay before disease progression after stopping treatment with ibrutinib. This was not considered reflective of clinical practice in the NHS by the clinical experts. They explained that most people would still be on treatment when their disease progressed, or if they had stopped treatment before progression would be expected to have disease progression soon after stopping (see section 3.4). The company stated at the second meeting that its analysis of the time between disease progression and stopping treatment for people in the SACT database supported what it had modelled.

  • The modelled post-progression survival, that is the time between a person's disease progressing and their death. The revised model predicted that post-progression survival for people in both the ibrutinib and standard therapies was about 1 year. The clinical experts explained that at least two thirds of patients whose disease progresses while on ibrutinib treatment achieve good response to further lines of chemotherapy and that the median time between disease progression and death in clinical practice is much more than a year.

  • The modelled overall survival in the standard therapies treatment arm. The committee noted that the modelled overall survival in the original appraisal for ibrutinib was longer for both ibrutinib and comparators than in the revised model although the life year gain in both was predicted to be around 3 years. It noted that the modelled overall survival estimates for ibrutinib reflected those seen in the SACT data were lower than the original appraisal which had used data from an older population (with an average age of 75) than study 1118E. The revised model for the current appraisal also predicted that nearly all patients in the standard therapies arm would have died by 6 years after starting treatment for relapsed or refractory Waldenstrom's macroglobulinaemia, and that the mean modelled survival on standard therapies was less than 2 years, which the clinical experts explained is unrealistic and clinically implausible. The clinical experts considered that the longer survival estimate (3 years more in each arm) predicted by the model used in the original appraisal for ibrutinib based on study 1118E data was more reliable and clinically plausible than outcomes predicted by the revised company model. The committee recalled that the company submission for the original appraisal noted that Waldenstrom's macroglobulinaemia is an indolent disease and provided estimates of median life expectancy ranging from 4 to 12 years.

    The committee considered that although the key finding of a 3‑year overall survival gain for ibrutinib compared with standard therapies as predicted by the model for the original appraisal and the revised model for the Cancer Drugs Fund review could be plausible, there remained considerable concern about the validity of the modelled overall survival in the standard therapies arm. Furthermore, the modelled time between stopping ibrutinib and disease progression, and time between disease progression on a person's first treatment for relapsed or refractory Waldenstrom's macroglobulinaemia did not reflect what is seen in clinical practice. The committee agreed that these 3 modelled outcomes were dependent on the hazard ratio for progression-free survival for ibrutinib compared with standard therapies (see section 3.5) and updated estimates of progression-free survival for ibrutinib (see section 3.4). However, it agreed that although these were the most reasonable given the available data, these were uncertain because of the limitation of not having progression-free survival data from SACT and data directly comparing progression-free survival for ibrutinib compared with standard therapies. The committee concluded that although the model gave some implausible outputs, given the limitations in the available data, there was no alternative analysis that could be done to address these concerns. Therefore, it should take into account the considerable uncertainty when considering the estimates of cost effectiveness.

Cost-effectiveness results

The cost-effectiveness estimates were over £30,000 per QALY gained

3.8 The committee noted that there is a confidential patient access scheme for ibrutinib and that some of the standard care therapies included in the standard therapies arm are available to the NHS at a confidential discount. Incremental life years gained with ibrutinib were 2.88 in the revised company base case using a hazard ratio for progression-free survival of 0.25. When higher hazard ratios (ranging from 0.28 to 0.75) were explored, the life years gained with ibrutinib decreased (ranging from 2.80 to 1.78 respectively). The exact incremental costs, quality-adjusted life years (QALYs) and incremental cost-effectiveness ratios (ICERs) associated with these analyses are confidential and cannot be reported here, but the cost-effectiveness estimate in the company base case, and all alternative analyses explored, were considerably higher than £30,000 per QALY gained.

Innovation

Additional health-related benefits not captured in QALY were not identified

3.9 The committee recalled its discussions about the innovative aspect of ibrutinib in the original appraisal. It accepted that the treatment has several benefits including oral administration, manageable adverse reactions and low toxicity. It further noted the particular importance to people of being able to have treatment at home, reducing hospital visits, and the company's statement in response to the appraisal consultation document that the modelling did not capture the psychological benefit of having an effective treatment. The committee concluded that ibrutinib could be considered a step-change in managing Waldenstrom's macroglobulinaemia. But it considered it likely that all the clinical benefits had already been included in the modelling, and did not consider that any additional health-related benefits, that had not been captured fully in the QALY calculation (if present), would be enough to lower the ICER to within the range normally considered cost effective.

Because of the uncertainty, an acceptable ICER would be comfortably below £30,000 per QALY gained

3.10 NICE's guide to the methods of technology appraisal notes that above a most plausible ICER of £20,000 per QALY gained, judgements about the acceptability of a technology as an effective use of NHS resources will take into account the degree of certainty around the ICER. The committee will be more cautious about recommending a technology if it is less certain about the ICERs presented.

The committee noted the high level of uncertainty, specifically:

  • The hazard ratio for progression-free survival for ibrutinib compared with standard therapies estimated from an indirect comparison is uncertain.

  • The revised approach for estimating ibrutinib progression-free survival indirectly from SACT data is highly uncertain.

    The committee recalled statements by clinical and patient experts that ibrutinib is an effective treatment option for Waldenstrom's macroglobulinaemia (see section 3.5). It acknowledged the difficulty in obtaining further evidence for a rare condition like Waldenstrom's macroglobulinaemia and the absence of any further analyses that could resolve the uncertainty in the evidence base, or an alternative approach to modelling. The committee concluded that although there was major uncertainty about the exact extent of the benefit of ibrutinib, it was satisfied that ibrutinib is a highly effective technology. Therefore, it agreed that an acceptable ICER could be over £20,000 per QALY gained. However, it would have to be comfortably below £30,000 per QALY gained, not at the upper limit, to reduce the risk of approving a cost-ineffective treatment for use in the NHS, and the potential opportunity costs, when the true cost effectiveness was uncertain.

Cancer Drugs Fund

Conclusion

  • National Institute for Health and Care Excellence (NICE)