3 Committee discussion
The appraisal committee considered evidence submitted by Celgene, a review of this submission by the evidence review group (ERG), NICE's technical report, and responses from stakeholders. See the committee papers for full details of the evidence.
Most people with newly diagnosed multiple myeloma would have lenalidomide maintenance treatment after an autologous stem cell transplant
3.1 After a first autologous stem cell transplant, newly diagnosed multiple myeloma is usually monitored until the first relapse, and not actively treated. Lenalidomide is the only treatment option licensed as a maintenance therapy to replace monitoring for this indication. Lenalidomide would be used to try to lengthen the time until first relapse. The clinical experts advised that most people with newly diagnosed multiple myeloma who have had a first transplant would have lenalidomide maintenance treatment if it was recommended, and that only a small proportion would have no active treatment. The committee was aware that lenalidomide (plus dexamethasone) is currently available in the NHS for treating multiple myeloma later in the treatment pathway. The clinical experts explained that if people had lenalidomide maintenance treatment they would not then have lenalidomide again later in the treatment pathway. However, the clinical and patient experts emphasised that the benefits of lenalidomide maintenance treatment would likely outweigh the benefits of using it later in the pathway. This is because, with each line of new therapy, a substantial proportion of people stop having treatment because they become too ill or have complications. Therefore, the most effective treatments should be given as early in the treatment pathway as possible. Also, the first remission is often the 'best' remission because it is when people with the condition are at their fittest. Clinical experts also explained that it is also when people have the highest quality of life before the negative effects of the disease and its treatments have accumulated. Therefore, extending the first remission maximises the chances of people maintaining a higher quality of life for the longest possible period. The patient experts also explained that lenalidomide is a well-tolerated treatment and that, during the ongoing coronavirus pandemic, it is particularly convenient. This is because it is taken orally and does not need a hospital visit. The committee concluded that, if lenalidomide maintenance treatment was to be recommended, most people with newly diagnosed multiple myeloma who have had an autologous stem cell transplant would have it.
Lenalidomide is an effective maintenance treatment for people who have had an autologous stem cell transplant
3.3 The main clinical evidence for lenalidomide maintenance treatment came from Myeloma XI, a phase 3 open-label randomised trial based in 110 NHS centres in the UK. A total of 1,971 people with newly diagnosed multiple myeloma were enrolled and stratified by their eligibility for an autologous stem cell transplant (only people eligible for a transplant are relevant for the population in this appraisal). The trial had an adaptive design in which ongoing trial results were used to inform changes in the protocol. Also, there were multiple levels of randomisation in the trial. The company's submission focused on a smaller cohort of 1,032 people from Myeloma XI. These people had had a first transplant and been randomised to have maintenance with lenalidomide 10 mg daily on days 1 to 21 of each 28‑day cycle, or to have monitoring of their disease with no lenalidomide treatment. The company considered this cohort to be directly relevant to this appraisal (when Myeloma XI is mentioned from this point, it is referring to this cohort of interest unless otherwise specified). The primary outcomes were progression-free survival and overall survival, both of which were longer with lenalidomide maintenance treatment than with monitoring. The clinical experts advised that the trial was representative of NHS practice, and that the results were generalisable to the population in this appraisal. Based on the results from Myeloma XI, the committee concluded that lenalidomide is an effective maintenance treatment for newly diagnosed multiple myeloma in people who have had an autologous stem cell transplant.
The company presented evidence from all trials that met the inclusion criteria for its systematic literature review
3.4 The company originally identified 4 trials of lenalidomide maintenance treatment in its systematic literature review: Myeloma XI, CALGB 100104, GIMEMA and IFM 2005‑02. It then applied a subsequent set of criteria to exclude CALGB 100104, GIMEMA, and IFM 2005‑02, leaving only Myeloma XI as a source of clinical-effectiveness evidence in its original submission. The company argued that Myeloma XI was the only trial that reflected the decision problem and UK clinical practice. However, it used both CALGB 100104 and Myeloma XI data to estimate survival in its cost‑effectiveness model. The ERG was of the view that the company's approach was inconsistent. The ERG was also concerned that the subsequent set of criteria used to exclude trials was arbitrary and not prespecified. It considered that IFM 2005‑02 should have been excluded based on the company's original systematic literature review criteria, but that CALGB 100104 and GIMEMA should have been included. The committee agreed that the company's approach was inconsistent and would have preferred the company to present all trials meeting the original systematic literature review criteria. The committee also acknowledged that the cohort of interest from Myeloma XI was likely to provide the most generalisable source of clinical-effectiveness evidence to NHS practice. However, because the marketing authorisation is based on trials with 28‑days of dosing, the committee stated that it needed to see evidence on the clinical effectiveness from CALGB 100104 because it used this dosage. In response to consultation, the company presented detailed methods and results from CALGB 100104 and GIMEMA. At its second meeting, the committee concluded it was satisfied that the company had presented all relevant evidence for lenalidomide maintenance treatment.
The safety profile of lenalidomide as a maintenance treatment compared with monitoring alone is likely to be acceptable
3.5 The company explained that there were no data on adverse events available from Myeloma XI for the monitoring arm of the cohort of interest. The ERG stated that this was an area of uncertainty because between-arm comparisons of adverse event rates were needed to understand the comparative safety profile of lenalidomide maintenance treatment. The company provided adverse-events data from both the lenalidomide and monitoring arms of CALGB 100104. The ERG thought that it was useful as supplementary information but that it was not directly generalisable to the population in the NHS. The clinical experts considered that the rates of adverse events in the lenalidomide arm of Myeloma XI for the cohort of interest were similar to those seen in clinical practice for other indications. A patient expert explained that results from a survey done by Myeloma UK showed that most people having lenalidomide maintenance treatment found it easy to take and tolerated it well. The committee concluded that there was some uncertainty about the risk of adverse events, but the safety profile of lenalidomide as a maintenance treatment compared with monitoring alone is likely to be acceptable.
3.6 The company chose a partitioned survival model comprising 3 health states (pre-progression, progressive disease and death). It explained that it had previously considered a more complex model structure such as a multistate model. However, there were not enough data to estimate transition probabilities for this approach, so it chose a partitioned survival model instead. The ERG stated that the simple structure of the company's model did not allow uncertainty in the model to be fully explored. It was particularly concerned about the effect of subsequent treatments. This was because survival in the company's model was based on Myeloma XI and CALGB 100104, and the treatments given at second line and beyond in these trials are not generalisable to current NHS practice. The treatment of myeloma has changed since Myeloma XI was started, which means that, despite it being a UK trial, the treatments used do not reflect current NHS practice. Also, CALGB 100104 has limited generalisability to the UK because it was based in the US. The ERG highlighted that the company's partitioned survival model structure did not allow alternative assumptions about subsequent treatments to be explored. This meant that the modelled survival may not have been representative of what would be seen in the NHS. The committee concluded that the company's model structure had limitations. It also concluded that there was likely to be uncertainty around the cost-effectiveness estimate because assumptions about the effects of subsequent therapies on survival could not be fully explored.
The rank-preserving structural-failure time method is appropriate to adjust for treatment switching in the CALGB 100104 trial
3.7 In the CALGB 100104 trial, people were offered the option to switch from placebo to lenalidomide if their disease had not yet progressed. In the committee's first meeting, the company explained that it used the rank-preserving structural-failure time method to adjust for treatment switching in CALGB 100104 and that it did not explore any alternative approaches. The committee recognised that different treatment switching adjustment methods were available and was disappointed the company did not provide any justification for using its chosen method. In response to consultation, the company explored several alternative methods, including the inverse probability of censoring weights and 2‑stage methods. After assessing the key assumptions and limitations associated with each approach, the company concluded that the rank-preserving structural-failure time method remained the most appropriate. The ERG was generally satisfied with the company's rationale. The committee was concerned that some people in CALGB 100104 could have multiple lines of lenalidomide, which is not an option in current NHS practice. The company clarified that it had not adjusted for this in its survival analysis. However, clinical experts explained that lenalidomide is not given more than once in the pathway. This is because it is now acknowledged that it is not likely to be effective if the disease has previously stopped responding to treatment. Therefore, even if it were given multiple times, this is unlikely to positively bias estimates of overall survival in the lenalidomide arm of the CALGB 100104 trial. The committee concluded that the company's use of the rank-preserving structural-failure time method to adjust for treatment switching in the CALGB 100104 trial had some limitations, but was appropriate.
Survival extrapolations should use Myeloma XI as the main evidence source but supplemented with CALGB 100104 data adjusted to reflect Myeloma XI
3.8 Survival models were needed to predict survival beyond the end of the clinical trials for lenalidomide maintenance treatment. The company and ERG had different preferred approaches to using trial data to extrapolate survival:
The company preferred to use data from both Myeloma XI and CALGB 100104 because CALGB 100104 provided longer-term data. It also mentioned that, despite heterogeneity between the trials, the survival results were very similar.
The ERG preferred to use Myeloma XI data only, because of key differences between the 2 trials, such as dosing, baseline characteristics and subsequent treatments.
In its original base case, the company fitted survival curves to Myeloma XI data and used CALGB 100104 data to help with curve selection. However, in response to technical engagement, it pooled data from Myeloma XI and CALGB 100104, and fitted curves to the pooled data. It confirmed that it used a simple method for pooling the individual patient data from the trials. This did not involve adjusting CALGB 100104 data to reflect Myeloma XI, for example by adjusting for differences in trial design or population. The ERG noted that it was unable to validate the company's methods for pooling data because not enough detail was provided. The committee was disappointed that the company's approach could not be scrutinised and validated based on the information provided and considered that the differences between the trials meant a simple pooling approach may have been inappropriate. In particular, the 28‑day dosing regimen in CALGB 100104 meant survival in the model was based on a dosage that the company, ERG, and patient and clinical experts explained would not be given in NHS practice. The committee recognised that CALGB 100104 had a longer median follow up (91 months) than Myeloma XI (31 months), so provided information about longer-term survival. At its first meeting, the committee concluded that it would prefer to see a survival analysis that used Myeloma XI as the main source of evidence. CALGB 100104 could be used to help extrapolation, with data adjusted to reflect the Myeloma XI population as far as possible and based on the underlying survival of patients in Myeloma XI. In response to consultation, the company presented survival analyses based on Myeloma XI data up to 60 months, followed by adjusted CALGB 100104 data, using propensity score weighting to adjust CALGB 100104 to better reflect Myeloma XI in its base case. The ERG was satisfied that propensity score weighting was an appropriate method and that the company's analysis was generally well conducted. However, it also highlighted that an important limitation of the propensity score weighting approach is that it cannot adjust for the difference in doses between the 2 trials. The committee concluded that the company had appropriately used the committee's preferred approach to extrapolate survival in its updated analyses (that is, Myeloma XI used as the main source of evidence, with CALGB 100104 used to inform longer-term extrapolation). It further concluded that, despite important limitations associated with the propensity score weighting approach, the company had appropriately adjusted CALGB 100104 data to better reflect Myeloma XI.
The company's method for selecting overall-survival curves based on adjusted CALGB 100104 data has limitations
3.9 In response to technical engagement, the company updated its base case to use a joint Weibull model to extrapolate survival based on pooled Myeloma XI and unadjusted CALGB 100104 data. The ERG preferred to use a joint log-logistic model and to extrapolate Myeloma XI data only. In its first meeting, the committee concluded that survival extrapolations should use Myeloma XI as the main source of evidence but could be supplemented with adjusted CALGB 100104 data for longer‑term survival (see section 3.8). Therefore, in response to consultation, the company revised its base case to use Myeloma XI data in the short term (up to 60 months), followed by adjusted CALGB 100104 data thereafter. However, the committee noted that when choosing the best fitting curve to extrapolate survival, the company had based its choice on adjusted CALGB 100104 data only, rather than all of the data in the model (that is, Myeloma XI data to 60 months followed by adjusted CALGB 100104 data for the remainder of the model time horizon). The company also reported that it selected the joint gamma distribution because it was the best fit to the adjusted CALGB 100104 data, yet it had used the joint generalised gamma in its model. The ERG highlighted several limitations with the company's approach. It would have preferred the company to have used the combined Myeloma XI and adjusted CALGB 100104 data as a basis for curve selection, rather than adjusted CALGB 100104 data alone. It also suggested that the company could have explored a piecewise approach with 2 different distributions for the initial Myeloma XI period (to 60 months) and adjusted CALGB period (after 60 months). Furthermore, the ERG questioned why the company had used the generalised gamma distribution instead of the gamma in its base case, as the company had not provided any supporting information or rationale for its selection. In its own analysis, the ERG chose the joint log‑logistic model for overall survival. After examining the company's and ERG's overall-survival curves, the clinical experts explained that the joint log‑logistic was likely the best representation of long-term survival in clinical practice. The committee concluded that the company's method for selecting overall‑survival curves based on CALGB 100104 data (adjusted to reflect Myeloma XI) has limitations and that it would have preferred the company to have explored a piecewise approach. It also concluded that the log‑logistic curve is the most appropriate choice for extrapolating overall survival.
Treatment waning should be included in the model, and 10 years may be a conservative estimate of when the treatment effect starts to wane
3.10 Treatment waning refers to whether or not the relative treatment effect between lenalidomide and monitoring of the condition is likely to reduce over time after people stop taking lenalidomide. Not including treatment waning in the model implies that the relative treatment effect stays the same and lenalidomide remains more effective than monitoring for the entire modelled time horizon, even if people are no longer on treatment. Based on its survival curves, the company took the view that there was no evidence of a treatment-waning effect with lenalidomide and did not include waning in its original base case. The ERG did not include a treatment-waning effect in its base case but explained that there were no long-term data to rule out the possibility that the relative treatment effect decreases over time. The ERG therefore did a scenario analysis that looked at different treatment-waning scenarios. It found the cost‑effectiveness estimate to be sensitive to assumptions about how long the treatment effect lasts for. At the committee's first meeting, the clinical experts advised that they would not expect lenalidomide to have a continued effect after people had stopped taking it. Based on this, the committee had agreed that the treatment effect of lenalidomide therapy may wane over time and that this should have been included in the company's model. In response to consultation, the company reiterated that there was no evidence of a treatment-waning effect and so did not include it in its base case. Instead, it did a scenario analysis in which it assumed lenalidomide loses efficacy at 10 years, which it stated was a conservative assumption aligned with available evidence from the CALGB 100104 follow-up period. At the committee's second meeting, the clinical experts suggested that people can stay on lenalidomide for a long time so the lasting treatment effect in the trials may happen because people are still on treatment, rather than because of a lasting treatment effect after people stop taking lenalidomide. They further explained that if people stop treatment, they are unlikely to progress immediately, so there is likely to be some lasting effect that will eventually disappear. The committee considered that it is unclear when the treatment effect of lenalidomide maintenance may start to wane, but that it is likely to be between 10 to 25 years. It concluded that treatment effect waning should be included in the model, and that 10 years may be a conservative estimate of when the treatment effect starts to wane.
3.11 The company's model included the costs of second- and third-line treatments given after maintenance treatment. The committee was aware that the subsequent therapies used in Myeloma XI are no longer generalisable to NHS practice. The clinical experts explained that most people who have had a first autologous stem cell transplant will go on to have a treatment recommended in the Cancer Drugs Fund at a later line of therapy. However, the NICE Cancer Drugs Fund position statement specifies that companies should not include treatments recommended for use in the Cancer Drugs Fund as treatment-sequence products in their economic modelling. This is because they do not yet reflect routine NHS practice. The committee acknowledged that this made it difficult to develop assumptions about subsequent therapies in the model, and that any assumptions were hypothetical and highly uncertain. In its first meeting, the committee had therefore concluded that modelled subsequent treatments should reflect as closely as possible treatments that are currently given in NHS practice, and what would be given in the absence of Cancer Drugs Fund treatments. In response to consultation, the committee and the ERG had provided several exploratory scenarios, which the committee went on to discuss (see section 3.12 and section 3.13).
Most people whose condition was monitored after their first transplant would have lenalidomide plus dexamethasone after relapse if treatments in the Cancer Drugs Fund are not available
3.12 At the committee's first meeting, the Cancer Drugs Fund clinical lead estimated that, if treatments recommended for use in the Cancer Drugs Fund were not available, about half of people whose condition was monitored after their first transplant would then have lenalidomide plus dexamethasone after their first relapse. The clinical experts agreed with this estimate. In its original base case, the company estimated this figure to be 15%. In response to consultation, the company revised its subsequent treatment assumptions, but did not substantially adjust the proportion of people in the observation arm having lenalidomide second line. The company explained that its revised scenarios were based on the subsequent therapies given in Myeloma XI and CALGB 100104 and were therefore aligned with the efficacy data used in the model. However, the committee thought that the numbers of people having lenalidomide plus dexamethasone after their first relapse remained too low in the company's revised assumptions. At the second meeting, the Cancer Drugs Fund clinical lead highlighted that even more people than usual are currently having lenalidomide plus dexamethasone after their first relapse. This is because the alternative treatment in the Cancer Drugs Fund (daratumumab with bortezomib and dexamethasone) is administered in hospital, and people have been reluctant to attend hospital during the ongoing coronavirus pandemic. The committee concluded that at least half of people whose condition was monitored after their first transplant would likely have lenalidomide plus dexamethasone after their first relapse if treatments in the Cancer Drugs Fund were not available, and that this should be reflected in the model.
The number of people having a second autologous stem cell transplant is decreasing as alternative treatment options become available
3.13 The committee discussed whether a second autologous stem cell transplant may be an option for some people after a first relapse following their first transplant. The company explained that second transplants would be highly unlikely in clinical practice, while the ERG stated that they are a relevant option. The clinical experts estimated that about 5% to 10% of people get a second transplant, although 1 expert thought this number could be as high as 20%. The clinical and patient experts agreed that the availability of effective treatments in the Cancer Drugs Fund has led to decreasing rates of second transplants. They thought that these rates would decrease more if lenalidomide maintenance treatment was recommended. The committee concluded that about 5% to 10% of people currently get a second autologous stem cell transplant and that this should be reflected in the model, but these numbers are likely to fall in the future as alternative treatment options become available.
3.14 Relative dose intensity is the percentage of the prescribed dose of lenalidomide that people take. Assumptions about the relative dose intensity could affect the cost-effectiveness estimate because it shows how much of the total cost of a prescribed drug is incurred (with a lower relative dose intensity meaning lower accrued drug costs). The company used individual patient data from Myeloma XI to estimate the relative dose intensity for lenalidomide maintenance treatment to be 86% to 87% depending on whether people were prescribed 5 mg or 10 mg in the trial. The ERG's opinion was that the company's relative dose-intensity estimate was too low, so the cost-effectiveness estimate was optimistic. It noted that the company's relative dose-intensity estimate from Myeloma XI was lower than in TOURMALINE‑MM1 (TMM1). This trial was identified by the ERG and was in people with relapsed or refractory multiple myeloma, so was not directly relevant to the appraisal. The ERG explained that TMM1 had used a higher lenalidomide dose of 25 mg daily on days 1 to 21 of each 28‑day cycle. It argued that the lower relative dose-intensity estimate from Myeloma XI compared with TMM1 was counterintuitive because people taking a higher dose would be expected to have more safety and tolerability issues, so would be less likely to maintain the target dose. The ERG used the relative dose intensity of 94.9% from TMM1 in its original base case. The ERG also considered that the company did not provide enough clear information to allow for its relative dose-intensity calculation to be validated. The committee was aware of the higher relative dose-intensity estimate from TMM1 compared with Myeloma XI. However, it decided that Myeloma XI was a better source of information because it was directly relevant to the decision problem and was based in the UK. Conversely, TMM1 included people with relapsed and refractory multiple myeloma and was international. The committee was satisfied with the company's decision to use Myeloma XI to estimate relative dose intensity, but considered that the company should have provided the full methods it used to determine this so that the ERG could validate it. In response to consultation, the company provided more detailed methods and explained that it based its calculations on prescribing data from Myeloma XI, accounting for reductions in dose and changes to the dosing frequency or treatment cycle length that were allowed in the trial. The ERG had outstanding concerns about the company's methods because it could still not interpret or recalculate the company's estimates. It considered the relative dose intensity value used in the company's model to be too low based on conversations with clinical experts. At consultation, the ERG provided an alternative estimate of 92% based on a simplified approach in which it calculated the average dose using the number of 5 mg and 10 mg treatment cycles in Myeloma XI. The ERG clarified that this method cannot account for changes to cycle length or other types of dose adjustment. The committee concluded that there were limitations associated with both the company's and ERG's approaches and the relative dose intensity remained uncertain. There was further uncertainty with this value because there may be patient-specific dose reductions in NHS practice, such as extending the length of the treatment cycle, which may be given to extend the maintenance phase (see section 3.2). However, in light of the uncertainty, it was reasonable to assume a value somewhere in between the company's and the ERG's estimates.
Maintenance therapy with lenalidomide is likely to be a cost-effective use of NHS resources when given on days 1 to 21 of each 28-day cycle
3.15 The committee went on to discuss the company and ERG base cases, and agreed that the scenario that best reflected its preferences incorporated the following assumptions:
survival estimates in the economic model based on Myeloma XI data to 60 months, followed by propensity-score weighted CALGB 100104 data thereafter (see section 3.8)
log-logistic distribution to extrapolate overall survival (see section 3.9)
waning of the treatment effect of lenalidomide applied at between 10 and 25 years (see section 3.10)
5% to 10% of people having a second autologous stem cell transplant (see section 3.13)
50% of people in the observation arm having lenalidomide plus dexamethasone after first relapse (see section 3.12)
relative dose-intensity value falling between the company's and ERG's estimates (see section 3.14).
The committee was presented with 2 different dosing schedules for these scenarios; the schedule of once daily lenalidomide on days 1 to 21 of repeated 28‑day cycles (which the committee had concluded best reflected the dose used in NHS practice) or once daily lenalidomide continuously on days 1 to 28 of repeated 28‑day cycles (which was the dose as recommended in the marketing authorisation, see section 2.2). The committee was aware that NICE's guide to the methods of technology appraisal states that the committee 'does not normally make recommendations regarding the use of a drug outside the terms of its marketing authorisation'. The committee first considered incremental cost-effectiveness ratios (ICERs) for lenalidomide as per the licensed schedule of 28 days of dosing per 28‑day cycle. However, the committee noted that the ICER for this scenario only adjusted the cost of treatment up from 21 to 28 days, while the effectiveness, time-on-treatment, relative dose intensity, medical resource use and adverse events were all the same as the scenario using 21 days of dosing. Furthermore, the committee considered that 28 days of dosing was highly unlikely to be used in NHS practice. The committee noted that NICE's guide to the methods of technology appraisal also states that evidence relating to using the technology under appraisal outside the terms of its marketing authorisation may inform deliberations. The committee therefore agreed that it was appropriate to consider the lower costs of lenalidomide administration that would arise when using 21 day rather than 28 day dosing in NHS practice. When taking this into account, the ICERs were within a range normally considered to be a cost-effective use of NHS resources (below £30,000 per quality-adjusted life year [QALY] gained). Because of confidential discounts for treatments used in the model, ICERs are confidential so cannot be reported here.
There is no evidence to suggest any additional benefits not adequately captured by the QALY and no equalities issues
3.16 The committee was aware that there is currently no active maintenance treatment for newly diagnosed multiple myeloma in adults who have had an autologous stem cell transplant in the UK, which represents a gap in NHS practice. However, it saw no evidence to suggest any additional benefits not adequately captured by the QALY. No equality or social value judgement issues were identified.