3 Committee discussion
The appraisal committee (section 5) considered evidence submitted by Bristol-Myers Squibb and a review of this submission by the evidence review group (ERG). See the committee papers for full details of the evidence.
People with untreated intermediate- or poor-risk renal cell carcinoma would welcome a new treatment option
3.1 For intermediate- or poor-risk advanced renal cell carcinoma, tyrosine kinase inhibitors such as pazopanib, sunitinib, tivozanib and cabozantinib are current standard care in the NHS. They can cause adverse effects such as fatigue, hand and foot syndrome, and chronic diarrhoea, which can substantially affect quality of life. The committee agreed that people with intermediate- or poor-risk advanced renal cell carcinoma would welcome a new treatment option.
Prognostic risk scores are not routinely used in UK clinical practice, but there are no barriers to their use
3.2 Nivolumab with ipilimumab is indicated for treating intermediate- and poor-risk advanced renal cell carcinoma. The clinical experts stated that prognostic scores to define intermediate‑ and poor‑risk are not used in clinical practice. They noted that the 2 best known risk scores are the International Metastatic Renal Cell Carcinoma Database Consortium (IMDC) risk score and the Memorial Sloan-Kettering Cancer Center (MSKCC) risk score. The clinical experts considered that the 2 scores were similar, but would prefer to use the IMDC risk score because it was used in the clinical trial providing evidence for this appraisal, CheckMate 214. They stated that clinicians routinely collect all components of the risk scores, and could start using them.
3.3 People with untreated advanced renal cell carcinoma could be offered 1 of 4 oral tyrosine kinase inhibitors, as recommended in NICE's technology appraisal guidance on cabozantinib, pazopanib, sunitinib or tivozanib. Cabozantinib and tivozanib were not included in the scope of this appraisal because they were not part of NHS clinical practice at the start of the appraisal. The clinical experts stated that, in practice, sunitinib and pazopanib are considered clinically equivalent. The committee recalled that, in previous appraisals, it considered sunitinib and pazopanib to be clinically equivalent, and there was no new evidence to change this conclusion. The committee concluded that pazopanib and sunitinib are the relevant comparators in this appraisal, and can be considered clinically equivalent.
3.4 The main evidence for nivolumab with ipilimumab came from CheckMate 214, an open-label, randomised controlled trial, with sunitinib as the comparator. The co-primary end points of the trial were overall survival and progression-free survival, amended in the protocol by the company to include overall response rate. In its original submission to NICE, the company presented data from an interim analysis based on a data cut in August 2017, reflecting a median follow-up of 25 months. In response to consultation, the company provided updated results in confidence from a further analysis (dated August 2018). The trial stratified people by prognostic risk score, as defined by the IMDC scoring system, and then randomised people equally to have either nivolumab with ipilimumab or sunitinib. The trial recruited:
180 people with poor-risk untreated renal cell carcinoma
667 people with intermediate-risk untreated renal cell carcinoma
249 people with favourable-risk untreated renal cell carcinoma (not included in this appraisal).
The company focused its submission to NICE on the subgroup of patients whose prognostic risk score was intermediate or poor because it expected that this would align with the anticipated marketing authorisation of nivolumab with ipilimumab. The company stated that although the trial included patients with favourable-risk disease, the trial was powered to investigate clinical outcomes in a combined intermediate- or poor-risk group. The committee concluded that the combined intermediate- or poor-risk group is appropriate for decision making.
3.5 The ERG considered that the baseline characteristics generally reflected NHS clinical practice. However, it highlighted that, based on expert opinion it had received, it would expect a larger proportion of people in clinical practice to have poor-risk renal cell carcinoma. The Cancer Drugs Fund clinical lead stated that the proportion of people with intermediate- and poor-risk disease who will have treatment in clinical practice was uncertain, but there was no evidence to suggest that it differed in CheckMate 214. The committee recognised that patients with intermediate- and poor-risk disease have different prognoses, so the absolute treatment effect in the combined group will partly depend on the distribution of risk scores. It did not see data on the proportion of people with intermediate- and poor-risk disease in clinical practice and agreed that this was an area of uncertainty. The clinical experts stated that the patients in the trial generally reflected people who are expected to have nivolumab with ipilimumab in clinical practice. However, they noted that people recruited to clinical trials are sometimes younger, in better health and able to tolerate a short wait before treatment begins. The clinical experts would therefore expect people in clinical practice to have poorer health and a less favourable prognosis than those in CheckMate 214. The committee considered this to be a possibility but, in the absence of evidence for this, concluded that the results of CheckMate 214 were generalisable to clinical practice in England.
3.6 CheckMate 214 used 2 definitions to measure progression-free survival. The primary definition included patients up to the first point when their disease progressed, or they died (patients going onto other treatments before disease progression were censored). The secondary definition included patients up to the first point when they had another treatment before disease progression, their disease progressed, or they died. The ERG considered the secondary definition to be more appropriate because removing people who have another treatment after either nivolumab with ipilimumab or sunitinib but before progression represents a form of 'informative censoring'. That is, the number of people who need subsequent treatment before progression may systematically differ between the 2 treatment arms. The committee considered that patients may have had second therapies before progression because they could not tolerate the study treatment. It agreed this should be captured in the estimates of progression-free survival. The committee concluded that the secondary definition of progression-free survival was the most appropriate for decision making. In response to consultation, the company incorporated this definition into its analysis.
Nivolumab with ipilimumab is more clinically effective than sunitinib, but its effect in the long term is yet to be established
3.7 In an interim analysis (August 2017), nivolumab with ipilimumab improved the progression-free survival and overall survival of people with intermediate- or poor-risk renal cell carcinoma compared with sunitinib:
Median progression-free survival (secondary definition) was 11.0 months (95% confidence interval [CI] 8.3 to 15.2) for nivolumab with ipilimumab and 8.3 months (95% CI 7.0 to 9.8) for sunitinib, with a hazard ratio of 0.76 (99.1% CI 0.60 to 0.95).
Median overall survival was not reached (95% CI 28.2 to not evaluable) for nivolumab with ipilimumab and was 26.0 months (95% CI 22.1 to not evaluable) for sunitinib, with a hazard ratio of 0.63 (99.8% CI 0.44 to 0.89).
Based on the strength of the survival benefit shown, CheckMate 214 was stopped early. The committee agreed that the benefit was substantial but was aware that trials stopped early for benefit can overestimate the benefit. It noted that the data were immature, with a median follow-up of 25.2 months, and only 32.9% of people randomised to nivolumab with ipilimumab having died at the time of analysis. It also noted that this was fewer than intended in the statistical analysis plan (50%). The committee was aware that, in the subsequent interim analysis (August 2018), the company had amended the trial protocol to allow people randomised to sunitinib to switch to nivolumab with ipilimumab when their disease progressed, and that the company did not adjust for this. The committee concluded that the later data cut reduced the uncertainty compared with the previous one, that the cross-over likely biased the hazard ratio towards the null, and that the data remained too immature to establish the long-term effect of treatment.
3.8 The company did an indirect treatment comparison of clinical effectiveness between nivolumab with ipilimumab and pazopanib. The committee recalled that pazopanib and sunitinib can be considered equally clinically effective (see section 3.3). It concluded that an indirect treatment comparison was not needed and did not consider it further.
3.9 The clinical experts explained that in their experience, using nivolumab with ipilimumab is well tolerated and has a preferable adverse event profile compared with tyrosine kinase inhibitors. The committee acknowledged that nivolumab and ipilimumab are associated with some rare but unpleasant, and potentially serious adverse events that are specific to immunotherapy. The clinical experts stated that clinicians are experienced in recognising and managing these serious adverse events. The committee concluded that nivolumab with ipilimumab is well tolerated compared with the alternative tyrosine kinase inhibitors.
3.10 The company considered in its base case that because of the immunomodulatory mechanism of nivolumab, a number of people who have nivolumab would return to a death rate equal to that of the general population. This means that some people who have nivolumab with ipilimumab are effectively 'cured'. The committee noted multiple concerns with this assumption:
The company based the probability of having an immunological effect on durable response, which it defined as the number of people whose disease achieved a complete or partial response at the time of the August 2017 data cut of CheckMate 214. The committee was aware that durable response was not defined in the protocol, and that the company defined it post hoc (at the analysis stage of the trial), irrespective of how long the disease responded. This meant that the number of people who had a durable response depended on how the company defined both durable and response. The committee heard that durable response was not used clinically. It also noted that the company had not presented validated evidence associating durable response and overall survival in untreated renal cell carcinoma.
The company assumed that 50% of people with a durable response (30% of CheckMate 214 population) would expect to be cured, amounting to 15% of those randomised to nivolumab with ipilimumab (that is, 50% of the 30% with durable response). The Cancer Drugs Fund clinical lead noted that clinicians would expect 20% of those treated with nivolumab and ipilimumab to have an immunological effect, although it is not known how this would translate into life expectancy. The committee considered the company's assumption that 15% of people would be cured was implausible because fewer than 15% of patients were still on treatment at the end of follow-up. This suggests that patients' disease had progressed, or patients could no longer tolerate treatment. The committee acknowledged the company's comment that it could also mean that patients stopped treatment because of sustained benefit. However, the committee considered this unproven because the trial did not stipulate a stopping rule.
The company assumed that patients who were cured dropped abruptly to the mortality rate of the general population after 9 years of treatment. The Cancer Drugs Fund clinical lead expected that patients who have had long-term immunotherapy would not live as long on average as people without advanced renal cell cancer because of the cumulative effect of the disease on health. The committee agreed that an abrupt drop to the death rate of the general population for several people whose disease achieved a complete or partial response at the time of the August 2017 data cut at 9 years seemed implausible. It suggested that a more reasonable assumption would be to apply standardised mortality ratios to general population mortality rates to account for the increased mortality expected in people with renal cell carcinoma after prolonged immunotherapy.
The committee agreed that the company's modelling of an immunotherapeutic effect relied on speculative assumptions that were not substantiated by evidence. It was aware that, with other cancers (such as malignant melanoma), some people who have nivolumab experience an extended survival benefit. However, it agreed that that it could not generalise the size of this effect from 1 cancer to another. The committee concluded that clinical experience suggests that some patients with renal cell cancer on nivolumab monotherapy might live relatively long lives. However, it noted that there was no robust evidence on the size of the association between a clinically meaningful definition of response and long-term survival for nivolumab with ipilimumab.
3.11 To model the immunological effect in its base case, the company assumed that half of patients who were offered nivolumab with ipilimumab died at a rate informed by projecting the rate of death beyond August 2017, and the other half abruptly assumed a death rate to match general population at about 9 years. This was because the model assumed all people who had a 'durable response' at the end of the trial were also the people that lived beyond 9 years. The ERG considered this method to be inappropriate. It noted that there was no structural link in the model between an immunological effect and progression-free survival. It would have preferred to see a recognised mixture-cure model or a response-based model linking progression-free survival to a 'cure' effect and overall survival. The committee concluded that, not only were the underlying assumptions untenable, but the modelling of the immunotherapy effect was also flawed.
3.12 The company assumed that, after 5 years of treatment, people stop taking nivolumab with ipilimumab, even if their disease had not progressed and they continued to benefit from treatment. The committee discussed the following issues about the modelling of a stopping rule:
The committee understood that, for immunotherapies for some other indications, NICE guidance has included a recommendation to stop treatment after a defined period of time. The Cancer Drugs Fund clinical lead explained that a stopping rule reflects concerns about the toxicities of immunotherapies; he was confident that patients and clinicians would accept a stopping rule and that the NHS could implement it. The committee was aware that, in contrast with some other immunotherapies, neither the marketing authorisation nor the main trial (CheckMate 214) includes a stopping rule.
The company did not explore the effect of a stopping rule on clinical outcomes. It assumed that people who stop treatment after 5 years continue to benefit from treatment as if they had never stopped it. However, the company also assumed that patients no longer incurred treatment costs. The committee did not see evidence that the effect of nivolumab with ipilimumab would continue after stopping treatment. It considered that, if the treatment effect did not continue, a stopping rule may result in a lower effect in people whose disease would otherwise continue to respond.
The committee appreciated that the company did not take into account stopping treatment when it modelled overall survival, in which it incorporated an immunological effect for patients 9 years after starting treatment (that is, 4 years after treatment stopped). The company modelled time-to-stopping treatment structurally independently from the proportion of people who had an immunological response. The committee considered that applying a stopping rule to the estimated time-to-stopping treatment underestimated the costs of treatment for patients having an immunological response.
The committee concluded that it is not appropriate to include a stopping rule for decision making because its effect on clinical outcomes are untested.
3.13 The company used progression-free survival assessed by an independent radiology review committee (IRRC). The ERG preferred to use progression-free survival assessed by trial investigators because drug costs in clinical practice would reflect decisions made by the clinician. The committee noted investigator-assessed progression-free survival was not the primary outcome of CheckMate 214 which, being unblinded, may have introduced bias. The committee concluded that the IRRC-assessed progression-free survival results, which included people who had subsequent treatment before progression (see section 3.6), should be used in the economic model.
The costs of treatment should reflect the new dosing regimen and a maintenance dose for nivolumab of 480 mg every 4 weeks
3.14 At the committee's first meeting, costings for nivolumab reflected a weight-based dosing regimen, as used in the CheckMate 214 trial. By the committee's second meeting, regulators had changed the dosing regimen to a flat dose of nivolumab given less frequently. The company presented costings based on this new dosing regimen. The committee discussed whether the weight-based and flat-dosing regimens would be equally effective. The clinical lead for the Cancer Drugs Fund explained that the 2 dosing regimens were likely to be associated with similar effects. The committee also noted that the flat-dosing regimen allows 2 maintenance dosages for nivolumab: 480 mg every 4 weeks or 240 mg every 2 weeks. The company presented its revised analysis based on maintenance dosing every 4 weeks. It argued that all patients are expected to have this schedule, and that this would save administration costs compared with the original weight-based dosing schedule every 2 weeks. The Cancer Drugs Fund clinical lead agreed that clinicians assess response and adverse effects of treatment every 2 weeks during the induction phase, and that frequent monitoring during the maintenance phase would not be necessary, so 4 weekly maintenance dosing is appropriate. The committee concluded that it was appropriate to use the new standard flat-dosing regimen, including the 480 mg every 4 weeks maintenance dosage, when calculating the costs of treatment.
3.15 The company assumed that people who take pazopanib stay on treatment slightly longer than those who take sunitinib and so incur costs for a longer period. The committee recalled that pazopanib and sunitinib are considered clinically equivalent. It concluded that the time-to-stopping treatment for pazopanib and sunitinib should also be considered the same.
3.16 The company used a regression model to estimate quality of life using utility values from data on EQ‑5D‑3L (EuroQol 5 dimensions, 3 levels) collected in CheckMate 214. Its preferred model assumed that people would have different utility values depending on their treatment arm and whether they were on treatment. The ERG considered that patients' utility values would also depend on whether their disease had progressed. The committee considered it likely that disease progression would worsen quality of life. It concluded that the company should include progression status in its regression model. In response to consultation, the company provided an updated regression model including progression status and treatment status (on or off). The ERG explained that progression status and treatment status may correlate; however, the model did not include patients whose disease had progressed on treatment. The committee agreed that the cost-effectiveness estimates were unlikely to be sensitive to the utility values used in the model, but that a model should include estimates of quality of life reflecting whether disease has progressed.
3.17 The committee considered the treatments used after either nivolumab with ipilimumab or after sunitinib in CheckMate 214. It concluded that they do not reflect treatments used in NHS clinical practice. The committee recalled that the company had not adjusted the model to reflect treatment switching from sunitinib as the first treatment to nivolumab with ipilimumab as the second treatment. The company relied on clinical opinion to reflect the expected costs of treatment offered in NHS clinical practice. The ERG preferred to use the distribution of treatments used as second treatment options and beyond seen in CheckMate 214 because they are linked to the trial's clinical outcomes. The committee would have preferred to see results from an analysis that included both the costs and the clinical benefits of treatments used as second treatment options and beyond in NHS clinical practice, but had not been presented with this. It concluded that, because all the analyses included the clinical benefits of treatments second line and beyond in CheckMate 214, it preferred to include the costs of those treatments. In response to consultation, the company included the distribution of subsequent therapies seen in CheckMate 214, but did not adjust for treatment switching.
3.18 To estimate parameters beyond the end of the trial, the company extrapolated survival outcomes (based on the August 2018 data cut) by fitting parametric curves to the observed data (a log-normal distribution for overall survival, cubic spline for progression-free survival, and gamma for time-to-stopping treatment). To extrapolate overall survival, progression-free survival and time-to-stopping treatment, the ERG considered it appropriate to use a piecewise model. This used Kaplan–Meier data directly from the trial followed by an exponential curve from the point where the cumulative hazard plots showed a constant hazard rate. It did this because later portions of the cumulative hazard plots of all 3 showed an exponential trend. The committee questioned whether using the Kaplan–Meier data, followed by an exponential curve, was appropriate for extrapolating the data for progression-free survival if people were cured. It noted that using the company-preferred cubic spline extrapolation curve for progression-free survival and the gamma curve for time-stopping treatment minimally affected the estimates of cost effectiveness compared with the overall survival extrapolation. For overall survival the committee appreciated that a log-normal distribution resulted in survival curves that predicted that a small proportion of patients not explicitly modelled as having been cured would effectively be cured. This was because the log-normal hazard rates meet general population mortality rates at about 20 years. Therefore, the committee considered that, if nivolumab with ipilimumab cures people with advanced renal cell cancer, then this may justify using a statistical distribution with a long-tail. In the absence of evidence on the long-term immunological effect, the committee could not determine which curve was more appropriate. In response to consultation, the company provided an updated version of the ERG's analysis for overall survival, however the company had not used the data from patients randomised to sunitinib who remained in the trial the longest when fitting the sunitinib survival curve. The committee considered that this may have underestimated long-term survival in the sunitinib group. The committee considered both the log-normal, and Kaplan–Meier with exponential extrapolation, curves clinically plausible, concluding that it would take both into account in its decision making.
3.19 The cost-effectiveness results are commercial in confidence because they include the confidential treatment costs of therapies used as second treatment options and beyond, and cannot be reported here. The committee noted that, given the immaturity of the data, there was substantial clinical uncertainty about the long-term effectiveness of nivolumab with ipilimumab. It therefore considered that for nivolumab with ipilimumab to be recommended for routine commissioning in the NHS, the incremental cost-effectiveness ratio (ICER) should be towards the lower end of £20,000 to £30,000 per quality-adjusted life year (QALY) gained. The committee did not consider that the company's cost-effectiveness estimates for nivolumab with ipilimumab compared with sunitinib or pazopanib reflected the true cost effectiveness for the reasons previously outlined (see sections 3.10 to 3.18). The ERG's estimated ICER was between £35,000 and £40,000 per QALY gained compared with sunitinib or pazopanib based on the following changes to the company's base case:
removing the benefit from any immunological effect (see section 3.10)
removing the 5‑year stopping rule (see section 3.12)
assuming time-to-stopping treatment for pazopanib is equivalent for sunitinib (see section 3.15)
including progression status in the utility regression model (see section 3.16)
using Kaplan–Meier followed by an exponential curve to extrapolate overall survival updated to the 2018 data cut (see section 3.18).
The committee agreed with most of the ERG's changes, including removing the immunological effect and stopping rules. Using the log-normal curve to extrapolate overall survival, instead of the Kaplan–Meier data followed by an exponential curve, lowered the ICER, but it was still higher than the lower end of the £20,000 to £30,000 per QALY gained range. The committee reiterated that, although some patients may live for a long time, this was subject to considerable uncertainty in the absence of more mature data. It concluded that nivolumab with ipilimumab did not meet the criteria to be recommended for routine commissioning in the NHS.
Life expectancy for people in the combined intermediate- and poor-risk disease group is likely to be more than 24 months
3.20 The committee considered the advice about life-extending treatments for people with a short life expectancy described in NICE's Cancer Drugs Fund technology appraisal process and methods addendum. The committee preferred mean estimates, rather than median estimates, when considering the end-of-life criteria, noting:
the median overall survival in the sunitinib arm of CheckMate 214 was 25.9 months
using the committee's preferred assumptions, the ERG's model estimated a mean of more than 36.0 months for the sunitinib arm.
The committee agreed that, for the combined poor‑ and intermediate‑risk group, there was no robust evidence that average life expectancy was less than 24 months. The committee concluded that nivolumab with ipilimumab did not meet the criterion for short life expectancy in the combined intermediate- and poor-risk group.
Nivolumab with ipilimumab extends life by more than 3 months, but does not meet end-of-life criteria
3.21 The ERG's revised economic model estimated that nivolumab with ipilimumab increased the life years gained by more than 20 months. The committee concluded that nivolumab and ipilimumab likely extends life by more than an average of 3 months. However, because the disease is not associated with a short life expectancy (see section 3.20), the committee concluded that nivolumab with ipilimumab does not meet the criteria for life-extending treatments for people with a short life expectancy.
The company proposes including nivolumab with ipilimumab in the Cancer Drugs Fund while CheckMate 214 continues to collect data
3.23 Having concluded that nivolumab with ipilimumab does not qualify for routine commissioning, the committee then considered if it could recommend nivolumab with ipilimumab as part of a managed access agreement. The committee discussed the arrangements for the Cancer Drugs Fund agreed by NICE and NHS England in 2016, noting the addendum to the NICE process and methods guides. The committee agreed that nivolumab with ipilimumab had plausible potential to be cost effective either at a lower price, or by lessening the clinical uncertainty about overall survival by collecting longer-term data. It considered a proposal by the company for including nivolumab with ipilimumab in the Cancer Drugs Fund as part of a managed access agreement. In this, the company would collect further data from clinical trials, and would provide nivolumab with ipilimumab at a discounted price to the NHS for the duration of the managed access agreement. The committee understood that the company planned another CheckMate 214 data cut and analysis for August 2019, and then yearly up to 2021. This would provide up to 6 years' worth of overall survival data.
3.24 The committee agreed that the remaining clinical uncertainty in this appraisal emanated primarily from extrapolating overall survival for nivolumab with ipilimumab. It reconsidered the model's predictions of overall survival to determine whether the additional data from CheckMate 214 could reduce uncertainty. The committee recalled that both the company's and ERG's choice of survival curves (log-normal and Kaplan–Meier plus exponential respectively) could be reasonable, but that there was not enough evidence to determine with certainty which was more appropriate. As the 2 curves diverged, the separation area between them widened, increasing the difference in the predicted length of life and the ICERs, and hence the uncertainty. The committee agreed that, if survival curves are re-fitted to more mature data, this would help predict the long-term benefit of nivolumab with ipilimumab with higher precision. However, it also recognised that, as the trial continues, more patients will switch treatments (as permitted in the protocol) from sunitinib to nivolumab with ipilimumab biasing the trajectory of the survival curves. The committee also discussed 2 other sources of data that would be gained if nivolumab and ipilimumab is available to NHS patients through the Cancer Drugs Fund:
NHS England's systemic anti-cancer therapy (SACT) database: the committee discussed what data SACT could contribute and agreed that it would provide information on the proportion of people with intermediate- and poor-risk renal cell carcinoma in clinical practice and treatments offered in the NHS after nivolumab with ipilimumab. This data would better define death rate early in therapy and would inform treatment duration early in therapy.
A new regulatory trial: the company explained that, as part of a post-marketing commitment with the European Medicines Agency, it must set up a randomised trial including people with poor- or intermediate-risk renal cell carcinoma. This must evaluate whether the effectiveness of nivolumab with ipilimumab compared with nivolumab alone justifies the additional toxicity of ipilimumab. The committee noted that the regulatory trial could potentially contribute more information about treatment duration, early death rates and time to progression. However, it agreed that this trial was unlikely to reduce the uncertainty about long-term effectiveness.
The committee concluded that the remaining uncertainty in the model revolved around long-term survival predictions for nivolumab with ipilimumab adjusted for treatment switching, which further data collection from CheckMate 214 would likely reduce. It also concluded that the SACT database would supplement the additional evidence from CheckMate 214 and validate some modelled parameters.
3.26 Based on the considerations in section 3.24 and section 3.25, the committee considered that it could recommend nivolumab with ipilimumab for use in the Cancer Drugs Fund. The committee agreed that at the end of the period in the Cancer Drugs Fund, the updated model should include:
curve-fitting repeated for all curves rather than only the 2 curves considered to date using all available data (see section 3.18)
data for (early) mortality, progression-free survival and time to treatment duration from the new randomised trial (see section 3.24)
survival estimates adjusted for patients who switched from the sunitinib to the nivolumab with ipilimumab arm (see section 3.17)
standardised mortality ratios applied to the general population mortality to account for the possibility that the survival of patients might never reach that of people who have never had the disease (see section 3.10).