3 Committee discussion
The appraisal committee (section 5) considered evidence submitted by Roche and a review of this submission by the evidence review group (ERG). See the committee papers for full details of the evidence. This guidance includes recommendations only for untreated urothelial carcinoma when cisplatin-based chemotherapy is unsuitable. Additional evidence became available after the committee discussion for people with urothelial carcinoma that has progressed after platinum-containing chemotherapy.
3.1 Urothelial carcinoma causes a number of symptoms, including haematuria (blood in the urine) and increased frequency, urgency and pain associated with urination. Surgical treatments such as urostomy can have a substantial impact on quality of life and restrict daily activities. The patient experts explained that chemotherapy is associated with unpleasant side effects such as fatigue, nausea and vomiting and places people at a greater risk of infection. The committee was aware that many people with locally advanced or metastatic urothelial carcinoma are older and may have comorbidities, which can affect treatment decisions. The committee recognised that locally advanced or metastatic urothelial carcinoma has a significant impact on quality of life.
3.2 Initial treatment is usually with a cisplatin-containing chemotherapy regimen. However, cisplatin can be damaging to the kidneys, so is not suitable for some people with impaired kidney function or a poor performance status. People for whom cisplatin is unsuitable will usually be offered carboplatin plus gemcitabine or, if they are not well enough to tolerate this or they choose not to have it, best supportive care. The clinical experts explained that none of the current treatments offer lasting benefit and that prognosis is poor. The patient experts explained that the side effects of chemotherapy can have a major negative impact on quality of life and that regular hospital visits for treatment disrupt usual activities. The clinical experts noted that there have been no new treatments for locally advanced or metastatic urothelial carcinoma for a number of years and that, unlike for other cancers, there is no targeted or personalised treatment. The committee concluded that there is an unmet need for effective treatment options for people with locally advanced or metastatic urothelial carcinoma.
Carboplatin plus gemcitabine and best supportive care are relevant comparators in untreated disease when cisplatin is unsuitable
3.3 The company submitted clinical- and cost-effectiveness analyses comparing atezolizumab with carboplatin plus gemcitabine. Although it was included in the NICE scope, the company did not submit a comparison with best supportive care. It considered that best supportive care would not be appropriate for people well enough to be offered treatment with atezolizumab, and that there were not enough data for comparison with best supportive care. The committee heard that in clinical practice, carboplatin plus gemcitabine may not be suitable for a significant proportion of people for whom cisplatin is unsuitable and this group of people therefore have best supportive care. The committee understood that because atezolizumab is an immunotherapy with a different side effect profile to carboplatin plus gemcitabine, there may be some people for whom atezolizumab is suitable who would otherwise have best supportive care. The committee concluded that best supportive care was an appropriate comparator for the population with untreated disease for whom cisplatin is unsuitable, but acknowledged the lack of data would make a comparison difficult.
3.4 The committee noted that in the IMvigor 210 trial (see section 3.5), patients continued to take atezolizumab until unmanageable toxicity or lack of clinical efficacy. This means that some people continued to take atezolizumab after disease progression. The committee understood that for other immunotherapies in the same class, consideration has been given to stopping treatment after a defined period of time, assuming that treatment benefits would continue. The committee was concerned that there was no standard definition of loss of clinical efficacy. The clinical experts explained that the symptoms associated with locally advanced or metastatic urothelial carcinoma can be very unpleasant, so it is possible to use the severity of a person's symptoms, alongside radiological scans and blood tests, to assess whether the drug is benefitting them despite their disease progression. The clinical experts further explained that in clinical practice treatment with atezolizumab would only continue after disease progression for people who have had previous chemotherapy, and that people with progressive disease having atezolizumab as their first treatment would be moved onto a chemotherapy regimen as soon as possible. The committee concluded that most people with untreated locally advanced or metastatic urothelial carcinoma would stop treatment with atezolizumab when their disease progresses.
Atezolizumab appears to be an effective treatment but there is substantial uncertainty in the clinical-effectiveness evidence
3.5 The clinical-effectiveness evidence for atezolizumab came from a phase II, single-arm trial, IMvigor 210. The trial included 119 patients who had not had chemotherapy and for whom cisplatin was considered unsuitable. The objective response rate for these patients was 22.7% at 15 months (95% confidence interval [CI] 15.52 to 31.27) and median overall survival was 15.9 months (95% CI 10.4 to not estimable). The clinical experts explained that historically, response rates have been around 25% for untreated disease, and survival is usually about 12 months. The committee was concerned that without a trial directly comparing atezolizumab with other treatments, it was difficult to assess the relative treatment benefit of atezolizumab. In addition, the committee noted that the trial data were immature and based on a small number of patients and so there is considerable uncertainty about the results. The clinical experts explained that the response rates and overall survival data from IMvigor 210 match their clinical experience with atezolizumab; some people whose disease initially responds well to treatment sustain a lasting response. Moreover, people whose disease responds to treatment can have a good quality of life and some patients survive for a significant period of time. They noted that this was something they had not seen before with chemotherapies and as such atezolizumab represents a major change in clinical practice. The committee concluded that atezolizumab appeared to be an effective treatment option for people with the disease and for whom cisplatin is unsuitable, but there was considerable uncertainty about the size of the clinical benefit compared with other treatments.
The simulated treatment comparison was not robust because it did not account for all of the important prognostic factors
3.6 Atezolizumab has only been studied in a single-arm trial, so to compare atezolizumab with gemcitabine plus carboplatin, the company did a simulated treatment comparison and network meta-analysis. The committee was aware that the simulated treatment comparison relies on assuming that all of the important prognostic factors are accounted for, but heard from the ERG that the company had used a relatively limited number of prognostic factors. The clinical experts explained that, of the prognostic factors identified by the company, performance status and the presence of liver metastases on study entry are the most important. They further explained that haemoglobin levels and primary tumour site may also have an important effect on prognosis, so the committee considered that it would have been appropriate for these to be included. The committee was concerned that some of the studies providing evidence for the comparators did not report data for liver metastases, potentially limiting the results of the simulated treatment comparison. The committee considered that it was unlikely that all of the important prognostic factors had been accounted for in the simulated treatment comparison and therefore concluded that the analysis was not robust. In response to consultation, the company provided results from a matching-adjusted indirect comparison to validate the results from the simulated treatment comparison. The company stated that the analysis included adjustment for all of the data on baseline characteristics that were available from the comparator studies and that the predictions were similar to those of the simulated treatment comparison. However, the committee was still concerned that it was unlikely that all effect modifiers and prognostic factors were accounted for in the matching-adjusted indirect comparison, because some prognostic factors were not reported in the published studies. The committee agreed that the matching-adjusted indirect comparison was useful, but did not change its view that the simulated treatment comparison was not robust.
The network meta-analysis results are also unlikely to be robust but provide the only estimate of relative effectiveness
3.7 The company linked the results of the individual simulated treatment comparisons together through a network meta-analysis. This was done for several outcomes, but only overall survival was used in the economic model. The committee was concerned that, for the network meta-analysis, the evidence network was sparse, including only 2 trials of gemcitabine plus carboplatin. Both trials had been done more than 5 years ago and included only a small number of patients. In addition, it was difficult to assess how similar the patients were in each of these trials, because the number of previous therapies and other baseline characteristics were not consistently reported. The committee concluded that, because of the limitations in the simulated treatment comparison and in the evidence networks, the network meta-analysis was not robust. The committee therefore had concerns about the reliability of the results; it accepted that atezolizumab is likely to be clinically effective, but the size of the treatment effect cannot be reliably established from the indirect comparison. The committee agreed that it would need to account for its concerns about the reliability of the results in its decision-making.
3.8 The clinical experts explained that in their experience of using atezolizumab, it is well tolerated and associated with fewer severe adverse events than chemotherapy. However, the committee was concerned that because there are no comparative clinical trial data, it is difficult to draw conclusions about the relative safety profile of the drug. In response to consultation, the company stated that results from a phase III trial of atezolizumab in patients with locally advanced or metastatic urothelial carcinoma who have had chemotherapy show that atezolizumab is better tolerated than cytotoxic chemotherapy. However, the committee understood that atezolizumab is still associated with some unpleasant and potentially serious adverse events. The clinical experts explained that they are actively working on ways to identify and manage the adverse events from immunotherapies. The committee concluded that the ongoing IMvigor 130 trial will provide more data on the adverse events associated with atezolizumab and current treatments for untreated metastatic urothelial carcinoma.
There are several plausible overall survival extrapolations but the ERG's approach is acceptable for decision-making
3.9 The company used a generalised gamma distribution to model atezolizumab overall survival, because this distribution fitted well to the observed data for atezolizumab. This approach led to 5‑year survival estimates of around 28% for atezolizumab and 12% for carboplatin plus gemcitabine. The company also provided alternative scenarios using different extrapolations in response to consultation. The ERG proposed an approach in which it used the Kaplan–Meier overall survival curve from the clinical trial and extrapolated the tail using an exponential distribution. The choice of distribution was based on the best fit to the comparator data, using the trial with the longest follow‑up and the largest number of patients (the De Santis trial). This approach led to 5‑year survival estimates of less than 10% for atezolizumab and around 1% for carboplatin plus gemcitabine. The committee considered that the company's approach led to 5‑year survival estimates that were implausibly high for both atezolizumab and carboplatin plus gemcitabine. In particular, the proportion of people alive at 5 years in the atezolizumab arm (28%) was higher than the proportion of patients whose disease had responded to treatment at 15 months (23%). In response to consultation, the company argued that it was plausible that more patients benefit from atezolizumab than achieve an objective tumour response because of its mechanism of action as an immunotherapy. However, the committee also noted that the 5‑year survival of 12% predicted by the company's approach for carboplatin plus gemcitabine was much higher than the observed 5‑year survival in the De Santis trial of around 1%. The committee considered that the ERG's extrapolation predicted 5‑year survival rates that were more plausible and consistent with the available data. In response to consultation, the company raised concerns that using the ERG's approach leads to implausible results, because the progression-free and overall survival curves meet or cross. The ERG acknowledged this, and proposed to address it by adjusting the progression-free survival extrapolation. This was because the ERG considered that the progression-free survival extrapolation was less robust than the overall survival extrapolation, because the former was based on the uncertain assumption that progression-free survival would be the same between atezolizumab and carboplatin plus gemcitabine. The ERG explained that a Weibull distribution fits the progression-free survival data well and the extrapolated curves do not cross the overall survival curves extrapolated using the exponential distribution. The committee accepted that this was a reasonable approach. The committee recognised that the extrapolation of overall survival was highly uncertain, and had a significant effect on the cost effectiveness. It considered that it was possible that the overall survival extrapolation could fall between the company and ERG's approaches. However, based on the evidence it had available, it concluded that the ERG's approach was more appropriate for decision-making, because it used more data and produced more clinically plausible results.
3.10 The company extrapolated the observed duration of atezolizumab treatment from IMvigor 210 because the trial was ongoing. The company chose a generalised gamma distribution. However, the ERG noted that the Weibull distribution provided a better statistical fit. In response to consultation, the company argued that on visual examination, the gamma distribution provided more plausible estimates of treatment duration than the Weibull distribution. The committee noted that the choice of distribution had only a small effect on the cost-effectiveness results and agreed that it was more appropriate to use the Weibull distribution because it fitted the data best.
The committee could not establish if the modelling of the atezolizumab treatment effect was reliable
3.11 The relative treatment effect for overall survival was based on the results of the indirect comparison (see section 3.6). The committee was concerned about the reliability of these results because the analyses were not robust. The committee also noted that because the company considered some of the results to be implausible, it had chosen to cap the hazard ratios. The committee noted the ERG exploratory analyses that varied the initial hazard ratio using the confidence intervals from the network meta-analysis. The cost-effectiveness results were very sensitive to whether the upper or lower bound was used, because the confidence intervals are very wide, reflecting the uncertainty of the comparisons. The committee understood that the company's survival estimates depend on an ongoing reduction in the relative risk of death with atezolizumab compared with carboplatin plus gemcitabine, which continues after treatment has stopped and is maintained for a lifetime. The ERG provided scenario analyses with the hazard ratio set to 1 at 2, 3 or 5 years to model the effect on the incremental cost-effectiveness ratio (ICER) if the treatment effect stopped over time. This increased the company's ICERs. The committee acknowledged that the duration of continued treatment effect is an area of uncertainty for new immunotherapies. Although it had not been presented with any evidence about whether the treatment benefit is maintained over time, the committee recognised that if it is not, this would increase the ICERs. The committee was also concerned that the company assumed in their model that the treatment effect did not diminish for people continuing treatment after disease progression; they would have the same treatment benefit from atezolizumab as people whose disease has not progressed. The committee thought that this was implausible, but recalled comments from the clinical experts that most people with progressive disease having atezolizumab as their first treatment would move to a chemotherapy regimen as soon as possible (see section 3.4). The committee recognised that the atezolizumab treatment effect was an important driver of the model results, but because of the limitations in the clinical evidence, the modelling could be unreliable.
3.12 No health-related quality-of-life data were collected in IMvigor 210. Instead, the company used utility values from an Australian health technology assessment of vinflunine for metastatic urothelial bladder cancer. The committee was concerned that the utility value of 0.71 used for the progressed disease health state was too high. This is because the average age of people in IMvigor 210 was 73, and the utility value for the age-matched general population was also likely to be around 0.71. The clinical experts also explained that they would expect health-related quality of life to decline as people's disease progressed. The ERG did a scenario analysis that reduced the on‑treatment utility for the comparators. This reflected the greater number of adverse events associated with chemotherapy, but did not address the committee's concerns about the utility value for the progressed disease health state. The committee noted a company sensitivity analysis in which the post-progression utility value was 0.5 rather than 0.71. Although this value was arbitrarily chosen, it had a large impact on the cost-effectiveness results, increasing the ICER for atezolizumab at the list price compared with carboplatin plus gemcitabine by £25,000 per quality-adjusted life year (QALY) gained. The committee considered that a utility value of 0.5 may be too low, and therefore the ICER that results from using it too high. The committee concluded that post-progression utility is an important driver of the model and the most plausible value is likely to be between 0.5 and 0.71.
3.13 The company's base-case ICER using the list price for atezolizumab was £44,158 per QALY gained compared with carboplatin plus gemcitabine. In the company's scenario analyses, using a post-progression utility value of 0.5 and a range of alternative overall survival extrapolations, the ICERs ranged from £61,003 to £77,452 per QALY gained. The ERG's preferred ICER was £95,211 per QALY gained using a post-progression utility value of 0.71 and £117,703 per QALY gained using a post-progression utility value of 0.5. The company agreed a confidential discount with the Department of Health and the committee considered analyses incorporating the discount. However, the results of these analyses cannot be reported here because they are considered confidential by the company.
3.14 The probabilistic sensitivity analyses submitted by the company increased the ICERs by around 8%. The company explained that the probabilistic results were unlikely to be reliable, because the uncertainty in the network meta-analysis meant that at extreme draws in the probabilistic analysis, an implausible proportion of patients in the comparator arms were alive at 20 years. The committee concluded that because of this problem, the company's probabilistic analysis may not necessarily be suitable for decision-making. However, given that the probabilistic ICERs were so much higher, it was likely that accounting for the significant uncertainty around the treatment effect would increase the ICERs. The committee highlighted that robust probabilistic sensitivity analysis is an essential requirement of company submissions.
the atezolizumab overall and progression-free survival based on the Kaplan–Meier curves with the tails extrapolated using the exponential and Weibull distributions respectively (see section 3.9)
the duration of atezolizumab treatment extrapolated using the Weibull distribution (see section 3.10) and
a lower on‑treatment utility value for the comparators (see section 3.12).
The committee accepted the ERG's choice of atezolizumab overall survival, progression-free survival and treatment duration extrapolation as suitable for decision-making, but noted that the ERG's analysis did not reflect all of its preferred assumptions. Firstly, the ERG's analysis assumed that the treatment benefit of atezolizumab continues for the duration of the model, but if the treatment effect stopped over time, or was reduced after disease progression the ICER would be higher (see section 3.11). Secondly, the analyses were deterministic and therefore the uncertainty was not appropriately reflected in probabilistic results. Finally, although the ERG had provided results using post-progression utility values of 0.5 and 0.71, the committee felt that a more appropriate value would be somewhere in this range. Therefore the committee concluded that the most plausible ICER based on the company's list price would be higher than £95,211 per QALY gained. The most plausible ICER based on the patient access scheme discount was confidential so cannot be reported here.
The committee could not make recommendations for subgroups based on PD‑L1 expression because cost-effectiveness analyses were not provided
3.16 The committee considered whether there were any subgroups for whom atezolizumab may be more cost effective. The committee was aware that atezolizumab works by inhibiting the PD‑L1 protein and that other immunotherapies with similar mechanisms of action had reported greater effectiveness in patients with higher levels of PD‑L1 expression. The committee considered that it was therefore possible that atezolizumab might be more cost effective in some groups. The company presented clinical results from IMvigor 210 based on the PD‑L1 expression of tumour-infiltrating immune cells greater than 1% and greater than 5%. These showed a higher objective response rate associated with higher PD‑L1 expression in the population who had previously had chemotherapy. This did not appear to be the case for the population with untreated disease for whom cisplatin is unsuitable; the clinical experts explained that PD‑L1 does not appear to be a good predictor of outcomes in this population. However, the committee noted that the company had not provided cost-effectiveness analyses based on PD‑L1 subgroup data. The committee would have liked to have seen these analyses. It was unable to make recommendations for any subgroups based on PD‑L1 expression.
3.17 The committee considered the advice about life-extending treatments for people with a short life expectancy in NICE's Cancer Drugs Fund technology appraisal process and methods.
3.18 Data from the company's model and from the literature showed that median overall survival was much less than 24 months for people having treatment with any standard care. The clinical experts also agreed that they would expect people with locally advanced or metastatic urothelial carcinoma to live for less than 24 months. The committee concluded that the short life expectancy criterion was met.
3.19 The committee noted that because of the lack of phase III data directly comparing atezolizumab with other treatments, it was difficult to draw conclusions about overall survival gain. However, data from the company's model and from the literature suggested a difference in median survival of at least 7 months. The committee emphasised the limitations in the evidence available, but concluded that it was most likely that atezolizumab would extend life by more than 3 months.
3.21 The committee concluded that the most plausible ICERs based on the patient access scheme discount (see section 3.15) were higher than those usually considered a cost-effective use of NHS resources, even for end-of-life treatments. The clinical and cost effectiveness of atezolizumab was highly uncertain because of serious limitations in the clinical evidence. The committee did not recommend atezolizumab for routine NHS use for people with untreated locally advanced or metastatic urothelial carcinoma for whom cisplatin is unsuitable.
3.22 Having concluded that atezolizumab could not be recommended for routine use, the committee then considered if it could be recommended for untreated locally advanced or metastatic urothelial carcinoma within the Cancer Drugs Fund. The committee discussed the new arrangements for the Cancer Drugs Fund agreed by NICE and NHS England in 2016, noting the addendum to the NICE process and methods guides. The committee was aware that the company was interested in atezolizumab being considered through the Cancer Drugs Fund.
3.23 The committee's preferred ICER for the population with untreated disease and for whom cisplatin is unsuitable was higher than those usually considered a cost-effective use of NHS resources for end-of-life treatments. The committee noted that the ICER was most sensitive to the extrapolation used for the atezolizumab overall survival curve. The committee preferred the ERG's choice of the exponential distribution, because it considered that the number of people taking atezolizumab estimated to be alive at 5 years in the company's model using the gamma distribution (28%) was implausible. The ERG's approach also produced estimates consistent with the observed data for carboplatin plus gemcitabine. The model using an exponential distribution predicted that around 10% of people would be alive at 5 years. Although the committee agreed that this was more plausible and the most reliable estimate for decision-making at this stage (see section 3.9), it acknowledged that this might later prove to be a conservative estimate. The committee recognised that as more trial data on clinical effectiveness become available, the true curve may lie somewhere between the company and the ERG's estimates. In this situation the ICER could decrease to a level that is considered a cost-effective use of resources and atezolizumab would provide sufficient extension to life to meet the end-of-life criteria. It concluded that atezolizumab has the potential to satisfy the criteria for routine use in the NHS as an end-of-life treatment, but more data are needed.
The company's Cancer Drugs Fund data collection proposal addresses most of the clinical uncertainties
3.24 The committee considered that the main uncertainty is that the relative effectiveness of atezolizumab is difficult to assess, because it has only been studied in a single-arm trial. This means that all comparisons are based on indirect methods. In response to consultation, the company submitted a proposal for data collection within the Cancer Drugs Fund. It proposed that this key uncertainty could be addressed by the IMvigor 130 trial, an ongoing randomised controlled trial comparing atezolizumab with carboplatin plus gemcitabine in people with previously untreated locally advanced or metastatic urothelial carcinoma. It is likely to finish in July 2020.
3.25 The company also suggested that other uncertainties could be addressed by this trial including:
the duration of treatment with atezolizumab
appropriate health-related quality-of-life values, because EQ‑5D data are being collected in the trial and
effectiveness for PD‑L1 subgroups, because the trial is stratified by PD‑L1 expression.
The committee was concerned that patients in the trial whose disease progresses would only be followed up for a short time, so health-related quality-of-life data for progressed disease may not be robust. The committee encouraged the company and NHS England to seek ways to collect health-related quality-of-life data for people with progressed disease. The committee was aware that the Systemic Anti-Cancer Therapy (SACT) dataset would also provide data on treatment duration and overall survival. The committee concluded that the IMvigor 130 trial and data from the SACT dataset would provide evidence to address most of the uncertainties in the clinical evidence. The committee recommended atezolizumab for inclusion in the Cancer Drugs Fund for people with untreated locally advanced or metastatic urothelial carcinoma for whom cisplatin is unsuitable.
3.26 No equality issues were identified.
3.27 The company did not highlight any additional benefits that had not been captured in the QALY.