3 Committee discussion

The appraisal committee (section 6) considered evidence submitted by Merck Sharp & Dohme and a review of this submission by the evidence review group (ERG). See the committee papers for full details of the evidence.

The condition

Metastatic urothelial carcinoma substantially decreases quality of life

3.1 Urothelial carcinoma causes a number of symptoms, including haematuria (blood in the urine) and increased frequency, urgency and pain associated with urination. The patient experts explained that chemotherapy is associated with unpleasant side effects such as fatigue, nausea and vomiting. Also, the condition and the treatment for it can have a significant effect on mental wellbeing. The committee was aware that many people with locally advanced or metastatic urothelial carcinoma are older and may have comorbidities, which can affect treatment decisions. The committee recognised that locally advanced or metastatic urothelial carcinoma substantially decreases quality of life.

Current treatments

People with untreated locally advanced or metastatic urothelial carcinoma would welcome an effective new treatment option

3.2 Initial treatment is usually with a cisplatin-containing chemotherapy regimen. However, cisplatin can be damaging to the kidneys, so is not suitable for some people with impaired kidney function or a poor performance status. When cisplatin is unsuitable people will usually be offered carboplatin plus gemcitabine or, if they are not well enough to tolerate this or they choose not to have it, best supportive care. The clinical expert explained that chemotherapy does not offer lasting benefit and that prognosis is poor. The patient experts added that the side effects of chemotherapy can have a major negative effect on quality of life and that regular hospital visits for treatment disrupt usual activities. The committee was aware that atezolizumab has a marketing authorisation for untreated locally advanced or metastatic urothelial carcinoma when cisplatin is unsuitable. However, because of uncertainty in the clinical-effectiveness evidence, it is currently recommended by NICE for use within the Cancer Drugs Fund only. The committee concluded that people with locally advanced or metastatic urothelial carcinoma would welcome an effective new treatment option.

Comparators

Carboplatin plus gemcitabine and best supportive care are relevant comparators in untreated disease when cisplatin is unsuitable

3.3 The company submitted clinical- and cost-effectiveness analyses comparing pembrolizumab with carboplatin plus gemcitabine. Although it was included in the NICE scope, the company did not submit a comparison with best supportive care. It considered that such a comparison would not be appropriate because there is an alternative active treatment available (carboplatin plus gemcitabine). Also, there were not enough data to enable a comparison. The committee understood that in clinical practice, carboplatin plus gemcitabine may not be suitable for a large proportion of people for whom cisplatin is unsuitable. This group of people therefore have best supportive care. Because pembrolizumab is an immunotherapy with a different side effect profile to carboplatin plus gemcitabine, there may be some people for whom pembrolizumab is suitable who would otherwise have best supportive care. The committee concluded that best supportive care was an appropriate comparator for the population with untreated disease for whom cisplatin is unsuitable, but acknowledged the lack of data would make a comparison difficult. The company did not provide a comparison with atezolizumab, which was also included in the scope. Atezolizumab is only available in the Cancer Drugs Fund because of the uncertainty about its effectiveness. The committee concluded that it was not part of established practice and therefore not an appropriate comparator at this time.

Clinical trial evidence

Pembrolizumab appears to be an effective treatment but there is substantial uncertainty in the clinical-effectiveness evidence

3.4 The clinical-effectiveness evidence for pembrolizumab came from a phase 2, single-arm trial, KEYNOTE‑052. The trial included 370 patients who had not had chemotherapy for advanced or metastatic disease and for whom cisplatin was considered unsuitable. The trial is ongoing, with a median duration of follow‑up of 9.5 months at the March 2017 data cut. In the overall population, median overall survival was 11.0 months (95% confidence interval [CI] 10.0 to 13.6 months). Median progression-free survival was 2.3 months (95% CI 2.1 to 3.4 months). The company also submitted data from a later data cut. This showed a slight increase in median overall survival but the exact figure cannot be reported here because it is considered confidential by the company. The clinical expert explained that some people taking pembrolizumab sustain a very long response, which is not seen with chemotherapy. The patient experts also noted that people whose disease responds to treatment can have a good quality of life so pembrolizumab represents a major change in clinical practice. The committee was concerned that without a trial directly comparing pembrolizumab with other treatments, it was difficult to assess pembrolizumab's relative treatment benefit. Also, the committee noted that the trial data were immature so there is considerable uncertainty about the long-term benefits. The committee concluded that pembrolizumab appeared to be an effective treatment option for people with untreated disease when cisplatin is unsuitable. However, there was considerable uncertainty about the size of the clinical benefit compared with other treatments and the duration of these benefits.

Subgroup analyses based on level of PD-L1 expression are not useful for decision-making

3.5 The company submitted clinical- and cost-effectiveness analyses for 2 subgroups based on level of PD‑L1 expression: patients with a combined positive score of 1 or more and patients with a combined positive score of 10 or more. These analyses showed higher median overall survival and progression-free survival associated with increasing level of PD‑L1 expression. However, the committee recalled that clinical experts had previously explained that PD‑L1 status does not appear to be a good predictor of outcomes in this population. In addition, the Cancer Drugs Fund clinical lead advised that PD‑L1 testing is not routinely done in the NHS for people with urothelial cancer. The ERG also explained that because no trials of carboplatin plus gemcitabine assessed PD‑L1 status, it would be inappropriate to make any comparisons between pembrolizumab and chemotherapy for subgroups based on PD‑L1 expression. The committee concluded that the subgroup analyses based on PD‑L1 expression were not useful for decision-making and it would not consider them further.

Indirect comparison

The results of the simulated treatment comparison lack validity

3.6 Pembrolizumab has only been studied in a single-arm trial. So, to compare pembrolizumab with carboplatin plus gemcitabine, the company did a simulated treatment comparison and network meta-analysis. These were based on the March 2017 data cut of KEYNOTE‑052, and 4 trials reporting outcomes with carboplatin plus gemcitabine. The committee was aware that a simulated treatment comparison relies on assuming that all of the important prognostic factors are accounted for, and was concerned that this was unlikely to be the case with the company's analysis. The ERG explained that when the expected outcomes with pembrolizumab were simulated using the baseline characteristics data for the patients in each of the comparator trials, all of the predicted pembrolizumab overall survival curves were higher than the actual curve from KEYNOTE‑052. That is, the company's model suggested that if a population with baseline characteristics matching those in the comparator trials had pembrolizumab, they would be expected to have better outcomes than the population in KEYNOTE‑052. The only way this would be the case was if the patients in the comparator trials were in better health on average than those in KEYNOTE‑052. However, the committee considered that the baseline characteristics across the comparator trials were similar to those in KEYNOTE‑052. In particular, compared with the De Santis (2012) trial, which had the most patients and longest follow‑up, patients in KEYNOTE‑052 had similar or more favourable values for each of the prognostic factors included in the simulated treatment comparison, except for visceral metastases. However, the ERG explained that despite the proportion of patients with visceral metastases being higher in KEYNOTE‑052, the associated coefficient in the simulated treatment comparison was very small. The difference in visceral metastases alone would not therefore be expected to have such a large effect on the results. The committee concluded that the results of the company's simulated treatment comparison lack validity.

The results of the network meta-analyses are unlikely to provide a robust estimate of relative effectiveness

3.7 The company linked the results of the individual simulated treatment comparisons for progression-free survival and overall survival together through network meta-analyses. In its base case, the company used a fractional polynomial approach, in which time-varying hazard ratios were estimated. The committee was concerned that there was substantial heterogeneity between the comparator studies. In particular, the way performance status and metastases had been measured and reported differed and the dosage and administration of carboplatin plus gemcitabine varied between studies. Also, apart from the De Santis trial, the other comparator studies were more than 10 years old and included only a small number of patients. The committee also noted that an alternative fractional polynomial model explored by the company fitted the data better than its base-case model. This better-fitting model estimated a lower treatment benefit with pembrolizumab than the company's base-case model. The committee concluded that because of the limitations in the simulated treatment comparison and in the evidence networks, the network meta-analyses were unlikely to provide a robust estimate of relative effectiveness.

The committee will consider both the company's and the ERG's indirect comparisons in its decision-making

3.8 The ERG highlighted that the De Santis trial of carboplatin plus gemcitabine was the largest and most well conducted of the comparator trials and had the longest follow‑up. Also, the patients had similar baseline characteristics to those in KEYNOTE‑052. Therefore an unadjusted comparison between the De Santis trial and KEYNOTE‑052 could provide a more robust estimate of relative effectiveness than the company's indirect comparison. Although the committee agreed that the baseline characteristics in the 2 trials were similar, it had concerns about this approach because of the risk of bias associated with unadjusted comparisons. However, given that the results of the company's simulated treatment comparison also lacked validity, it concluded it would consider both approaches in its decision-making.

Adverse events

Pembrolizumab is well tolerated in clinical practice but more comparative data will be useful

3.9 The clinical expert explained that pembrolizumab is well tolerated and associated with fewer severe adverse events than chemotherapy. However, the committee was concerned that because there are no comparative clinical trial data, it is difficult to draw conclusions about pembrolizumab's relative safety profile. The committee concluded that an ongoing trial, KEYNOTE‑361, will provide more data on the adverse events associated with pembrolizumab and current treatments for untreated metastatic urothelial carcinoma.

Assumptions used in the economic model

The ERG's overall survival extrapolation produces more plausible estimates for the comparator arm but is based on an unadjusted comparison

3.10 The company used a piecewise approach to model pembrolizumab overall survival, using a log normal distribution to extrapolate the tail of the Kaplan–Meier curve from 32 weeks. To obtain overall survival for the comparator arm, the company applied the time-varying hazard ratios from the network meta-analysis to the pembrolizumab curve. The company justified using a piecewise approach for extrapolating the pembrolizumab data on the basis that the log cumulative hazard against the log time plot showed a change in hazard. The ERG explained that although this was the case, it only ruled out Weibull or exponential distributions, and did not necessarily mean a piecewise approach was appropriate. The ERG fitted a fully parametric log normal distribution to the pembrolizumab data. This distribution fitted the observed data best, although several other distributions also had a good fit to the data. Because the ERG preferred an unadjusted comparison, they fitted a spline model with 2 knots to the De Santis carboplatin plus gemcitabine data, rather than using hazard ratios from the network meta-analysis. The company's approach predicted that 10% of people in the comparator arm would be alive at 2 years, and 1% at 5 years, whereas the ERG's approach matched the observed De Santis data: 14% at 2 years and 2% at 5 years. The ERG's approach predicted that fewer people in the pembrolizumab arm would be alive at 5 years (11%) than in the company's approach (14%). The committee considered that the ERG's extrapolation for the comparator arm was more robust than the company's because it more closely matched the available long-term follow‑up data. However, the committee acknowledged that because there are limited data on long-term outcomes with immunotherapies, it is difficult to assess which of the pembrolizumab extrapolations was more plausible. Also, although the fully parametric log normal distribution had the best statistical fit to the pembrolizumab data, 4 other distributions also had a good statistical fit. These other distributions predicted that between 7% and 14% of patients would be alive at 5 years. The committee recognised that the extrapolation of overall survival was highly uncertain, and had a substantial effect on cost effectiveness. It concluded that although the ERG's approach produced more plausible estimates for the comparator arm, it was based on an unadjusted comparison. Therefore the committee considered both the company's and the ERG's approaches in its decision-making.

The ERG's progression-free survival extrapolations are more appropriate, but have a limited effect on the cost-effectiveness results

3.11 The company used a piecewise approach to model pembrolizumab progression-free survival, using a Weibull distribution to extrapolate the tail of the Kaplan–Meier curve from 9 weeks. To model progression-free survival for the comparator arm, the company used the time-varying hazard ratios from the network meta-analysis. The ERG fitted separate spline models with 3 knots to both the pembrolizumab and the De Santis data for carboplatin plus gemcitabine. The De Santis progression-free survival data were not available to the company and so were not included in its network meta-analysis. This meant that the company and the ERG's extrapolated curves for the comparator arm had a different shape. For the pembrolizumab arm, both the company's and ERG's extrapolated curves were similar. The committee considered that the ERG's approach to modelling progression-free survival was more appropriate because it used more robust data for the comparator arm. However, the committee noted that the different methods for modelling progression-free survival had only a limited effect on the cost-effectiveness results.

A 2-year stopping rule for pembrolizumab is appropriate

3.12 In the KEYNOTE‑052 trial protocol, pembrolizumab must be stopped 2 years from the date of the first dose. This stopping rule was reflected in the company's economic model but not in the summary of product characteristics, which states that treatment should continue until disease progression or unacceptable toxicity. The committee understood that for other indications, NICE guidance has included a recommendation to stop pembrolizumab after a defined period of time. The Cancer Drugs Fund clinical lead confirmed that it would be possible to implement a 2‑year stopping rule in the NHS. The committee concluded that incorporating a 2‑year stopping rule in its decision-making was appropriate.

A lifetime treatment effect is implausible

3.13 The company's 2‑year stopping rule in the economic model applied only to costs and not to treatment benefit. The committee noted that there were no data from KEYNOTE‑052 on the effect of implementing the stopping rule on outcomes, because at the trial cut‑off no patient had taken pembrolizumab for 2 years. The company assumed in its base case that pembrolizumab remains effective throughout the model's time horizon of 20 years even after stopping treatment at 2 years. However, it explored scenarios in which the hazard ratios for overall survival and progression-free survival were set to 1 at different time points to model stopping of the continued treatment effect. The ERG's base case accounted for the effect of the stopping rule on the pembrolizumab treatment effect by switching patients in the pembrolizumab arm to the overall survival and progression-free survival hazards of the carboplatin plus gemcitabine arm at 2 years. The committee was aware that the duration of continued treatment effect after implementation of a stopping rule is an area of uncertainty for new immunotherapies, but it concluded that a lifetime continued treatment effect was implausible.

Utility values should be based on progression state

3.14 EQ‑5D data from KEYNOTE‑052 were used in the company's base case to estimate separate utility values for 5 groups based on time to death (from less than 30 days to more than 360 days). The company stated that this approach captured diminishing quality of life after progression, which is not possible using values for the progression-free and progressed disease health states. However, the committee was concerned that the utility value for people with longer than 360 days to live (0.753) was similar to the average utility value for the age-matched general population. The ERG also noted that in the model, patients can spend longer than 360 days in the progressed disease state. This meant that the utility value for this group can remain high despite them having progressed disease and no longer having treatment. The ERG preferred to use progression-based utility values from KEYNOTE‑052 instead: 0.68 for the progression-free state and 0.61 for the progressed disease state. The committee noted that the utility value for people living longer than 180 days (0.685) was higher than the average value for progression-free survival from the trial. It therefore considered that it was more appropriate to use the progression-based utility values. However, it acknowledged concerns from the company that because of the limited number of records for the post-progression health state, progression-based utilities do not show a large difference between pre- and post-progression, so progression status is unlikely to sufficiently reflect changes in quality of life. The clinical expert confirmed that they would expect quality of life to decrease substantially after disease progression for this population. The committee was therefore concerned that a value of 0.61 for the progressed disease health state may be too high. It noted that the ERG provided a scenario analysis using a utility value of 0.55 for progressed disease, based on the value for people living between 30 and 90 days from the time to death approach. This increased the incremental cost-effectiveness ratio (ICER) by around £6,000 per quality-adjusted life year (QALY) gained. The committee concluded that this value might be a better reflection of post-progression utility.

Cost-effectiveness estimates

The company proposed pembrolizumab for use within the Cancer Drugs Fund

3.15 The company submitted a proposal for the committee to consider pembrolizumab for use in the Cancer Drugs Fund rather than for routine commissioning. This included a proposed confidential commercial access agreement for pembrolizumab within the Cancer Drugs Fund. The committee considered ICERs based on this commercial offer in its decision-making.

The ERG's ICERs are higher than the company's ICERs

3.16 The company updated its model after clarification to address 2 minor errors and to include the correct proposed commercial access agreement for pembrolizumab. The company's base-case ICER using the March 2017 data cut of KEYNOTE‑052 was £35,970 per QALY gained compared with carboplatin plus gemcitabine. Although a later data cut from KEYNOTE‑052 was available, the company had not been able to include this in its simulated treatment comparison in time for the appraisal committee meeting. The ICER using the ERG's preferred assumptions was £65,642 per QALY gained. The ERG's analysis included:

  • relative treatment effect based on an unadjusted comparison of the March 2017 KEYNOTE‑052 data with the De Santis data for carboplatin plus gemcitabine (see section 3.8)

  • progression-based utility values (see section 3.14)

  • no continued benefit with pembrolizumab after stopping treatment (see section 3.13).

The most plausible ICER is likely to be in the range £43,702 to £65,642 per QALY gained, but could be higher

3.17 The committee considered that given the 2‑year stopping rule and lack of data showing longer-term benefits, it was more appropriate to assume that there would be no continued treatment benefit rather than assuming it persists indefinitely (see section 3.13). However, it noted that this had only a small effect on both the company's and ERG's cost-effectiveness estimates. The committee accepted that the utility values should be progression-based (see section 3.14), meaning that the most plausible ICER was at least £43,702 per QALY gained, as in the company's scenario analysis. The committee was concerned about the validity of the company's estimates from the simulated treatment comparison and network meta-analysis but acknowledged that the ERG's unadjusted analysis may also be subject to residual bias. The committee therefore concluded that the most plausible ICER was likely to be in the range £43,702 to £65,642 per QALY gained. However, the committee also considered that the ERG's scenario analysis using a post-progression utility value of 0.55 may be a better reflection of quality of life in this health state (see section 3.14). This would mean that the upper end of the range could be higher than £65,642 per QALY gained.

End of life

Life expectancy for people with untreated locally advanced or metastatic urothelial carcinoma is less than 24 months

3.18 The committee considered the advice about life-extending treatments for people with a short life expectancy in NICE's final Cancer Drugs Fund technology appraisal process and methods. For people with untreated locally advanced or metastatic disease and when cisplatin-containing chemotherapy is unsuitable, mean overall survival was less than 24 months in both the company's and the ERG's models for people having treatment with carboplatin plus gemcitabine (10.3 months and 13.2 months respectively). The committee concluded that the short life expectancy criterion was met.

Pembrolizumab extends life by at least 3 months, and meets the criteria for end-of-life treatments

3.19 The committee noted that the median overall survival for pembrolizumab based on the March 2017 data cut of KEYNOTE‑052 was 11.0 months. The company's economic model estimated that the mean survival with pembrolizumab was 27.0 months compared with 10.3 months with carboplatin plus gemcitabine. Extension to life using the ERG's preferred assumptions was around 9.5 months with pembrolizumab and was greater than 3 months in all of the survival extrapolations tested. The committee concluded that pembrolizumab would extend life by more than 3 months, and therefore met the end-of-life criteria.

Cancer Drugs Fund

Pembrolizumab is recommended for use within the Cancer Drugs Fund

3.20 The committee understood that it was not considering pembrolizumab for routine use, and discussed the arrangements for the Cancer Drugs Fund agreed by NICE and NHS England in 2016, noting the addendum to the NICE process and methods guides. The committee noted that although there was uncertainty about the relative treatment effect and long-term survival with pembrolizumab, using the company's approach pembrolizumab has plausible potential to be cost effective. The committee was aware that further data on overall survival would be available from KEYNOTE‑052 because patients are still being followed up. Also, KEYNOTE‑361, an ongoing randomised controlled trial comparing pembrolizumab with carboplatin plus gemcitabine in people with untreated locally advanced or metastatic urothelial carcinoma, would provide a more robust estimate of the relative treatment effect. The committee understood that the Systemic Anti-Cancer Therapy (SACT) dataset would also provide data on treatment duration and overall survival. The committee concluded that the data from KEYNOTE‑052, KEYNOTE‑361 and from the SACT dataset would provide evidence to address most of the uncertainties in the clinical evidence. The committee concluded that pembrolizumab met the criteria to be considered for inclusion in the Cancer Drugs Fund. It recommended pembrolizumab for use within the Cancer Drugs Fund as an option for people with untreated locally advanced or metastatic urothelial carcinoma when cisplatin-containing chemotherapy is unsuitable, if treatment does not exceed 2 years and the conditions in the managed access agreement are followed.[1]

Other factors

3.21 No equality issues were identified.

3.22 The company did not highlight any additional benefits that had not been captured in the QALY calculations.



[1] July 2018: The European Medicines Agency restricted the use of pembrolizumab for untreated urothelial carcinoma. It should now only be used in adults with high levels of PD-L1. For more information, see the summary of product characteristics for pembrolizumab.

  • National Institute for Health and Care Excellence (NICE)