3 Committee discussion
The appraisal committee (section 4) considered evidence submitted by Roche Products and a review of this submission by the evidence review group (ERG). See the committee papers for full details of the evidence.
People with advanced basal cell carcinoma that is inappropriate for surgery and radiotherapy would welcome treatment options
3.1 The committee understood that surgery and radiotherapy are the standard of care for treating locally advanced and metastatic basal cell carcinoma (BCC), also collectively termed 'advanced BCC'. When these are inappropriate there are no other active treatments available and best supportive care is the only option. The experts stated that this group of patients is typically older and relatively frail, with a number of comorbidities, and vismodegib has been an important treatment option since its availability on the Cancer Drugs Fund. The committee concluded that other treatment options would be welcomed.
3.2 The committee heard that advanced BCC is not tracked in the national cancer registries so definitive UK mortality and morbidity data are not readily available. However, clinical experience indicates that patients with locally advanced BCC are likely to have a similar life expectancy to the general population. There is increased mortality in some patients with metastatic BCC, but the clinical experts agreed that death is unlikely to be a result of the primary tumour. However, 1 clinical expert highlighted the burden of disease in this population. They explained that people with advanced BCC that is inappropriate for surgery and radiotherapy often develop large disfiguring lesions because of extensive tissue destruction, particularly on the face and neck. As a result, they have significant psychological stress and can become socially isolated. Additionally, wounds bleed, are painful and need regular dressing. The expert noted that a subgroup of people with Gorlin syndrome, an inherited condition, can develop a large number of tumours. People with Gorlin syndrome tend to develop BCC early in life and will therefore face the lifelong impact of multiple surgeries because the condition cannot be treated by radiotherapy. The committee heard that vismodegib can improve appearance, reduce pain and bleeding and subsequently improve nutrition, social interaction and overall quality of life. The committee concluded that patients would value access to effective treatments that improve quality of life.
3.3 The committee discussed the evidence from the ERIVANCE (n=104) and STEVIE (n=1,215) trials, including the baseline characteristics of the patients:
Both trials included substantially more patients with locally advanced BCC than with metastatic BCC. The clinical experts advised that this reflects clinical practice, in which metastatic BCC is much less common than locally advanced BCC.
The median age of people in the trials differed. The clinical experts explained that the population in ERIVANCE was substantially younger (median age 62 years) than expected in UK clinical practice (approximately 70 years) and that the population in STEVIE, with a median age of 72 years, was more representative.
The clinical experts stated that there were more patients with Gorlin syndrome in ERIVANCE and STEVIE than would be expected in the UK clinical practice. The committee also noted that patients with Gorlin syndrome had a lower median age, a better baseline performance score and a higher median number of target lesions than those without Gorlin syndrome. This may affect the overall results if not adjusted for.
The committee concluded that STEVIE more closely reflected UK clinical practice than ERIVANCE, and that it should be the basis for decision-making.
3.4 The committee discussed the results from STEVIE, noting that the objective response rate was 68.5% (95% confidence interval [CI] 65.7 to 71.3) in the locally advanced BCC population and 36.9% (95% CI 26.6 to 71.2) in the metastatic BCC population. Median progression-free survival for patients with locally advanced BCC was 23.2 months (95% CI 21.4 to 26.0) and 13.1 months (95% CI 12.0 to 17.7) for patients with metastatic BCC. Results for health-related quality of life, measured using the Skindex-16 questionnaire, indicated that there was no clinically meaningful improvement for metastatic BCC and only clinically meaningful improvement in emotion scores for locally advanced BCC. The committee was aware of the observational nature of this evidence, the higher proportion of patients with Gorlin syndrome than in clinical practice, the lack of long-term data and that data on metastatic BCC were based on very small numbers of patients. The experts agreed that benefits in patients with metastatic BCC were less clear, and are likely to be seen in a small proportion of people with metastatic BCC. But they highlighted that patients with locally advanced BCC had experienced important benefits from vismodegib in practice. The committee noted that overall survival data were immature and were not presented for the locally advanced BCC population. Overall survival data from ERIVANCE were presented for the metastatic BCC population but the committee had previously concluded that this trial was not generalisable to UK clinical practice and also it included only 33 patients with metastatic BCC. The committee noted that vismodegib was not expected to have important overall survival benefits because the condition does not directly affect mortality. The committee concluded that although the clinically relevant benefits associated with vismodegib are plausible, the evidence presented was associated with substantial uncertainty and there were no direct comparative data with best supportive care.
3.5 All patients in ERIVANCE and 98% of patients in STEVIE had an adverse event. The committee noted that there were 110 deaths reported in STEVIE, and that 7 of these related to vismodegib, but it acknowledged that all patients in the study had significant pre-existing risk factors or comorbidities at baseline (as stated by the company). The committee remained concerned about the vismodegib-related deaths in the trial; it heard from the clinical experts that further analysis of the STEVIE safety data is awaited, but in practice vismodegib is generally well tolerated. The committee also noted that adverse events was the most frequent reason reported for discontinuing treatment. The most frequent adverse events reported in both trials were muscle spasms, alopecia, dysgeusia (distortion of sense of taste) and weight loss. One clinical expert stated that although muscle spasms were initially a concern, vismodegib is licensed at a dose that does not affect muscles seriously. Lack of nutrition and weight loss because of dysgeusia are manageable. The committee concluded that the most frequent side effects with vismodegib are manageable, but further analysis of safety data would be informative.
3.6 The committee was aware that the company did a 6‑month landmark analysis comparing vismodegib with best supportive care because there were no trials directly comparing the treatments. The committee noted that data from people who did not respond to treatment ('non-responders') in STEVIE were used to estimate a hazard ratio for people who did respond to treatment ('responders') compared with non-responders. The company then adjusted this hazard ratio to reflect the hazard ratio for non-responders compared with the intention-to-treat patients, as a proxy for vismodegib compared with best supportive care at the 6‑month landmark. The committee had concerns about this use of the non-responder data, considering that this population would have already had vismodegib. It considered the other limitations around the company's landmark approach, highlighted by the ERG:
The ERG stated that adjusting the hazard ratio as described above resulted in it being a time-varying hazard ratio, but evidence of a time-varying treatment effect was not presented. The company stated that this was done to account for the proportion of non-responders changing over time but it acknowledged the methodological limitations of this approach. The ERG explored the effect of using an unadjusted hazard ratio, and the committee preferred this approach.
The ERG stated that the choice of landmark should be made prospectively, based on a clinically meaningful time point. It noted that the company's 6‑month landmark, although not implausible, was chosen retrospectively and other possible landmarks, apart from a 3month exploratory landmark, which the ERG agreed was not appropriate, were not explored. A clinical expert stated that patients typically have a response within 3 months of treatment but the committee considered that further exploration around the landmark would have increased its confidence in the analysis. However, in response to the consultation, the company stated that this was not possible because of the lack of data available.
The ERG noted that the definition of non-responder varied by the outcome assessed, and explored using a consistent definition; the committee agreed that this was more appropriate.
The ERG stated that the company used a common treatment effect hazard ratio for advanced BCC to reflect the uncertainty in the hazard ratios, but the ERG considered this approach to be flawed because locally advanced BCC and metastatic BCC are clinically and prognostically different and should therefore be analysed separately. The clinical experts agreed, and the committee noted that the conditions were also defined independently in the marketing authorisation. The committee agreed that using a common treatment effect hazard ratio was not appropriate. However, the committee was aware of the very small numbers of patients with metastatic BCC. It considered that combined results from the cost-effectiveness model, based on the proportion of patients with locally advanced BCC and metastatic BCC in STEVIE, may be more robust for its decision-making.
The company adjusted the hazard ratios for age and Eastern Cooperative Oncology Group (ECOG) status. The ERG stated that this was not based on a systematic selection process and that other baseline characteristics such as Gorlin syndrome, nerve infiltration and tumour location are important covariates but were not included in the landmark analyses. The ERG explored adjusting for Gorlin syndrome; the committee agreed with this and also considered that further covariate adjustment would have been informative. The committee also noted that the hazard ratios derived from the 6-month landmark varied considerably between people with locally advanced BCC and metastatic BCC. For people with metastatic BCC, the hazard ratio for progression-free survival was less than 1, suggesting that people who do not respond to treatment have better progression-free survival than those who do. The committee heard from the clinical experts that this is implausible. For people with locally advanced BCC, the hazard ratio for overall survival was 2.035; the committee questioned the plausibility of this estimate, because it heard from the clinical experts that vismodegib was unlikely to directly affect survival.
Having fully considered the landmark analysis, the committee stated that the data were at high risk of bias. It questioned the availability of real-world data to supplement this analysis. It heard that there are US and French registries but data from these are not mature. Nonetheless, the committee considered that these data would have been helpful to support the company's comparison of vismodegib and best supportive care. The committee concluded that the results of the landmark analysis were not sufficiently robust for its decision-making.
3.7 The company did a post hoc analysis of the Gorlin syndrome subgroup, as requested by the ERG during the clarification stage. The results, although not statistically significant, suggested that people with Gorlin syndrome have a higher response rate and longer duration of response with vismodegib than people without Gorlin syndrome. The ERG explained that these responses could be linked to the lower age and better baseline performance score of people with Gorlin syndrome, which were not adjusted for in the analysis. The committee was aware of the very small numbers of patients in this subgroup and concluded that it would not be appropriate to consider it separately.
3.8 Having established that the company's landmark analysis was associated with considerable uncertainty, the committee noted that the company's model used the 6‑month landmark analysis results from STEVIE. The resulting incremental cost-effectiveness ratios (ICERs) were therefore highly uncertain. However, the committee agreed that it would consider the results presented. It recalled that the ERG's suggested amendments to the company's landmark analysis were appropriate and it would consider results incorporating these changes:
adjusting for Gorlin syndrome in addition to age and ECOG status
using a consistent definition for non-responders across outcomes (that is, people with stable disease excluding those who have progressed or died before the landmark from the analysis)
using unadjusted hazard ratios.
3.9 The committee noted that the company generally selected the best-fitting model by comparing with the observed Kaplan–Meier data and using the Akaike Information Criterion and Bayesian Information Criterion. However, for time to treatment discontinuation the Weibull curve was chosen despite the log-logistic curve being a better fit. The company stated that this was because the locally advanced BCC and metastatic BCC time to treatment discontinuation log-logistic curves cross in the model. However, the ERG explained that this is caused by the company's approach to modelling the time to treatment discontinuation curves and was not related to the fit of the log-logistic curves. In response to consultation, the company stated that to prevent the locally advanced BCC time to treatment discontinuation log-logistic curve from crossing, it had to be capped to the progression-free survival curve from year 5, suggesting that patients who remained on treatment from year 5 could only discontinue because of progression or death. It considered this to be clinically implausible. The ERG suggested that because only 7% of patients remained on treatment after year 5, and the uncertainty in the long-term predictions of the economic analysis, the log-logistic curve, which is a better fit, would still be preferred. The committee agreed with this and also considered that the smoother drop in the tail of the log-logistic curve more accurately reflected the typically slow progression of the disease. For people with metastatic BCC, the company did not provide any rationale for choosing Weibull over the better fitting log-logistic model, despite the time to treatment discontinuation Weibull curve also crossing. The committee concluded that the log-logistic distribution was a better fit for extrapolating time to treatment discontinuation.
3.10 The committee noted that there was an unusual plateau at the end of the overall survival Kaplan–Meier curve for patients with locally advanced BCC and metastatic BCC. The ERG highlighted that no patients with locally advanced BCC or metastatic BCC would die for 18 or 16 months, respectively, before the end of follow-up, which was 44 months for people with locally advanced BCC and 38 months for people with metastatic BCC. The committee heard from the ERG that this is unlikely considering that by 26 months, people in STEVIE would have been 74 years on average. The committee agreed it was particularly implausible that no patients with metastatic BCC would die for 16 months because the estimated mortality for people with metastatic BCC is 1 to 2 years after diagnosis. The committee concluded that because of the uncertainty around the overall survival data, the extrapolated tails of the overall survival curves would be associated with a high level of uncertainty regardless of the distribution used.
The overall survival curves do not accurately reflect mortality for people with advanced BCC compared with the average UK population
3.11 The committee noted that the overall survival curves suggested an increased mortality risk in people with locally advanced BCC compared with the average age- and sex-matched UK population. However, a clinical expert explained that they would expect the overall survival curve for vismodegib to be close to an age and sex-matched background survival curve for the average UK population because mortality is rarely directly attributable to locally advanced BCC. Additionally, the committee agreed that mortality with metastatic BCC was underestimated by the assumption in the model that people with metastatic BCC would survive for more than 10 years. The committee was aware that the company explored capping the survival curve for vismodegib by the background mortality curve. The ERG explained that this method implies that the benefit between vismodegib and best supportive care diminishes over time and stops after both curves have crossed the background mortality curve (around at month 150). The committee agreed that this was a realistic scenario. On balance, the committee concluded there was substantial uncertainty associated with the evidence around overall survival.
3.12 The committee noted that no algorithm existed for mapping the Skindex-‑16 and MD Anderson Symptom Inventory data, collected during STEVIE, to EQ-5D. Therefore, health state utilities in the base-case analysis were derived from mapping SF‑36 data from ERIVANCE to EQ‑5D. The committee noted that the population baseline age in ERIVANCE (median 62 years) did not reflect patients in STEVIE (median 72 years) or those in UK clinical practice (approximately 70 years). It also heard from the ERG that the assessment of response or progression differed between ERIVANCE and STEVIE and that the underlying SF‑36 data seem to carry a high degree of uncertainty. This is because despite there being mainly non-statistically significant changes in quality of life over time in ERIVANCE measured with the SF‑36, the mapped EQ‑5D utility values suggest a decrease in quality of life over time. The company highlighted that quality-of-life benefits may not be fully captured because the SF‑36 lacks sensitivity. However, the committee recalled that it contains domains on social functioning, anxiety and depression. Nonetheless the committee acknowledged that the results may not fully reflect the feedback from the clinical experts that the main benefit of vismodegib is on the quality of life of patients. After the first committee meeting, the company submitted a scenario analysis to illustrate the potential underestimation of health-related quality of life in the economic analysis. However, the committee considered that this was purely an academic exercise because of the lack of data to inform a most plausible utility value. The ERG also highlighted that the model did not capture disutility from treatment-related adverse events. The committee recognised that the quality-of-life benefits may have been underestimated in the model, but was mindful that in the absence of robust evidence vismodegib's effect on quality of life is uncertain.
3.13 The company assumed that 67% of patients who progress after having vismodegib never have best supportive care, and that post-progression best supportive care for people who have had vismodegib differs from post-progression best supportive care for those who have not had vismodegib. The clinical experts explained that an initial delay in restarting best supportive care after vismodegib was plausible because these patients may initially only need monitoring or a less intensive regimen of best supportive care because they will have a lower disease burden. However, all patients will eventually go on to have best supportive care as their disease progresses, and this regimen will be the same irrespective of previous vismodegib treatment. The committee also heard from the clinical experts that the length of the delay in re-starting best supportive care after vismodegib varies in clinical practice, and depends on the location and type of the BCC. One clinical expert cited a 1- to 2‑year delay; another stated that a delay of 4 to 6 years is plausible after a good response to treatment and with a slow growing tumour. However, as vismodegib has only been available for 4 years, the committee considered that it would be unrealistic to assume a longer delay. On balance, the committee concluded that it would consider results assuming a delay of 3 years after progression before re-starting best supportive care, and this would include the same treatment as people on best supportive care whose disease has progressed.
3.14 The committee's preferred assumptions were:
removing the half-cycle correction from the model
removing the company's adjustment to the hazard ratios derived by the landmark approach to reflect a hazard ratio for non-responders compared with the intention-to-treat population, instead of a non-responders compared with a responders hazard ratio
changing from a Weibull to a log-logistic time to treatment discontinuation curve in the locally advanced BCC and metastatic BCC models
capping the overall survival vismodegib curve by the background mortality curve
assuming that 67% of people on vismodegib who have progressed are initially monitored after progression but then move to best supportive care at 3 years (see section 3.12)
assuming that people on vismodegib moving to best supportive care have the same treatment regimen as people on best supportive care whose disease has progressed.
The ERG also presented 2 further scenarios: assuming no survival benefit with vismodegib, and assuming a survival benefit with vismodegib incorporated using the committee's preferred hazard ratio adjusted for age, ECOG and Gorlin syndrome.
3.15 The committee noted that although vismodegib was unlikely to have a direct impact on survival, it could not rule out survival benefits altogether. However, it considered this was likely to be a small benefit and potentially only in a small population. The committee was mindful of the lack of evidence of a survival benefit (see section 3.2 and section 3.3) and considered that the overall survival benefit assumption incorporated in the model was too optimistic. The committee considered the ICERs based on both assumptions but considered that the true ICER was closer to the estimate assuming no survival benefit with vismodegib. Additionally, the committee recalled its consideration that even though locally advanced BCC and metastatic BCC are distinct conditions it would consider the results of the combined analyses including both populations so as not to disadvantage the very small numbers of patients with metastatic BCC. The committee noted that the combined ICERs were £96,548 per quality-adjusted life year (QALY) gained when a survival benefit was assumed and £4,694,943 per QALY gained when no survival benefit was assumed. It recalled that the economic assessment may not have fully captured the quality-of-life benefits of vismodegib, but taking this into account would not lower the estimate of cost effectiveness to an acceptable level. The committee was also aware that a confidential patient access scheme for vismodegib was available but noted that the ICERs remained substantially above a level that could be considered a cost-effective use of NHS resources.
3.16 The committee discussed the new arrangements for the Cancer Drugs Fund agreed by NICE and NHS England in 2016, noting the addendum to the NICE process and methods guides. The committee agreed that there is substantial uncertainty about the clinical benefit of vismodegib in terms of overall survival and that this was a key driver of the results in the model, but it did not consider that data collection for around 2 years would sufficiently reduce the uncertainty around this. Importantly, the committee did not see any plausible potential for vismodegib to satisfy the criteria for routine use based on its confidential patient access scheme price. It therefore concluded that vismodegib did not meet the criteria to be included in the Cancer Drugs Fund.
3.17 No equality or social value judgement issues were identified.
3.18 The Pharmaceutical Price Regulation Scheme (2014) payment mechanism was not relevant in considering the cost effectiveness of the technology.
3.19 The committee discussed the company's comments about the innovative nature of vismodegib. It heard from the clinical expert that vismodegib is the only treatment option available for advanced BCC that is inappropriate for surgery and radiotherapy. It delays disease progression and improves quality of life. The committee recalled its consideration that the model may not have fully captured the quality-of-life benefits associated with vismodegib. However, the committee considered that accounting for this would not lower the ICERs to an acceptable level, especially considering the uncertainties in the evidence. The committee concluded that vismodegib may be associated with benefits, but more robust comparative data are needed to support its long-term efficacy profile.