3 Committee discussion

The appraisal committee considered evidence submitted by Bristol–Myers Squibb, a review of this submission by the evidence review group (ERG), NICE's technical report, and responses from stakeholders. See the committee papers for full details of the evidence.

The appraisal committee was aware that 3 issues were resolved during the technical engagement stage, and agreed that:

  • The model time horizon (issue 7, see technical report page 8) used by the company in the economic model of 40 years was sufficient to capture data for everyone having nivolumab or taxanes.

  • Nivolumab is likely to improve overall survival by at least 3 months (issue 13, see technical report page 14), meeting the second criterion for end of life treatment.

  • The approach used to calculate the cost of monitoring response to treatment (issue 12, see technical report page 13) was appropriate.

Clinical need

People would welcome a new treatment option

3.1 The clinical experts explained that people with unresectable advanced, recurrent or metastatic oesophageal squamous cell carcinoma, whose disease has progressed after fluoropyrimidine and platinum-based combination therapy, have a poor prognosis and no curative treatment options. It disproportionately affects people from lower socioeconomic backgrounds, and smoking and alcohol consumption are risk factors. The taxanes paclitaxel and docetaxel are standard treatment for most people and weekly or 3‑weekly hospital visits are needed for infusions. People often feel unwell and may experience debilitating fatigue and loss of appetite. Many people find the weekly or 3‑weekly treatment regimens difficult to tolerate because of the associated adverse events. Frequent blood tests are needed to monitor neutropenia. The NHS England clinical lead noted that taxanes have limited efficacy and people are often not well enough to have third-line treatment if taxanes do not control the disease. People who are unable to tolerate taxane chemotherapy have best supportive care, which has no effect on disease progression. Older people are less likely to tolerate chemotherapy, and about 40% of people diagnosed with squamous oesophageal cancer are over 75. The committee recognised the unmet need for a treatment with lower toxicity than chemotherapy, that provides long-term benefit and improves quality of life. The clinical expert explained that if people are not well enough to tolerate taxane therapy, they are unlikely to be well enough to tolerate nivolumab. Although immunotherapy is generally better tolerated than chemotherapy, it still carries risks, notably immune-related side effects. The committee concluded that patients and clinicians would welcome an effective treatment that is better tolerated, particularly if it offers an option of further third-line treatment after disease progression.

Trial design

The ATTRACTION‑3 study is appropriate for estimating clinical effectiveness

3.2 The company's clinical evidence came from ATTRACTION‑3. This included people with unresectable oesophageal squamous cell carcinoma whose disease was refractory or who were intolerant to combination therapy with fluoropyrimidine and platinum-based drugs, and who had a life expectancy of at least 3 months. People were randomly assigned to have either nivolumab or taxane chemotherapy. Disease was monitored every 6 weeks and assessed using RECIST 1.1 criteria. People could continue treatment after first disease progression in both treatment groups, based on the investigators' judgement. The clinical expert explained that immunotherapies are associated with pseudo-progression, which is a distinct radiological pattern of apparent progression from baseline that is not confirmed with subsequent assessment. For this reason, if there is evidence of progression but the person feels well, they usually continue having nivolumab for another cycle and then radiological progression is assessed at the next monitoring appointment. The committee concluded that ATTRACTION‑3 was an appropriate source of clinical data and could be used for estimating clinical effectiveness.

Clinical evidence

The results from ATTRACTION-3 are generalisable to people in the NHS

3.3 ATTRACTION‑3 was done in the US, Europe and Asia. Of the people included in the study, 96% had an Asian family background, and two-thirds of these people had a Japanese family background. Oesophageal squamous cell cancer is more prevalent in Asia than in Western countries. The clinical expert commented that although the trials were mainly done in Asia, there is no difference in the underlying biology of oesophageal squamous cell cancer compared with people in the UK. Also, treatment is similar because of consensus in the management of advanced oesophageal cancer. The company accepted that the population in the clinical trial was generally younger and fitter (with an Eastern Cooperative Oncology Group [ECOG] performance status of 0 to 1) than the population seen in NHS practice. The committee agreed with the clinical expert and concluded that the clinical trial was broadly generalisable to people with advanced oesophageal squamous cell cancer in the UK.

Nivolumab improves overall survival but disease progresses faster in the first 3 months of treatment

3.4 Nivolumab is associated with a difference in median overall survival of 2.4 months compared with the combined taxane therapy arm (median overall survival 10.91 months for nivolumab, 8.51 months in the taxane arm). However, median progression-free survival was slightly lower for nivolumab (1.68 months compared with 3.35 months), as was the overall response rate (19.3% compared with 21.5%). The 36‑month follow‑up data from ATTRACTION‑3 confirmed the overall survival benefit seen at 24 months. More people had disease progression with nivolumab than with taxanes, and most of the overall survival benefit from nivolumab was after progression. The committee questioned why the benefit was predominantly seen after progression rather than before, which is what would be expected if nivolumab had the potential to be curative. It discussed whether this could be because of people having nivolumab after disease progression and it slowing progression; a carry-over effect after stopping nivolumab into the progression phase; or because people remained well enough for follow-on therapies at progression. The committee concluded that it was unclear why the survival benefit mainly happened after disease progression.

People are at more risk of dying having nivolumab in the first 3 months

3.5 Results up to 36 months for overall survival were provided by the company and analysed by the ERG. At 2 months and 4 months, people having nivolumab had worse overall survival than people having taxanes. However, from 6 months onwards overall survival was higher for nivolumab compared with taxanes (the data cannot be reported here because the company submitted it as academic in confidence). The clinical expert explained that this pattern in overall survival is commonly found with immunotherapies. This is because of the delay in benefit as the immune system is activated, while chemotherapy immediately acts on the cancer cells. The higher death rate in the first 3 months seen with nivolumab was particularly concerning because people in ATTRACTION‑3 were expected to survive at least 3 months. The NHS England clinical lead suggested that people generally have worse performance scores in the NHS than in the trial. In clinical practice, it is possible to distinguish between people who are and are not likely to tolerate nivolumab therapy. Based on the available data, the committee concluded that nivolumab improves overall survival despite a greater death rate in the first 3 months.

Adverse events

Nivolumab is better tolerated than taxanes, but immunotherapies can cause significant side effects

3.6 Fewer patients experienced drug-related adverse events in the nivolumab group compared with taxanes in the clinical trial (the data cannot be reported here because the company submitted it as academic in confidence). The clinical experts agreed that nivolumab is better tolerated than taxanes, and that taxane therapy can be associated with long-term adverse events, such as neuropathy of the hands and feet. The NHS England clinical lead noted that nivolumab is also associated with rare but potentially life-threatening gastrointestinal, renal, endocrine and hepatic adverse events. The clinical expert commented that there are standard guidelines for managing immunotoxicity associated with treatments like nivolumab, which are well managed in clinical practice. The committee concluded that nivolumab is better tolerated than taxanes, but immunotherapies can cause significant immune-related side effects.


Taxane chemotherapy is the relevant comparator

3.7 The clinical trial compared nivolumab with a combined taxane arm (paclitaxel and docetaxel). The clinical experts and the NHS England clinical lead agreed that there is a class effect for taxanes, both in efficacy and side-effect profile. Best supportive care was not considered to be a relevant comparator, because people who are not well enough to tolerate taxane therapy are unlikely to benefit from nivolumab. The committee concluded that the relevant comparator for nivolumab therapy is taxane chemotherapy.

Cost effectiveness

There is uncertainty over the method of extrapolating overall survival

3.8 The company used a semi-parametric approach to model overall survival to capture the changing risk of death over time with nivolumab treatment. Kaplan−Meier curves from the trial were used in both groups up to 5.75 months, based on the ERG's preferred cut-point for 24‑month data. After this, the company used a log-logistic distribution in the nivolumab arm and a Weibull distribution in the taxane arm. The ERG critique was based on visual inspection of the extrapolation. This was because it had not had the opportunity to critique each extrapolation to determine the most appropriate method for each arm or calculate how the selected extrapolations affected the cost effectiveness of nivolumab. The ERG noted that the nivolumab extrapolation seemed to fit the trial data well. But, it advised that the taxane extrapolation was not a good fit to the Kaplan−Meier data and underestimated long-term overall survival of patients having taxane therapy. The committee noted that the ERG may have preferred alternative extrapolations from different cut-points than those proposed by the company if it had been able to fully critique the extrapolations of trial data. The committee concluded that there is substantial uncertainty over the most appropriate method of extrapolating overall survival in the nivolumab and taxane arm.

No adjustment was made to efficacy or additional costs of third-line therapy

3.9 In the clinical trial, patients were able to continue initial treatment (see section 3.2) and have subsequent treatment (surgery, radiotherapy or pharmacotherapy) after disease progression. The proportion of people having subsequent therapy after progression was similar in both the nivolumab and taxane groups. However, more people in the nivolumab arm continued having their initial treatment, compared with the taxane arm. The clinical expert explained that nivolumab may be continued after disease progression until the next scheduled scan confirms that the disease has progressed, but treatment would be stopped when progression was confirmed. However, because it is better tolerated than taxanes, more people would be able to have further active treatment after nivolumab than after taxanes. The company did an exploratory analysis of overall survival, which censored people having subsequent therapy. The results showed that having subsequent therapy does not have a big effect on the overall survival of nivolumab compared with taxanes. The committee recognised that the opportunity for active third-line treatment is an important consideration for patients. It concluded that nivolumab would be more likely to be continued in the short term after progression than taxanes, as seen in the trial. It is not possible to tell whether any differences between the third-line treatments in ATTRACTION‑3 and in the NHS would affect the relative effectiveness of nivolumab in the NHS compared with the trial.

Utility values

Post-progression utility should be the same in the nivolumab and taxane arms

3.10 The company estimated the utilities before and after progression using a statistical model fit to EQ‑5D data from the clinical trial, with missing values imputed under the assumption that they were missing at random. Baseline utility was worse in the taxane arm compared with the nivolumab arm, but this difference was not adjusted for. Nivolumab had a higher utility before progression than taxanes because of its more favourable safety profile (the data cannot be reported here because the company submitted it as academic in confidence). The company model also assumed a higher utility after progression for nivolumab compared with taxanes because of the continued benefit of nivolumab. The ERG considered it plausible that the pre-progression utility would be higher for nivolumab than taxanes because of the improved adverse event profile. But, it questioned the size of the difference because differences in baseline utility had not been adjusted for. It provided an estimate based on values from an alternative statistical model fit by the company that did not include imputation of missing values. For post-progression utility, the ERG did not consider there to be enough justification for a post-progression utility benefit with nivolumab compared with taxanes. Instead, it used a pooled estimate of utility in the nivolumab and taxane arms, giving equal utility values for both treatments. The company also provided a scenario analysis that varied pre-progression utility values according to the company and ERG preferences, and post-progression utility values based on pooled and non-pooled estimates. The committee considered it plausible that the utility before progression for nivolumab was higher than for taxanes, based on differences in tolerability and adverse events. In the post-progression phase, the NHS England clinical lead advised that a constant utility after progression was not plausible. This is because, in reality, utility will fluctuate over time and can be influenced by the choice of follow-on treatments. The choice of utility values had a significant effect on the incremental cost‑effectiveness ratio (ICER). The committee concluded that a differential utility before progression was reasonable, but the size of difference was likely to have been overestimated by the company. The post-progression utility in the short term after nivolumab treatment could be higher than after taxanes because of less spill over of toxic effects. It was unlikely to be better for the whole time that the disease was progressing from when treatment stopped up to the time of the patient's death. The most realistic scenario was for post-progression utility to be the same for nivolumab and taxane therapy.


The company's model underestimates the cost of hospitalisation

3.11 The company estimated the cost of each episode of hospitalisation at £534.07 based on an average of 1 bed day per person. The ERG did not consider this method appropriate, instead using the cost of full length of hospitalisation without adjusting for the length of stay. This increased the cost of hospitalisation to £3,379.73. The committee noted that this remains an uncertainty that has a substantial effect on the ICER, and that the company had not given adequate justification for the estimate of hospital costs based on the duration of stay of 1 bed day. The NHS clinical lead commented that patients could be admitted for short periods for procedures such as oesophageal stenting. However, people who had to be admitted because of toxicity from either taxanes or nivolumab would be too ill to be discharged after 1 day. The committee concluded that there remained uncertainty about the average cost of hospitalisation related to the length of hospital stay. However, the most realistic estimate of hospitalisation costs was likely to be between the company and the ERG's preferred cost calculation.

Taking into account the updated commercial access arrangement, nivolumab is likely to be cost effective

3.12 There were uncertainties remaining in the model, particularly related to the extrapolation of overall survival and time on treatment, which the ERG was unable to critique. The committee considered the ERG administration costs and utilities before and after progression to be the most appropriate (see section 3.10). There was still substantial uncertainty about the hospitalisation costs for nivolumab compared with taxanes (see section 3.11). At its second meeting after consultation, the committee noted that the company base-case ICER was £48,205 per quality‑adjusted life year (QALY) gained. The ERG provided analyses of the effect of its preferred assumptions for utility, administration and hospitalisation costs on the company's base-case ICER. These resulted in ICERs that exceeded what NICE considers a cost-effective use of NHS resources even for technologies given special consideration as life-extending treatments for people with a short life expectancy. After the second committee meeting the company updated its commercial arrangement and submitted an updated analysis using the assumptions preferred by the ERG for utilities and administration costs. It also gave a range of hospitalisation costs including both the ERG- and company-preferred estimates. The resulting range of ICERs cannot be reported here because the commercial arrangement is confidential. The committee noted that the commercial arrangement reduced the ICERs so that, other than when the ERG‑preferred hospitalisation costs were used, the ICERs were in the range that could be considered a cost-effective use of NHS resources. The committee was aware that there was remaining uncertainty about the most appropriate extrapolation of overall survival and time on treatment, which could also affect the ICER. It concluded that incorporating the commercial arrangement meant that most of the ICERs were in the range that could be considered cost effective, even though some uncertainties remained.

End of life

Nivolumab meets the end of life criteria

3.13 The committee considered the advice about life-extending treatments for people with a short life expectancy in NICE's guide to the methods of technology appraisal. It considered whether nivolumab meets the end of life criteria for people with unresectable, advanced or recurrent oesophageal cancer who have had fluoropyrimidine and platinum-based therapy. The company and ERG both agreed based on their analyses that life expectancy in this population is less than 24 months. The committee concluded that nivolumab was indicated for people with a short life expectancy. The observed median overall survival benefit with nivolumab of 2.58 months was extrapolated. This gave an expected overall mean survival benefit of 7.8 months in the company's base-case model and 4.0 months in the ERG model. The committee considered that the extension-to-life criterion was met based on the trial data.


  • National Institute for Health and Care Excellence (NICE)