3 Evidence

The Appraisal Committee (section 7) considered evidence submitted by Bristol‑Myers Squibb and a review of this submission by the Evidence Review Group (ERG; section 8). See the Committee papers for full details of the evidence.

Clinical effectiveness

3.1 The company presented evidence from 3 ongoing phase III randomised controlled trials (RCTs; CheckMate‑066, CheckMate‑067 and CheckMate‑037). These trials evaluated the clinical effectiveness of nivolumab monotherapy, administered intravenously (IV) every 2 weeks at a dose of 3 mg per kg of body weight. The company also included a phase I dose escalating study (CheckMate‑033) as supporting evidence.

3.2 CheckMate‑066 was a multicentre, international (no centres in the UK), double-blind RCT that compared nivolumab (n=210) with dacarbazine 1000mg/m2 IV every 3 weeks (n=208), in people with untreated advanced melanoma without a BRAF mutation. CheckMate‑067 was a multicentre, international (7 UK centres), double-blind RCT that compared nivolumab monotherapy (n=316) or nivolumab combined with ipilimumab (n=314) with ipilimumab monotherapy 3mg/kg IV every 3 weeks (n=315) in people with untreated advanced melanoma with and without the BRAF mutation. The company did not present results for the nivolumab plus ipilimumab arm because it is outside the scope of this appraisal. CheckMate‑037 was a multicentre, international (5 UK centres), open-label RCT that compared nivolumab (n=272) with the investigators' choice of chemotherapy (n=133), in people with BRAF mutationnegative advanced melanoma that progressed on or after ipilimumab, and BRAF mutationpositive advanced melanoma that progressed on or after ipilimumab and a BRAF inhibitor (vemurafenib or dabrafenib). The investigators' choice of chemotherapy was dacarbazine or carboplatin plus paclitaxel.

3.3 The company stated that baseline demographics and disease characteristics were generally well balanced across the trials, with the exception of a higher proportion of patients with a history of brain metastases (19.5% compared with 13.5%) and elevated LDH (51.1% compared with 34.6%) in the nivolumab arm of CheckMate‑037.

3.4 Overall survival data were only available from CheckMate‑066. In CheckMate‑067 and 037 the minimum follow‑up period was not reached or an insufficient number of events (deaths) had occurred at the time of analysis. Overall survival from CheckMate‑066 was based on an interim analysis at a median follow‑up of 8.9 months in the nivolumab group and 6.8 months in the dacarbazine group. In the nivolumab group 50 out of 210 (23.8%) of patients had died at the time of the analysis therefore median overall survival could not be estimated. Patients in the dacarbazine group had a median survival of 10.8 months. The corresponding hazard ratio for death in the nivolumab group compared with the dacarbazine group was 0.42 (95% confidence interval [CI]: 0.30 to 0.60).

3.5 All 3 trials reported progression‑free survival, defined as the time interval between randomisation and disease progression or death. Nivolumab was associated with statistically significant increases in progression‑free survival, compared with dacarbazine and ipilimumab in CheckMate‑066 and 067 respectively. However, in CheckMate‑037 there was no statistically significant difference in progression‑free survival between nivolumab and the comparator (investigators' choice of chemotherapy); see table 1 for results. The company stated that the results from CheckMate‑037 were confounded by immaturity of the data, imbalances in the prognostic factors between trial groups, high withdrawal rates in the comparator arm and false‑positive progression assessments in the nivolumab arm resulting from the use of Response Evaluation Criteria in Solid Tumours (RECIST criteria).

Table 1 Clinical‑effectiveness outcomes from the CheckMate trials

Outcomes

Nivolumab

Comparator

Hazard ratio (95% CI)

p value

Overall survival

CheckMate‑066 (nivolumab [n=210] vs dacarbazine [n=208])

Events (death) %

23.8

46.2

0.42

(0.30 to 0.60)

<0.001

Median survival (months)

Not reached

10.84

Not applicable

Not applicable

Progression free survival

CheckMate‑066 (nivolumab [n=210] vs dacarbazine [n=208])

Events (death or progression), %

51.4

78.4

0.43

(0.34 to 0.56)

<0.001

Median PFS (months)

5.06

2.17

<0.05

CheckMate‑067 (nivolumab [n=316] vs ipilimumab [n=315])

Events (death or progression) %

55.1

74.3

0.57

(0.43 to 0.76)

<0.001

Median PFS (months)

6.9

2.9

<0.05

CheckMate‑037 (nivolumab [n=122] vs investigators' choice of chemotherapy [n=60])

Events (death or progression) %

58.2

43.3

0.82

(99.99% CI 0.32 to 2.05)

Not significant

Median PFS (months)

4.67

4.24

Not significant

Abbreviations: CI, confidence interval; n, number; PFS, progression free survival; vs, compared with.

3.6 The objective response rate (defined as the proportion of patients with complete or partial response assessed by RECIST criteria) was the primary outcome in CheckMate‑037 and a secondary outcome in CheckMate‑066 and CheckMate‑067. Patients treated with nivolumab had statistically significantly better objective response rate than in the comparator arms in all trials.

3.7 The company included health‑related quality of life results only from CheckMate‑066. EQ‑5D utility index scores and EORTC QLQ‑C30 global health status scores were higher at baseline and for the entire period of observation for nivolumab compared with dacarbazine. However, there was no improvement in quality of life from baseline in the nivolumab arm, or a consistent difference in quality of life between nivolumab and dacarbazine.

3.8 The company presented a series of a priori subgroup analyses from the trials, showing improved effectiveness of nivolumab compared with dacarbazine and ipilimumab across most subgroups. Subgroup analyses by BRAF mutation status in CheckMate‑067 and Checkmate‑037 showed that nivolumab was more effective than the comparator treatments in both subgroups (people with or without BRAF mutation). However, the magnitude of the effect was higher in the subgroup without BRAF mutation. For example, in the nivolumab group of CheckMate‑067, median progression-free survival was 7.89 months in people with BRAF mutation‑negative melanoma and 5.62 months in people with BRAF mutation‑positive melanoma. The median progression-free survival in the ipilimumab group was 2.83 months in people with BRAF mutation‑negative melanoma and 4.04 months in people with BRAF mutation‑positive melanoma. Subgroup analyses based on the expression of programmed death receptor ligand 1or PD‑L1 (defined as PD‑L1 positive if 5% or more cells expressed PD‑L1, and PD‑L1 negative or indeterminate with less than 5% expression) showed nivolumab to be more effective than the comparators regardless of PD‑L1 expression. The results were comparatively better in patients who were PD‑L1 positive than in the patients who were PD‑L1 negative or indeterminate.

3.9 The company also included evidence from a non‑randomised, dose‑escalation study (CheckMate‑003) of nivolumab in patients with solid tumours, including melanoma (n=107), to support the assumption of the maximum treatment duration with nivolumab of 2 years. In patients with advanced melanoma treated with the licensed dose of nivolumab (n=17), median overall survival was 20.3 months and median duration of response to treatment was approximately 2 years.

3.10 The company presented adverse event data from all 3 trials. Fewer people treated with nivolumab had treatment‑related adverse events, particularly of grade 3–4, than those treated with the comparators in all 3 trials. The company also highlighted that in CheckMate‑067 nivolumab was associated with a favourable safety profile compared to ipilimumab, particularly for common immune system related adverse events.

3.11 The company compared the clinical effectiveness of nivolumab, indirectly, with the comparators listed in the scope using 2 separate networks; for BRAF mutation‑negative advanced melanoma (compared with ipilimumab and dacarbazine) and BRAF mutation‑positive advanced melanoma (compared with BRAF inhibitors vemurafenib and dabrafenib). The company used patient‑level data (for nivolumab, dacarbazine and ipilimumab) or estimated 'pseudo' patient-level data (for vemurafenib and dabrafenib) from the trials. It selected the best‑fitting survival function for the outcomes needed for economic modelling after adjusting for covariates for each treatment arm.

Cost effectiveness

3.12 The company submitted a semi‑Markov survival model to estimate the cost effectiveness of nivolumab in people with previously untreated advanced (unresectable, metastatic) melanoma. The economic analyses were presented separately for BRAF mutation‑negative disease (compared with dacarbazine and ipilimumab) and BRAF mutation‑positive disease (compared with dabrafenib, ipilimumab and vemurafenib). The model had 3 health states: pre‑progression, progression and death. Utility in the progression-free and progressed states was subdivided into 2 further states: 30 days or more before death; and less than 30 days before death. For modelling resource use, the entire time horizon was divided in to 4 periods: first year after treatment initiation; second year after treatment initiation; third and subsequent years after treatment initiation; and 12 weeks before death. The model adopted a lifetime time horizon of 40 years and a cycle length of 1 week. The model perspective was NHS and personal social services and costs and benefits were discounted at a rate of 3.5% per year.

3.13 The company based the patient characteristics in the model on CheckMate‑066 for BRAF mutation‑negative disease and from the vemurafenib arm of BRIM‑3 for BRAF mutation‑positive disease. The model allowed subsequent treatment with ipilimumab for people having nivolumab and other comparator treatments except ipilimumab. In the base case, 29.7% and 22.0% people with BRAF mutation‑negative and BRAF mutation‑positive melanoma respectively, had subsequent ipilimumab treatments.

3.14 In the model, the clinical-effectiveness estimates for nivolumab and dacarbazine were based on patient-level data from CheckMate‑066. For ipilimumab, patient-level data from the MDX010‑20 trial were used. MDX010‑20 was a phase III trial that evaluated the efficacy of ipilimumab in people with previously treated advanced melanoma. For BRAF inhibitors, the company identified 2 trials; BRIM‑3 and BREAK‑3. Both were phase III trials that evaluated vemurafenib and dabrafenib respectively, in people with BRAF mutation‑positive advanced melanoma. The company generated pseudo patient‑level data from published Kaplan–Meier curves for BRAF inhibitors based on the vemurafenib BRIM‑3 (base case) or dabrafenib BREAK‑3 (scenario analysis) and assumed that both are equally effective. The company considered the log‑normal and generalised gamma distributions to be the best fit for overall survival and progression‑free survival respectively, for BRAF inhibitors. The company used the same methods for deriving transition probabilities in BRAF mutation‑positive disease as it did in BRAF mutation‑negative disease, except that the baseline patient characteristics for the BRAF mutation‑positive disease were taken from the BRIM‑3 trial.

3.15 Time to progression was modelled using Kaplan–Meier data from CheckMate‑066 (for nivolumab and dacarbazine) and from MDX010‑20 (for ipilimumab) for the first 100 days, followed by fitted parametric curves using the Gompertz distribution in the base case. For pre‑progression survival, the company used Kaplan–Meier data adjusted by covariates for the length of follow-up because none of the fitted curves provided an acceptable visual fit to the observed data.

3.16 The company applied survival data from the trials in the model for the first 3 years for nivolumab and ipilimumab, and for the first 2 years for dacarbazine and BRAF inhibitors. Long‑term overall survival was modelled using the registry data from the American Joint Committee on Cancer for BRAF inhibitors and dacarbazine. For modelling long‑term survival in patients treated with nivolumab and ipilimumab, the company used pooled data on survival from 12 ipilimumab studies as reported by Schadendorf et al (2015) and applied these from year 3 onwards for nivolumab and ipilimumab. The company also applied mortality data for England as background mortality in the model.

3.17 For 'time on treatment' with nivolumab, a log‑logistic parametric curve was fitted to the CheckMate‑066 trial data to calculate the proportion of patients continuing to have nivolumab in each cycle. The base case assumed maximum duration of treatment with nivolumab of 2 years. The model estimated that at 2 years, 23% of patients with BRAF mutation‑negative melanoma and 20% of patients with BRAF mutation‑positive melanoma would still be having nivolumab. The treatment effect of nivolumab was assumed to be maintained on discontinuation of therapy in the base case, based on observational data from CheckMate‑003 and UK clinical expert opinion. This assumption was tested in scenario analyses. For dabrafenib, vemurafenib and dacarbazine the model assumed that treatment would continue until disease progression, in accordance with the marketing authorisations. The company stated that although ipilimumab is usually given for a maximum of 4 doses, patients could have ipilimumab for up to 16 doses (4 doses for the induction and up to 12 further doses if needed, based on the design of MDX010‑20).

3.18 The model included adverse events for endocrine disorder (any grade), diarrhoea (grade 2+) and other adverse events (grade 3+), based on data from CheckMate‑066 for nivolumab and dacarbazine, and CheckMate‑067, BREAK‑3 and BRIM‑3 trials for ipilimumab, dabrafenib and vemurafenib respectively.

3.19 The company used EQ‑5D values from CheckMate‑066, using regression analysis to estimate utility values for health states in the model. The values used for each stage (pre‑progression and post progression) depended upon time to death (30 days or more and less than 30 days):

  • pre‑progression stage and 30 days or more from death, 0.8018

  • pre‑progression stage and less than 30 days from death, 0.7795

  • post‑progression stage and 30 days or more from death, 0.7277

  • post‑progression stage and less than 30 days from death, 0.7054.

3.20 The modelled utility decrements for adverse events were based on Beusterien et al., 2009. These were applied at the start of the model and then periodically to patients who were still on treatment after every 35 weeks.

3.21 The resource use categories in the model were treatment costs, health‑state resource‑use costs and cost for treating adverse events. The same sources were used for estimating these costs in a recent NICE appraisal of ipilimumab for previously untreated advanced melanoma. Resource use for health states was estimated based on the MELODY observational study that collected data on resource use in patients with advanced melanoma. Other costs were sourced from MIMS, NHS reference costs 2013/4, and Personal Social Services Research Unit (PSSRU) 2014.

3.22 The company presented base‑case results using the list prices for all drugs (see tables 2 and 3). In the company's base-case analyses, nivolumab provided a total of 4.31 and 4.27 quality-adjusted life years (QALYs) in the BRAF mutation‑negative melanoma and BRAF mutation‑positive melanoma groups respectively. When compared with ipilimumab the absolute increment in QALY gained with nivolumab was 1.67 and 1.82 for BRAF mutation‑negative melanoma and BRAF mutation‑positive melanoma respectively. The fully incremental comparisons with all comparators demonstrated that for BRAF mutation‑negative melanoma, ipilimumab was extendedly dominated (that is, it had an incremental cost effective ratio [ICER] relative to dacarbazine, higher than that of the next most effective strategy, nivolumab). Nivolumab had an ICER of £23,583 per QALY gained compared with dacarbazine (see table 2). Similarly, in BRAF mutation‑positive melanoma nivolumab dominated (that is, provided more QALYs at lower cost than) both dabrafenib and vemurafenib. Nivolumab was more costly and more effective than ipilimumab, with an ICER of £7346 per QALY gained (see table 3). Because ipilimumab, vemurafenib and dabrafenib are recommended by NICE only with patient access schemes, these results were not used for decision‑making and are included here for illustration only. The ERG re‑ran these analyses incorporating the confidential discounted prices agreed in the patient access schemes for all 3 comparators; these results are commercial in confidence and cannot be reported here.

3.23 The company's deterministic sensitivity analyses showed that results were most sensitive to changes in the parameters defining the fitted parametric curves, time on treatment, utility parameters and administration cost.

ERG comments

3.24 The ERG considered that the CheckMate trials were well designed and well conducted and provide appropriate evidence for the clinical effectiveness of nivolumab. The ERG noted that the results (notably for overall survival) presented by the company were interim and therefore uncertain.

3.25 The ERG agreed with the company that differences between the trials would not allow a meaningful meta-analysis, particularly because there was not a common comparison group. The ERG expressed concern about 2 of the clinical assumptions underlying the indirect treatment comparison; that previous melanoma treatment experience does not have an independent impact on treatment effect in advanced melanoma, and that there is no difference between treatment effects by BRAF mutation status.

3.26 The ERG commented that the structure of the model was consistent with the disease pathway and that the methods applied in the economic analyses were appropriate and followed the methodological guidance stipulated in the NICE reference case. The ERG noted that the company presented economic analyses only for previously untreated melanoma although the marketing authorisation includes people who have had previous treatment.

3.27 The ERG did not agree with all of the company's modelling assumptions and noted that there is considerable uncertainty in the cost‑effectiveness results because of the assumptions made, particularly for long‑term overall survival and time on treatment for nivolumab.

3.28 The ERG did not agree with the company's assumption that patients having nivolumab would have similar long‑term survival as those having ipilimumab. It commented that extrapolation of survival data from CheckMate‑66 would have been the most appropriate method for estimating long‑term survival. In exploratory analyses the ERG extrapolated long‑term survival for nivolumab using a Gompertz distribution in its preferred scenario (see tables 2 and 3).

3.29 The ERG did not agree with the company's choice of survival curve used in the model for time to progression for nivolumab. The ERG suggested that other survival curves (instead of Gompertz) may be plausible for nivolumab, and it used a Weibull distribution (best visual fit) in its preferred scenario (see tables 2 and 3).

3.30 For BRAF mutation‑positive melanoma, the ERG noted that the total cost for the BRAF inhibitors in the model depended on the type of survival curve chosen to model their effect on progression-free survival. The company used a generalised gamma curve; the ERG explored other survival curves for the BRAF inhibitors and considered a log‑normal distribution to be the best fit for its preferred scenario (see table 3).

3.31 The ERG conducted exploratory analyses that included using alternative survival functions for time to progression for nivolumab, and for progression-free survival for BRAF inhibitors (vemurafenib and dabrafenib). The ERG also explored using extrapolated survival data from CheckMate‑066 to model long‑term survival for nivolumab.

3.32 The ERG's preferred scenario included a combination of some of the scenarios mentioned in sections 3.28, 3.29 and 3.30:

  • a Weibull distribution for time to progression for the nivolumab arm

  • a lognormal distribution for progression‑free survival for BRAF inhibitors (vemurafenib and dabrafenib)

  • a Gompertz distribution for extrapolated trial data, for long‑term overall survival for the nivolumab arm.

3.33 The ERG explored the effect of 2 alternative assumptions for maximum treatment duration with nivolumab on its preferred scenario: 3 years, or no maximum treatment duration.

3.34 The results of the ERG exploratory analyses are summarised in tables 2 and 3. Please note that all analyses presented here used the list price for comparators and were not used in the decision making process.

Table 2 Results of the company's base‑case analysis and the ERG's exploratory analyses for BRAF mutation‑negative melanoma (using the list price for all comparators)

Technology

Incremental costs* (£)

Incremental QALYs

ICER (£/QALY)

Company's base-case analysis

Dacarbazine

Ipilimumab

£48,429

1.41

Extendedly dominated by dacarbazine and nivolumab

Nivolumab

£72,578

3.08

£23,583

ERG exploratory analyses

Nivolumab (Weibull for TTP)

£72,237

2.73

£18,117

Nivolumab (extrapolated long‑term OS from trial)

£70,761

2.02

£36,072

ERG preferred scenario (see section 3.32)

£69,725

1.32

Dominated by ipilimumab

ERG preferred scenario + nivolumab for 3 years

£84,257

1.31

Dominated by ipilimumab

ERG preferred scenario +

no maximum treatment duration for nivolumab

£155,177

1.28

Dominated by ipilimumab

Abbreviations: ERG, Evidence Review Group; ICER, incremental cost‑effectiveness ratio; OS, overall survival; QALYs, quality‑adjusted life years; TTP, time to progression.

Dominated: provides fewer QALYs at greater cost than the comparator. Extendedly dominated: a combination of 2 of its comparators provides equal health at a reduced cost.

* These incremental costs do not take account of the confidential discounts agreed in the patient access schemes for dabrafenib, ipilimumab and vemurafenib.

Table 3 Results of the company's base‑case analysis and the ERG's exploratory analyses for BRAF mutation‑positive melanoma (using the list price for all comparators)

Technology

Incremental costs* (£)

Incremental QALYs

ICER (£/QALY)

Company's base-case analysis

Ipilimumab

Nivolumab

£13,374

1.82

£7346

Dabrafenib

£6228

−2.57

Dominated by nivolumab

Vemurafenib

£24,659

−2.56

Dominated by nivolumab

ERG exploratory analyses

Nivolumab (Weibull for TTP)

£13,060

1.48

£8836

Dabrafenib (lognormal for PFS)

£4860

−0.75

Dominated by nivolumab

Vemurafenib (lognormal for PFS)

£19,605

−0.74

Dominated by nivolumab

Nivolumab (extrapolated long‑term OS from trial)

£10,978

0.40

£27,171

ERG preferred scenario (see section 3.32)

£4860

−0.76

Dominated by ipilimumab

ERG preferred scenario + nivolumab for 3 years

£22,574

−0.18

Dominated by ipilimumab

ERG preferred scenario + no maximum treatment duration for nivolumab

£83,858

−0.21

Dominated by ipilimumab

Abbreviations: ERG, Evidence Review Group; ICER, incremental cost‑effectiveness ratio; OS, overall survival; PFS, progression free survival; QALYs, quality‑adjusted life years; TTP, time to progression.

Dominated: provides fewer QALYs at greater cost than the comparator. Extendedly dominated: a combination of 2 of its comparators provides equal health at a reduced cost.

* These incremental costs do not take account of the confidential discounts agreed in the patient access schemes for dabrafenib, ipilimumab and vemurafenib.

3.35 Full details of all the evidence are in the Committee papers.