3 Committee discussion

The evaluation committee considered evidence submitted by Menarini Stemline, a review of this submission by the external assessment group (EAG), and responses from stakeholders. See the committee papers for full details of the evidence.

The condition

3.1

Multiple myeloma is an incurable, relapsing and remitting cancer of plasma cells. Relapsed multiple myeloma refers to previously treated myeloma that has progressed. Refractory refers to multiple myeloma that shows no response to treatment or that has progressed on or within 60 days of the last treatment. The patient experts explained that the complications of multiple myeloma can be significant and debilitating. They added that the relapsing and remitting nature of the condition can have a substantial psychological impact. The clinical experts explained that the main aim of treatment is to:

  • reduce the severity and duration of symptoms

  • reduce morbidity associated with disease progression

  • extend life.

    They noted that the classes of treatment available for multiple myeloma include proteasome inhibitors, immunomodulatory agents and anti‑CD38 monoclonal antibodies. For untreated multiple myeloma, people may have a stem cell transplant. For this evaluation, the relevant population is people who have had at least 4 prior lines of treatment. Their condition also has to be refractory to 2 proteasome inhibitors, 2 immunomodulatory agents and an anti‑CD38 monoclonal antibody ('penta-refractory'). Both the clinical and patient experts noted the lack of effective treatments for penta-refractory multiple myeloma at the fifth or later line of treatment. They described that, currently, people cycle through combinations of the main treatment classes until their condition has relapsed or become refractory. Also, they explained that people often experience significant toxicity and limited efficacy from retreatment with similar classes of treatments. For these reasons, the clinical and patient experts explained that they would welcome another treatment option with a novel mechanism of action. After 4 or more treatments, available options for people with relapsed or refractory multiple myeloma are:

  • pomalidomide plus dexamethasone (see NICE's technology appraisal guidance on pomalidomide for multiple myeloma previously treated with lenalidomide and bortezomib)

  • panobinostat plus bortezomib and dexamethasone (see NICE's technology appraisal guidance on panobinostat for treating multiple myeloma after at least 2 previous treatments)

  • conventional chemotherapy

  • best supportive care (BSC).

Comparators

3.2

NICE's final scope listed all available treatment options as comparators. In its submission, the company considered that BSC was the only appropriate comparator. The company noted that the licence for selinexor specifies that people eligible for treatment must have penta-refractory multiple myeloma. So, it reasoned that pomalidomide plus dexamethasone was not a relevant comparator because treatment with a further immunomodulatory agent would be inappropriate. Similarly, the company did not include panobinostat plus bortezomib and dexamethasone as a comparator because treatment with a further proteasome inhibitor would also be inappropriate. It also noted that, while some people may have chemotherapy, any limited use should be classed under BSC. The EAG generally agreed but considered that panobinostat plus bortezomib and dexamethasone is a treatment option for a limited group of people who have had 4 or more treatments. So, it thought that this should be included as a comparator. The clinical experts explained that panobinostat plus bortezomib and dexamethasone should not be considered a relevant comparator. They agreed with the company that, in people with multiple myeloma that is refractory to 2 proteasome inhibitors, they would not risk retreatment because of toxicity and limited efficacy. The committee concluded that the only relevant comparator for this evaluation was BSC.

Clinical evidence

Clinical trial results for selinexor

3.3

The clinical evidence for selinexor plus dexamethasone came from the STORM trial. This was a phase 2b, single-arm, 2‑part, open-label, multicentre study done in 6 countries (no one was recruited in the UK). The second part of STORM (STORM Part 2) included a prespecified population of people who had selinexor plus dexamethasone for penta-refractory multiple myeloma (n=83). This provided the evidence for this evaluation. The committee considered the outcomes from the final data cut (September 2019), by which time everyone had stopped treatment with selinexor. The primary outcome, overall response rate assessed by an independent review committee, was 25.3% (95% confidence interval [CI] 16.4 to 36.0). Median progression-free survival was 2.8 months (95% CI 1.9 to 4.3), and median overall survival was 8.4 months (95% CI 5.9 to 11.2). Given that STORM Part 2 was a single-arm trial, estimates of comparative effectiveness were derived using an indirect treatment comparison (see section 3.6).

Clinical trial results for BSC

3.4

The company's systematic literature review did not identify any head-to-head evidence comparing selinexor with BSC in people with penta-refractory multiple myeloma. Because of this, the company used evidence on standard care from the MAMMOTH study to inform efficacy estimates for BSC. MAMMOTH was a multicentre, retrospective, observational, cohort study done in the US. A subset of people in MAMMOTH had penta-refractory multiple myeloma. Ninety percent of this group had treatments after becoming penta-refractory, including immunomodulatory agents, proteasome inhibitors and anti‑CD38 monoclonal antibodies. The median overall survival of people with penta-refractory multiple myeloma in MAMMOTH was 5.6 months (95% CI 3.5 to 7.8).

Generalisability of the clinical evidence

3.5

The EAG noted that, in STORM Part 2, people were younger than the average age at which people in the NHS would be expected to reach penta-refractory status. It also highlighted that, compared with people with penta-refractory multiple myeloma having treatment in the NHS, people in STORM Part 2 had:

  • better Eastern Cooperative Oncology Group (ECOG) performance status

  • a higher proportion of stem cell transplants

  • more prior treatments.

    The EAG expressed similar concerns about the people in MAMMOTH. The clinical experts explained that the people in STORM Part 2 and MAMMOTH were not too dissimilar from the people they see in clinical practice. They explained that the only people with penta-refractory multiple myeloma considered for treatment at fifth line or later will likely be younger and have better ECOG performance status than people not considered for treatment. This is because of the toxicity associated with several treatment lines for multiple myeloma. The NHS England Cancer Drugs Fund lead clarified that only people with an ECOG performance status of 0 to 2 would be eligible for selinexor plus dexamethasone in the NHS. This aligns with the people included in STORM Part 2. The EAG considered that use of subsequent active treatments in the penta-refractory population in MAMMOTH was higher than expected for people on BSC. It thought that these people had had subsequent treatments that would not be routinely commissioned in the NHS. The EAG also noted the subsequent active treatment use in STORM Part 2. But the company considers the details of these to be confidential, so they cannot be reported here. The clinical experts considered that retreatment with previously used regimens would have limited efficacy. So, they thought this would not have biased the results of STORM Part 2 and MAMMOTH. The committee concluded that there were differences between the people with penta-refractory multiple myeloma in NHS clinical practice and the people included in STORM Part 2 and MAMMOTH. It also concluded that there were differences between the people included in STORM Part 2 and the people included in MAMMOTH. But it considered that these studies provided the best source of evidence available and accepted that they were appropriate for decision making.

Comparative effectiveness

3.6

In the absence of head-to-head data, the company attempted 3 unanchored matching-adjusted indirect comparisons (MAICs) based on various prognostic factors and treatment effect modifiers. The populations could not be matched in 1 MAIC, so results were available for 2. The company noted that these had effective sample sizes of less than 14 people. It considered that these were too low to be able to draw robust conclusions about comparative efficacy. It also highlighted the exclusion of important effect modifiers and prognostic factors from the MAICs. The company also noted that the method for MAICs exacerbated the generalisability issues discussed in section 3.5. It then completed a simulated treatment comparison (STC) to mitigate against the small effective sample sizes in the MAICs. The STC was used to derive a hazard ratio for overall survival of 0.43 in favour of selinexor plus dexamethasone. The EAG had several concerns about the STC, including:

  • uncertainty in the regression models

  • lack of clinical plausibility

  • that the assumption of proportional hazards was violated.

    The EAG considered that neither the MAIC nor STC approaches were robust for estimating comparative efficacy. But it thought that 1 of the MAICs was the most reasonable and least biased of the available options. This was the 'full' MAIC, which matched for the most prognostic factors and effect modifiers possible between the MAMMOTH and STORM Part 2 populations. The committee agreed that there was significant uncertainty in the methods used to estimate comparative efficacy, and that this uncertainty was unresolvable with the available data. But it was willing to accept the EAG's approach of using the 'full' MAIC, provided that the uncertainty was adequately reflected in the incremental cost-effectiveness ratio (ICER) threshold.

Economic model

Company's modelling approach

3.7

The company presented a partitioned survival model with 3 health states: progression free (including on or off treatment), progressed disease and death. The probability of being in each health state was calculated using the progression-free and overall survival curves. The time-on-treatment curve was used to determine those on or off treatment. The model had a 30‑year (lifetime) horizon, a cycle length of 1 week and applied a half-cycle correction. The EAG was generally satisfied with the modelling approach. The committee concluded that the model was acceptable for decision making.

Overall survival

Company's approach
3.8

The company modelled overall survival by fitting standard parametric survival curves to the Kaplan–Meier (KM) data from STORM Part 2. Based on clinical advice, the company selected the Log-normal curve, on the expectation that about 5% of people having treatment with selinexor plus dexamethasone would be alive at 5 years. The company then applied the hazard ratio derived from the STC to model the BSC curve (see section 3.6). This method resulted in 5‑year survival estimates of 6.4% for selinexor plus dexamethasone and 0.2% for BSC. The EAG considered that the survival estimate for selinexor plus dexamethasone was implausible. This was because the modelled treatment duration of selinexor plus dexamethasone was just 2.5 months and the modelled progression-free survival was just 3.8 months. Also, clinical advice to the EAG suggested that everyone would be expected to have died at 5 years. So, the EAG considered that overall survival was overestimated in the company's approach.

EAG's approach
3.9

At the EAG's request, the company fitted independent curves to the 'full' MAIC‑adjusted selinexor KM data from STORM Part 2 and to the unadjusted BSC KM data from MAMMOTH. The EAG selected the Weibull curve for both arms based on survival estimates and because the curves crossed at about 3.5 months, which was similar to the naive KM data. This method resulted in 5‑year survival estimates of 4.4% for selinexor plus dexamethasone and 0% for BSC. The company repeated its concerns with this method of using the 'full' MAIC to estimate overall survival (see section 3.6). The EAG explained that this was likely an 'optimistic' approach. This was because the Weibull curve was higher than the selinexor plus dexamethasone KM data after about 3.5 months, suggesting that it overestimated survival. The EAG also modelled a 'pessimistic' scenario, which:

  • extrapolated the unadjusted selinexor KM data with the Weibull curve

  • applied the hazard ratio derived from the 'full' MAIC to model BSC

  • assumed no differential treatment effect for 7 months because the 'full' MAIC-adjusted selinexor KM curve overlapped with the unadjusted BSC KM curve for the first 7 months.

Committee's preferred approach
3.10

The committee discussed the different approaches presented to estimate overall survival. The clinical experts explained that they would expect 1‑year survival to be about 40% with selinexor and about 10% with BSC. They also said that they would expect 5‑year survival to be less than 5% with selinexor and to be 0% with BSC. The committee considered that neither the company's nor EAG's preferred methods satisfied these criteria. In addition, the committee agreed with the EAG that, given the limited time on treatment and progression-free survival, the company's overall survival benefit of selinexor plus dexamethasone was unlikely. The committee concluded that all estimates of overall survival were highly uncertain. It considered that it had not been presented with an accurate assessment of the difference in overall survival of people who had selinexor plus dexamethasone compared with people who had BSC. The committee disagreed with the company's approach because it did not sufficiently capture the overlapping nature of the KM curves. The committee also agreed with the EAG that the proportional hazards assumption did not hold. The committee rejected the EAG's 'pessimistic' scenario because the clinical experts did not think an initial period of no differential treatment effect was clinically plausible. The committee noted the EAG's view that the uncertainty in modelling was unresolvable with the available data. It recalled the issues that the company raised with the 'full' MAIC. It concluded that it likely overpredicted survival and led to optimistic cost-effectiveness estimates. But it thought that it was the best approach available.

Severity

3.11

The company provided evidence that penta-refractory multiple myeloma is a severe condition. The committee considered the severity of the condition (the future health lost by people living with the condition and having standard care in the NHS). A committee may apply a greater weight (a severity modifier) to quality-adjusted life years (QALYs) if technologies are indicated for conditions with a high degree of severity. By this mechanism, the committee can account for the unmet need associated with the condition. The company provided absolute and proportional QALY shortfall estimates in line with NICE's health technology evaluations manual. To inform the baseline characteristics in the QALY shortfall calculations, the company used the mean age and sex distribution from STORM Part 2. The company used the total QALYs from the model for the BSC arm to inform the total expected QALYs for people with the condition on standard care. QALYs in the general population were estimated using the approach and sources recommended by Schneider et al. (2021). Using its base-case assumptions, the company considered that a QALY weighting of 1.7 should apply because of the proportional QALY shortfall result. The committee recalled the EAG's critique that people in STORM Part 2 were likely younger than the population expected to be eligible for selinexor plus dexamethasone in the NHS. Increasing the starting age of people in the calculation decreased the QALYs accrued by the general population. But the committee recalled that the clinical experts thought that a starting age of about 70 years would be appropriate. Under the committee's preferred assumptions, with a starting age of 70 years, the severity modifier was 1.7. So, the committee concluded that the 1.7 severity modifier should be applied to the cost-effectiveness estimates. The case for applying the severity modifier was not met in NICE's technology appraisal guidance on selinexor with bortezomib and dexamethasone for previously treated multiple myeloma. This was because of the differences in standard care at the different points in the treatment pathway.

Cost-effectiveness estimates

Company and EAG cost-effectiveness estimates

3.12

The company's probabilistic base-case ICER, including the severity weighting of 1.7, was £24,009 per QALY gained. When including the severity weighting of 1.7, the EAG's probabilistic optimistic ICER was £31,701 per QALY gained and its pessimistic ICER was £73,206 per QALY gained. The committee discussed the wide range of ICERs presented and concluded that there was significant uncertainty in the cost-effectiveness estimates. It agreed that this uncertainty was driven by the unresolvable issues relating to the modelling of comparative efficacy and overall survival.

Committee's preferred assumptions

3.13

The committee's preferred assumptions were:

  • BSC was the only appropriate comparator and panobinostat plus bortezomib and dexamethasone should not be considered (see section 3.2).

  • Data from STORM Part 2 and MAMMOTH, although different from the expected population with penta-refractory multiple myeloma in NHS, could be used for decision making (see section 3.3, section 3.4 and section 3.5).

  • The EAG's 'optimistic' analysis for estimating comparative efficacy and modelling overall survival could be used for decision making (see section 3.6, section 3.8, section 3.9 and section 3.10).

  • Including the EAG's 5 additional assumptions about resource use, administration costs for oral chemotherapy, cyclophosphamide dosage, adverse event costs, and end of life care costs was appropriate. The cumulative effects of these 5 scenarios had a minimal effect on the ICER and were all accepted by the committee.

    The committee's preferred assumptions, with the 1.7 severity weighting applied, resulted in an ICER of £31,701 per QALY gained. The committee noted that, given the uncertainty around the overall survival estimates, the true ICER was likely higher than this. But, acknowledging the novel mechanism of selinexor, it was willing to consider this as the most plausible ICER. Because of this committee-preferred ICER, the company entered into a commercial arrangement that offered a discount on the price of selinexor (see section 3.17).

Acceptable ICER

3.14

NICE's health technology evaluations manual notes that judgements about the acceptability of a technology as an effective use of NHS resources will take into account the degree of certainty around the ICER. The committee will be more cautious about recommending a technology if it is less certain about the ICERs presented. Because of the uncertainty in the cost-effectiveness estimates, the committee was prepared to accept an ICER of around £20,000 per QALY gained as a cost-effective use of NHS resources.

Other factors

Equality

3.15

The committee noted that the risk of multiple myeloma is higher in men than in women, in older people, and in people of African and Caribbean ethnic backgrounds. The committee considered that its recommendation applied equally, regardless of sex, age or ethnic background. It concluded that these differences in prevalence did not itself represent an equality issue in this evaluation.

Innovation

3.16

The committee considered whether selinexor plus dexamethasone was innovative. The clinical experts considered that selinexor provided an alternative option with a novel mechanism of action. The committee considered that people with penta-refractory multiple myeloma would value an alternative oral treatment with a different mechanism of action. It concluded that there were benefits of selinexor at the fifth and later lines of treatment which were uncaptured in the economic modelling. It accounted for this by accepting an ICER that it considered optimistic as the preferred ICER.

Conclusion

3.17

The committee concluded that, using its preferred assumptions and with the 1.7 severity weighting applied, the ICER for selinexor plus dexamethasone was above the threshold normally considered to be a cost-effective use of NHS resources. But the company agreed a commercial arrangement that decreased the ICER to below this threshold. So, the committee recommended selinexor plus dexamethasone as a treatment option for people with multiple myeloma who have had 4 or more treatments.

  • National Institute for Health and Care Excellence (NICE)