Dimethyl fumarate for treating relapsing‑remitting multiple sclerosis

Clinical effectiveness

3.1 The manufacturer conducted a systematic review of the literature to identify studies evaluating the clinical effectiveness and safety of dimethyl fumarate for treating adults with relapsing‑remitting multiple sclerosis. It identified 2 phase III randomised controlled trials (RCTs): DEFINE and CONFIRM.

3.2 The DEFINE trial was an international multicentre (198 centres in 28 countries) double‑blind phase III RCT in 1237 adults with relapsing‑remitting multiple sclerosis. Patients were stratified by geographical region and randomised in a 1:1:1 ratio to dimethyl fumarate 240 mg twice daily (n=410), dimethyl fumarate 240 mg 3 times daily (n=416) or placebo (n=408). The CONFIRM trial was an international multicentre (200 centres in 28 countries) double‑blind phase III RCT in 1430 adults with relapsing‑remitting multiple sclerosis. Patients were stratified by geographical region and randomised in a 1:1:1:1 ratio to dimethyl fumarate 240 mg twice daily (n=359), dimethyl fumarate 240 mg 3 times daily (n=345), glatiramer acetate 20 mg once daily (n=350; open‑label) or placebo (n=363). In both the DEFINE and CONFIRM trials, patients were treated for 96 weeks and had a follow‑up visit at 100 weeks if they completed treatment. Patients stopped treatment if they did not tolerate the study drug or withdrew consent. For dimethyl fumarate, only data relating to the licensed dosage (240 mg twice daily) were presented in the manufacturer's submission.

3.3 Patients were eligible for inclusion in the DEFINE and CONFIRM trials if they were aged between 18 and 55 years, had a diagnosis of relapsing‑remitting multiple sclerosis confirmed by the McDonald criteria, had an Expanded Disability Status Scale (EDSS) score of between 0 and 5 inclusive (the EDSS ranges from 0 to 10 in 0.5‑unit increments, higher scores representing higher levels of disability) and had either had at least 1 relapse during the previous year and a previous MRI scan showing lesions consistent with multiple sclerosis, or had gadolinium‑enhancing lesions on MRI scans done within 6 weeks of randomisation. The manufacturer noted there were no significant differences in baseline characteristics between the treatment groups of the DEFINE and CONFIRM trials. Most patients were white (79% in DEFINE, 84% in CONFIRM) and were women (74% in DEFINE, 70% in CONFIRM). The mean age of patients was 38.5 years and 37.3 years in the DEFINE and CONFIRM trials respectively. In the DEFINE trial, 29 patients were treated at 7 UK centres. The CONFIRM trial did not include any UK centres.

3.4 The primary outcome measures in the trials were the proportion of patients with a relapse at 2 years (DEFINE) and the annualised relapse rate at 2 years (CONFIRM). Relapses were defined as new or recurrent neurological symptoms not associated with fever or infection, lasting 24 hours or longer, and with new objective neurological findings. An intention‑to‑treat population was the primary population for the analysis of efficacy outcomes in both trials adjusted for age, EDSS score, number of relapses in the year before randomisation and geographical region using proportional hazards regression in the DEFINE trial, and negative binomial regression in the CONFIRM trial. In the DEFINE trial, the proportion of patients with a relapse at 2 years was statistically significantly reduced with dimethyl fumarate compared with placebo (27% versus 46%; hazard ratio [HR] 0.51, 95% confidence interval [CI] 0.40 to 0.66). In the CONFIRM trial, the annualised relapse rate at 2 years was 0.22 with dimethyl fumarate and 0.40 with placebo (relative risk [RR] 0.56, 95% CI 0.42 to 0.74), and 0.29 with glatiramer acetate (RR compared with dimethyl fumarate not reported); the difference between dimethyl fumarate and placebo was statistically significant. The CONFIRM trial was not powered to detect differences between dimethyl fumarate and glatiramer acetate (active comparator). The manufacturer performed a number of sensitivity analyses that supported the results of the primary efficacy analysis of the DEFINE and CONFIRM trials comparing dimethyl fumarate with placebo.

3.5 Secondary outcomes reported annualised relapse rate at 2 years (DEFINE), the proportion of patients with a relapse at 2 years (CONFIRM), and in both trials, progression of disability on the EDSS and number of MRI lesions (in a subset of patients) at 2 years. In the DEFINE trial, the annualised relapse rate at 2 years was statistically significantly reduced with dimethyl fumarate compared with placebo (0.17 compared with 0.36; RR 0.47, 95% CI 0.37 to 0.61). In the CONFIRM trial, the proportion of patients with a relapse at 2 years was statistically significantly reduced with dimethyl fumarate compared with placebo (29% compared with 41%; HR 0.66, 95% CI 0.51 to 0.86). Patients taking dimethyl fumarate had a statistically significantly reduced risk of disability progression sustained for 3 months at 2 years compared with those taking placebo in the DEFINE trial (16% compared with 27%; HR 0.62, 95% CI 0.44 to 0.87). However, in the CONFIRM trial, the difference in the risk of disability was not statistically significant (13% compared with 17%; HR 0.79, 95% CI 0.52 to 1.19). The manufacturer suggested that the 3‑month disability progression results in the CONFIRM trial may have been affected by the fact that the proportion of patients censored (whose disability may or may not have progressed) was higher in the placebo arm than in the dimethyl fumarate arm. It explained that patients were censored if they withdrew from the study or switched treatments before 3‑month progression could be confirmed. Analyses of disability progression sustained for 6 months at 2 years comparing dimethyl fumarate with placebo in DEFINE (HR 0.77, 95% CI 0.52 to 1.14) and CONFIRM (HR 0.62, 95% CI 0.37 to 1.03) were presented in the European Public Assessment Report, and these differences were not statistically significant. The number of lesions on T1‑ and T2‑weighted and gadolinium‑enhancing MRI at 2 years was statistically significantly lower with dimethyl fumarate than with placebo in the DEFINE and CONFIRM trials.

3.6 The manufacturer presented results for the pre‑specified subgroup analyses for the DEFINE and CONFIRM trials in its submission. It stated that the results (treatment effect) of these analyses were generally consistent with the results for the overall population. The DEFINE trial results were similar for patients who had not had treatment for multiple sclerosis before (treatment‑naive); (proportion relapsed: HR 0.37, 95% CI 0.24 to 0.57; annualised relapse rate: RR 0.33, 95% CI 0.21 to 0.52; 3‑month disability progression at 2 years: HR 0.38, 95% CI 0.22 to 0.65) and treatment‑experienced patients (proportion relapsed: HR 0.65, 95% CI 0.48 to 0.89; annualised relapse rate: RR 0.61, 95% CI 0.45 to 0.84) showing statistically significant differences between patients taking dimethyl fumarate and placebo. However, for the 3‑month disability progression at 2 years outcome, disability progression was not statistically significantly reduced with dimethyl fumarate compared with placebo for the treatment‑experienced subgroup (HR 0.83, 95% CI 0.54 to 1.29). The manufacturer also noted that the CONFIRM trial results were similar for treatment‑naive patients (annualised relapse rate: RR 0.64, 95% CI 0.44 to 0.95; proportion relapsed: HR 0.73, 95% CI 0.51 to 1.05; 3‑month disability progression at 2 years: HR 0.56, 95% CI 0.30 to 1.03) and for treatment‑experienced patients (proportion relapsed: HR 0.57, 95% CI 0.38 to 0.84; annualised relapse rate: RR 0.47, 95% CI 0.31 to 0.69; 3‑month disability progression at 2 years: HR 1.07, 95% CI 0.60 to 1.89) showing statistically significant differences between patients taking dimethyl fumarate and placebo for all outcomes except for 3‑month disability progression at 2 years in patients who had already had treatment. No tests of interaction were presented by the manufacturer.

3.7 Both trials measured health‑related quality‑of‑life using the global well‑being visual analogue scale (VAS; which assesses a patient's global well‑being on study treatment on a linear scale, with 0 as 'poor' and 100 as 'excellent'), the Short Form 36 Health Survey (SF‑36) and the EuroQol‑5 dimensions survey (including the EQ‑5D descriptive system and the EQ VAS). In the DEFINE trial, patients randomised to dimethyl fumarate had a statistically significantly better health‑related quality‑of‑life compared with those randomised to placebo when measured by the mean change in: global well‑being VAS from baseline (0.4 compared with −4.0; p=0.0031), the physical component score of the SF‑36 from baseline (0.5 compared with −1.4; p<0.001), 6 of 8 SF‑36 subscales from baseline, and EQ VAS from baseline (−0.3 compared with −4.2; p<0.001). In the CONFIRM trial, patients randomised to dimethyl fumarate showed a statistically significantly better health‑related quality‑of‑life compared with those randomised to placebo when measured by the mean change in: global well‑being VAS from baseline (0.3 compared with −3.9; p<0.001), the physical component score of the SF‑36 from baseline (0.5 compared with −0.7; p=0.0217), and 3 of 8 SF‑36 subscales from baseline.

3.8 The manufacturer reported that the overall incidence of adverse reactions was similar in patients taking dimethyl fumarate and placebo respectively (96% compared with 95% in DEFINE, and 94% compared with 92% in CONFIRM). The most common adverse reactions reported for dimethyl fumarate compared with placebo were flushing (38% compared with 5% in DEFINE and 31% compared with 4% in CONFIRM), hot flush (8% compared with 2% in DEFINE and 5% compared with 2% in CONFIRM), upper abdominal pain (10% compared with 7% in DEFINE and 10% compared with 5% in CONFIRM), nausea (13% compared with 9% in DEFINE and 11% compared with 8% in CONFIRM) and vomiting (10% compared with 6% in DEFINE and 7% compared with 4% in CONFIRM). The manufacturer noted that most adverse reactions were mild to moderate in severity and that incidences were highest in the first month and decreased thereafter. The percentages of patients stopping treatment because of adverse reactions were 16% of those taking dimethyl fumarate and 13% of those taking placebo in the DEFINE trial, and 12% of those taking dimethyl fumarate and 10% of those taking placebo in the CONFIRM trial. The manufacturer reported that the incidence of serious adverse reactions in patients taking dimethyl fumarate was comparable to patients taking placebo (18% compared with 21% in DEFINE and 17% compared with 22% in CONFIRM).

3.9 The manufacturer presented the results of both a fixed‑effects and a random‑effects meta‑analysis of the efficacy and safety outcomes of the DEFINE and CONFIRM trials. It estimated that dimethyl fumarate was statistically significantly better than placebo for all efficacy outcomes analysed including disability progression sustained for 3 months and for 6 months, both at 2 years. The manufacturer's meta‑analysis also estimated that patients taking dimethyl fumarate experienced statistically significantly more gastrointestinal events, flushing and skin reactions compared with those taking placebo or glatiramer acetate. It estimated no statistically significant differences in the number of withdrawals for any reason between treatments, but that statistically significantly more patients taking dimethyl fumarate withdrew because of adverse reactions compared with those taking placebo or glatiramer acetate.

3.10 To estimate the relative effectiveness of dimethyl fumarate compared with the comparators defined in the scope, the manufacturer conducted a mixed treatment comparison of 27 trials using a fixed‑effects frequentist approach that assessed outcomes including annualised relapse rate, proportion of relapsing patients at 24 months, and confirmed disability progression sustained for 3 months and for 6 months, both at 2 years. The following comparators were included in the manufacturer's mixed treatment comparison: beta interferon‑1a (Avonex, Rebif‑22 and Rebif‑44), beta interferon‑1b (Betaferon), glatiramer acetate, fingolimod, natalizumab and placebo.

3.11 The manufacturer presented results of the mixed treatment comparison unadjusted for covariates. The manufacturer did a covariate analysis that showed that the chosen covariates had little or no impact on the outcomes of interest, although the baseline relapse rate was found to be a significant covariate for the annualised relapsed rate outcome. The manufacturer's unadjusted mixed treatment comparison suggested that dimethyl fumarate statistically significantly reduces the annualised relapse rate and the proportion of patients with relapses at 2 years compared with placebo, glatiramer acetate and all beta interferons. The manufacturer's mixed treatment comparison also suggested dimethyl fumarate statistically significantly reduces disability progression sustained for 3 months at 2 years compared with placebo. No statistically significant differences were estimated between dimethyl fumarate and any comparator, including placebo, for disability progression sustained for 6 months at 2 years. The manufacturer labelled the effect size and credible intervals from its mixed treatment comparison as academic in confidence, and therefore they cannot be presented here. The manufacturer stated that it had not explored the subgroups specified in the scope of the appraisal because they had not been analysed in most of the trials included in its mixed treatment comparison.

Cost effectiveness

3.12 The manufacturer did not identify any published studies of the cost effectiveness of dimethyl fumarate for treating adults with relapsing‑remitting multiple sclerosis. It submitted a cohort‑based Markov model that reflected the natural history of relapsing‑remitting multiple sclerosis with a cycle length of 1 year and assumed a patient can be offered 1 of 9 treatments: dimethyl fumarate, a beta interferon‑1a treatment (Avonex, Rebif‑22 or Rebif‑44), beta interferon‑1b (Betaferon, Extavia), glatiramer acetate, fingolimod or natalizumab. The manufacturer conducted the economic analysis from an NHS and personal social services perspective and chose a time horizon of 30 years. Costs and health effects were discounted at an annual rate of 3.5% and a half‑cycle correction was applied.

3.13 The manufacturer's model was structurally similar to models used in previous NICE technology appraisal guidance on treatments for multiple sclerosis: Fingolimod for the treatment of highly active relapsing-remitting multiple sclerosis (NICE technology appraisal guidance 254), Natalizumab for the treatment of adults with highly active relapsing-remitting multiple sclerosis (NICE technology appraisal guidance 127) and Beta interferon and glatiramer acetate for the treatment of multiple sclerosis (NICE technology appraisal guidance 32). The model estimated disease progression through 21 health states defined by EDSS scores (ranging from 0 to 9.5), which cover disability in patients with relapsing‑remitting multiple sclerosis (10 states), patients with secondary progressive multiple sclerosis (10 states) and death. In each cycle of the model, a patient with relapsing‑remitting multiple sclerosis could move to a higher or lower EDSS state or remain in the same state. Patients could also advance from relapsing‑remitting multiple sclerosis to secondary progressive multiple sclerosis, but could not subsequently move back to relapsing‑remitting disease. Only patients with relapsing‑remitting multiple sclerosis and an EDSS score of 6 or less were assumed to receive disease‑modifying treatment in the model.

3.14 Patient baseline characteristics were pooled from the DEFINE and CONFIRM trials. The probabilities of changing EDSS state or having a relapse (fixed for each EDSS state) were based on natural history data (underlying disease progression) and trial data (disease progression with treatment). The manufacturer estimated the natural history of disability progression using the placebo arms of the DEFINE and CONFIRM trials up to and including an EDSS score of 7, and using a longitudinal data set of patients with multiple sclerosis in London Ontario, Canada for EDSS scores of more than 7, because of the small number of observations for the more severe EDSS states in the trials. The Ontario longitudinal data set was also used by the manufacturer to estimate the natural history of:

3.15 To estimate disability progression and the annualised relapse rate of each treatment compared with placebo, the manufacturer used results from its mixed treatment comparison (that is, using the 'disability progression sustained for 3 months at 2 years' and 'annualised relapse rate' data). The manufacturer applied treatment effects only to patients with relapsing‑remitting multiple sclerosis because it assumed that patients with secondary progressive multiple sclerosis stop treatment. The economic model did not allow patients to switch treatments, so they remained on their original treatment until progression to EDSS score of 7 or more, because of adverse reactions, or conversion to secondary progressive multiple sclerosis. Because there was no evidence, the manufacturer also assumed that the treatment has no effect on disease progression to secondary progressive multiple sclerosis. The manufacturer noted that this assumption was adopted in Fingolimod for the treatment of highly active relapsing-remitting multiple sclerosis (NICE technology appraisal guidance 254). The model assumed that the treatment effect diminishes over time (waning) to 75% after 2 years and to 50% after 5 years. The manufacturer explained that because of a lack of long‑term data on the clinical effectiveness of dimethyl fumarate it applied a similar approach to that adopted by the Committee in NICE technology appraisal guidance 254. The manufacturer assumed that patients followed the natural history of disease progression after stopping treatment.

3.16 To estimate the probabilities for all‑cause mortality in the multiple sclerosis population, the manufacturer took England and Wales national mortality data and adjusted for patients with multiple sclerosis by age and EDSS score using mortality multipliers from a Danish population diagnosed with multiple sclerosis from 1948 onward reported in Pokorski et al. (1997). Mortality was assumed to be the same in patients with relapsing‑remitting multiple sclerosis and secondary progressive multiple sclerosis. The manufacturer presented the results of a scenario analysis that explored setting the rate of mortality in people with relapsing‑remitting multiple sclerosis equal to the rate of mortality in the general population of England and Wales.

3.17 Resource use and costs in the economic model depended on a patient's EDSS score, on whether they had relapsing‑remitting multiple sclerosis or secondary progressive multiple sclerosis, and on whether they were in relapse. The unit costs for each of the drugs and their administration were all originally taken from the 'British National Formulary 64' and 'NHS Reference Costs 2011/12'. The manufacturer updated its economic model during the factual accuracy check of the ERG report to include the prices from the NHS risk‑sharing scheme for beta interferons and glatiramer acetate in its base‑case analysis. The cost of dimethyl fumarate in the model included the patient access scheme. Resource use and costs associated with monitoring patients on treatment were based on the licensed indications presented in the summaries of product characteristics of the drugs. The manufacturer took resource‑use data for managing the disease from a regression analysis of data from the UK Multiple Sclerosis Survey that included 115 different healthcare resources. The manufacturer estimated a mean annual cost for each EDSS score in patients with relapsing‑remitting multiple sclerosis and patients with secondary progressive multiple sclerosis, and the mean cost per relapse independent of the clinical form of multiple sclerosis (that is, £2028 per relapse both in people with relapsing‑remitting multiple sclerosis and people with secondary progressive multiple sclerosis).

3.18 To estimate health‑related quality‑of‑life, the manufacturer used pooled EQ‑5D data from the DEFINE and CONFIRM trials for the EDSS states for relapsing‑remitting multiple sclerosis. The manufacturer estimated the utility values for secondary progressive multiple sclerosis using the differences between utility values for relapsing‑remitting multiple sclerosis and secondary progressive multiple sclerosis from the UK Multiple Sclerosis Survey. The manufacturer also subtracted the difference between utility for relapse and no relapse for each EDSS state as reported in the UK Multiple Sclerosis Survey from its EQ‑5D trial data to estimate the utility values for patients with relapse. The manufacturer's economic model also incorporated carer's disutility for each EDSS score, in line with estimates from Fingolimod for the treatment of highly active relapsing-remitting multiple sclerosis (NICE technology appraisal guidance 254) and Natalizumab for the treatment of adults with highly active relapsing-remitting multiple sclerosis (NICE technology appraisal guidance 127). The maximum disutility is assumed to be 0.14 for a carer of a person with multiple sclerosis with an EDSS score of 9.

3.19 The economic model included costs and disutility values associated with adverse reactions. The manufacturer only included adverse reactions reported in the trials when the incidence was 5% or higher, or when the absolute incidence in the dimethyl fumarate arm was 3% higher than in the placebo arm. The manufacturer took resource use and costs for each adverse reaction from published sources and validated them by clinical expert opinion. Disutility values were based on clinical expert opinion, published sources when available or the manufacturer's assumption.

3.20 The manufacturer presented deterministic pairwise incremental cost‑effectiveness ratios (ICERs) for dimethyl fumarate compared with each of the treatments included in its economic model. Dimethyl fumarate dominated Avonex (that is, dimethyl fumarate gave more QALYs and cost less than Avonex): the manufacturer estimated incremental cost savings of £223 and 0.194 incremental QALYs gained. For dimethyl fumarate compared with Rebif‑22, the manufacturer estimated incremental costs of £6093 and 0.286 incremental QALYs gained with an ICER of £21,341 per QALY gained. For dimethyl fumarate compared with Rebif‑44, the manufacturer estimated incremental costs of £2592 and 0.163 incremental QALYs gained with an ICER of £15,909 per QALY gained. For dimethyl fumarate compared with Betaferon, dimethyl fumarate dominated Betaferon; the manufacturer estimated incremental cost savings of £2834 and 0.386 incremental QALYs gained. For dimethyl fumarate compared with glatiramer acetate, the manufacturer estimated incremental costs of £6516 and 0.331 incremental QALYs gained with an ICER of £19,716 per QALY gained. The patient access scheme price for fingolimod was not included in the manufacturer's base‑case analysis because it is not publicly available and the manufacturer of fingolimod did not provide the patient access scheme to the manufacturer of dimethyl fumarate. Assuming a 35% reduction in the list price of fingolimod, dimethyl fumarate dominated fingolimod in the active relapsing‑remitting multiple sclerosis population: the manufacturer estimated incremental cost savings of £18,347 and 0.264 incremental QALYs gained. For dimethyl fumarate compared with natalizumab, the manufacturer estimated incremental cost savings of £46,256 and an incremental QALY loss of 0.103 leading to savings of £448,729 per QALY lost.

3.21 The manufacturer explored parameter and structural uncertainty in its economic model by presenting the results of univariate sensitivity analyses, 2‑way sensitivity analyses and scenario analyses. The results from the univariate sensitivity analyses suggested the manufacturer's economic model was most sensitive to changes in the effect of treatment on the disability progression rate (ICERs increased when the effect of dimethyl fumarate was reduced by 20%, or the effect of the comparator was increased by 20%). The manufacturer commented that its scenario analyses indicated that its economic model is robust to most of the structural assumptions. The results from the scenario analyses were most sensitive to changes in the time horizon. In its scenario analyses, the manufacturer varied the price of fingolimod by reducing its list price in 5% increments. It estimated that dimethyl fumarate dominated fingolimod in the active relapsing‑remitting multiple sclerosis population unless fingolimod's list price is decreased by more than 60%.

3.22 The manufacturer also presented results from probabilistic analyses in the active relapsing‑remitting multiple sclerosis population. Dimethyl fumarate dominated Betaferon and fingolimod (with a 35% reduction in the list price of fingolimod). For dimethyl fumarate compared with Rebif‑22, the manufacturer estimated an ICER of £30,898 per QALY gained. For dimethyl fumarate compared with Rebif‑44, the manufacturer estimated an ICER of £23,408 per QALY gained. For dimethyl fumarate compared with Avonex, the manufacturer estimated an ICER of £2573 per QALY gained. For dimethyl fumarate compared with glatiramer acetate, the manufacturer estimated an ICER of £30,331 per QALY gained. For dimethyl fumarate compared with natalizumab, the manufacturer estimated incremental cost savings and an incremental QALY loss leading to savings of £610,134 per QALY lost.

ERG comments on the clinical effectiveness

3.23 The ERG stated that the DEFINE and CONFIRM trials were of good quality and had a low risk of bias. The ERG commented that the trial populations more closely reflect people with relapsing‑remitting multiple sclerosis who meet the Association of British Neurologists' prescribing criteria for disease‑modifying therapy (that is, adults with active relapsing disease defined as 2 or more clinically significant relapses in the previous 2 years) than people with relapsing‑remitting multiple sclerosis in general. The ERG explained that:

3.24 The ERG stated that 3‑month disability progression was used as an outcome measure in Fingolimod for the treatment of highly active relapsing-remitting multiple sclerosis (NICE technology appraisal guidance 254). However, this is not consistent with the European Medicines Agency's draft guideline on the clinical investigation of medicinal products for the treatment of multiple sclerosis that advises the use of 6‑month disability progression (because at 3 months the possibility of recovery exists). Because the manufacturer's data for 6‑month sustained disability progression showed less clear evidence of benefit than the 3‑month sustained disability progression, the ERG concluded that some uncertainty remained regarding the effect of dimethyl fumarate on disability progression.

3.25 The ERG stated that the rates of adverse reactions and serious adverse reactions for patients taking dimethyl fumarate were similar to those for placebo. The ERG noted that higher incidences of flushing and gastrointestinal events were reported for dimethyl fumarate, but these appeared to be confined mainly to the first 3 months of treatment. It was unclear whether this was also the case for skin reactions.

3.26 The ERG stated that the 2‑year duration of trials was short compared with:

3.27 The ERG commented that the manufacturer's mixed treatment comparison included all relevant trials. It noted that these trials appeared to be at low, or unclear, risk of bias, although the manufacturer did not assess allocation concealment. The ERG stated that some networks were sparsely populated because of the number of outcomes analysed and the availability of data from the included trials. It also noted a moderate level of clinical and methodological heterogeneity between the trials included. This included differences in baseline characteristics such as mean EDSS score and the inclusion criteria regarding the number of relapses in the period before randomisation. For example, the mean or median relapse rate in the year before randomisation ranged between 1.0 and 2.4, which the ERG considered to be clinically meaningful. However, the ERG concluded the level of heterogeneity between trials was not sufficient to make the comparisons unreasonable.

3.28 The ERG stated that using a fixed‑effects frequentist approach in the manufacturer's mixed treatment comparison was likely to be appropriate for assessing most of the outcomes because the small number of trials comprising the networks did not allow an estimation of the between‑study variance. However, a random‑effects model may have been more appropriate for assessing the annualised relapse rate than a fixed‑effects model because the network included a sufficient number of trials. The ERG noted that the estimated confidence intervals for the annualised relapse rate outcome may therefore have been slightly underestimated (that is, too narrow).

3.29 The ERG commented that the manufacturer did not address the relative effectiveness of dimethyl fumarate compared with fingolimod or natalizumab in the subgroups specified in the final scope. It acknowledged that the populations included in the trials were broader than those defined in the comparator drugs' marketing authorisations. However, the ERG concluded that because the manufacturer did not analyse patients with highly active relapsing‑remitting multiple sclerosis or rapidly evolving severe relapsing‑remitting multiple sclerosis, the relative effectiveness of dimethyl fumarate compared with fingolimod and natalizumab was unknown in these subgroups respectively.

ERG comments on the cost effectiveness

3.30 The ERG confirmed that the economic model structure adopted by the manufacturer was structurally similar to that used in previous NICE technology appraisals of multiple sclerosis. It stated that including improvement to lower EDSS states reflected the actual experience of patients in the trials of dimethyl fumarate and the experience of people with relapsing‑remitting multiple sclerosis generally. Although sustaining disability progression for 6 months may be more closely associated with permanent progression, the ERG noted that the use of 3‑month sustained disability progression outcome data in the manufacturer's economic model was reasonable because patients' disease could improve to lower EDSS states. The ERG commented that the economic model predictions for the patients across the EDSS states seemed reasonable compared with the distribution of dimethyl fumarate patients across the EDSS states within the time period of the trials.

3.31 The ERG preferred the manufacturer's ICERs calculated from the probabilistic sensitivity analysis to the deterministic ICERs because the economic model is non‑linear. However, the ERG noted that the manufacturer had not assigned probability distributions to a number of parameters including the parameter accounting for treatment waning over time and the annual risk of stopping treatment. The ERG explained that these 2 parameters have a significant effect on the estimated ICERs because disease progression is the key driver of the economic model. The ERG noted that the main driver of the economic model was the hazard ratio of 3‑month disability progression but it did not explore any analyses around this parameter because it felt that the manufacturer's mixed treatment comparison and probability distributions were adequate. However, the ERG stated that a fixed‑effects mixed treatment comparison may underestimate the uncertainty in the treatment effect, and therefore the uncertainty in the cost‑effectiveness estimates of dimethyl fumarate may also be underestimated. It concluded that although the probabilistic results were more meaningful and represented a less biased approximation of the ICER compared with deterministic results, the full impact of the uncertainty around the ICER had not been completely accounted for.

3.32 The ERG's exploratory analyses resulted in base‑case deterministic pairwise ICERs within £100 of those presented by the manufacturer during its factual accuracy check of the ERG report, and are therefore not presented here; for further details see the ERG addendum. The ERG also presented base‑case incremental results using hazard ratios as the outcome measure for 3‑month disability progression at 2 years, which showed that:

3.33 The ERG considered that the resource‑use for neurology visits in the manufacturer's economic model was too high in year 1 for beta interferons, too low in year 1 for natalizumab, and too low after year 1 for dimethyl fumarate and fingolimod. It also chose to explore a scenario assuming that the cost of a neurology visit was equal to the cost of visiting a neurologist (£205) because the manufacturer assumed that the cost of a neurology visit was equal to the cost of a day‑case admission (£590). The ERG also used alternative estimates for the inclusion of annual MRI scans for patients taking natalizumab and the inclusion of nurse visits for patients taking injectable treatments. Using these alternative monitoring resource assumptions, the ERG estimated that its base‑case ICER increased from £21,414 to between £21,419 and £28,973 per QALY gained for dimethyl fumarate compared with Rebif‑22.

3.34 The ERG explored alternative assumptions around the rates of stopping treatment. Changing the rates of stopping treatment to 50% of the original relative risks of stopping treatment estimated in the mixed treatment comparison after 2 years and to 0% after 2 years of treatment increased the ERG's estimated base‑case ICER for dimethyl fumarate compared with Rebif‑22 from £21,414 to £23,278 and £23,292 per QALY gained respectively. The ICER for dimethyl fumarate compared with glatiramer acetate increased above £30,000 per QALY gained when using the lower confidence intervals of the relative risks of stopping treatment. The ERG commented that when patients stop their initial treatment in the manufacturer's economic model they receive placebo or 'best supportive care', and then progress more quickly through the EDSS states. Switching from treatment to no treatment reduces costs to a greater extent than it reduces QALYs; if more patients stop treatment, the treatment becomes more cost effective. The ERG raised the concern that if best supportive care were specified in the scope as one of the comparators, the ICER for dimethyl fumarate compared with an active treatment would never be lower than the ICER for dimethyl fumarate compared with best supportive care. However, the ERG considered that in clinical practice patients who stop treatment because of adverse reactions will take another active treatment if an alternative (with a differing side‑effect profile) is available.

3.35 The ERG considered that it was appropriate for the manufacturer to use utility values estimated from the trials of dimethyl fumarate. However, it noted that by using the utility values from the trials as a proxy for 'people with relapsing‑remitting multiple sclerosis without relapse' in its economic model, the manufacturer may have underestimated the health‑related quality‑of‑life of these patients because some of the patients included in the trials will have been in relapse. The ERG explored this uncertainty by incorporating into its exploratory analyses utility values from 2 other sources based on the UK Multiple Sclerosis Survey. These sources were the utility values reported in Natalizumab for the treatment of adults with highly active relapsing-remitting multiple sclerosis (NICE technology appraisal guidance 127), and the utility values estimated from a multivariate linear regression analysis of the UK Multiple Sclerosis Survey in Orme et al. (2007). However, the ERG noted that only 35.5% of the UK Multiple Sclerosis Survey population had relapsing‑remitting multiple sclerosis. It was also concerned about the utility values in all sources because the differences between secondary progressive multiple sclerosis and relapsing‑remitting multiple sclerosis, and between relapse and no relapse were based on a population that did not entirely reflect the scope of the appraisal. For dimethyl fumarate compared with Rebif‑22, the ERG estimated that its base‑case ICER changed from £21,414 to £18,700 and £22,144 per QALY gained when using the utility values from Orme et al. (2007) and NICE technology appraisal guidance 127 respectively.

3.36 The ERG considered that using different EDSS state costs for people with relapsing‑remitting multiple sclerosis and people with secondary progressive multiple sclerosis is appropriate. It was aware of 3 sources reporting costs by EDSS state that used the resource‑use data from the UK Multiple Sclerosis Survey (including the source used in the manufacturer's economic model). However, despite using the same resource‑use data, the 3 sources estimated different costs. The ERG explained that the variation between estimates may be because each source used different unit costs and only 1 of the sources separated medical and non‑medical costs. It stated it was unclear which of the 3 sources was the most appropriate but these differing estimates of EDSS state costs did not have a significant impact on the ICERs. For dimethyl fumarate compared with Rebif‑22, the ERG estimated that its base‑case ICER changed from £21,414 to between £17,239 and £21,377 per QALY gained. The ERG judged the cost per relapse in the manufacturer's economic model of £2028 to be too high after receiving advice from clinicians that only 20% of relapses need hospitalisation. When the ERG varied the cost per relapse to between £3039 and £280 the ICER changed from £21,414 to between £18,660 and £26,074 per QALY gained respectively, for dimethyl fumarate compared with Rebif‑22.

3.37 Although the manufacturer did not include the relative risks of adverse reactions from its mixed treatment comparison in its economic model, the ERG stated that the manufacturer's approach to estimating the incidence of adverse reactions was reasonable. Using the relative risk of adverse reactions from the manufacturer's mixed treatment comparison, or assuming no adverse reactions, the ERG estimated that its base‑case ICER changed from £21,414 to £26,683 or to £24,869 per QALY gained respectively, for dimethyl fumarate compared with Rebif‑22. The ERG also explored revised disutility values for influenza and flu‑like symptoms because the manufacturer's estimate seemed unreasonably high. Updating these disutility values had very little impact on the ICERs.

Manufacturer's additional evidence

3.38 The manufacturer provided additional evidence in its response to consultation. The manufacturer presented the results of the outcomes of relapse and of disability progression adjusted only for baseline relapse rate from the DEFINE and CONFIRM trials. These results were similar to the analyses presented in its original submission adjusted for age, EDSS score, baseline relapse rate and geographical region (see sections 3.4 to 3.5). The manufacturer's revised mixed treatment comparison adjusted for baseline relapse rate estimated similar treatment effects relative to placebo to those originally presented in its unadjusted analysis (see sections 3.10 to 3.11). The manufacturer stated that its original mixed treatment comparison unadjusted for covariates (although the individual trial data within the mixed treatment comparison may have been adjusted for baseline rate) was more appropriate than the mixed treatment comparison adjusted for baseline relapse rate because the statistics describing model fit performed better and the unadjusted results more closely reflected the individual trial results. The results from the trials and mixed treatment comparison adjusted for baseline relapse rate are marked as academic in confidence by the manufacturer and cannot be presented here.

3.39 The manufacturer presented interim data from its ongoing open‑label ENDORSE extension study of dimethyl fumarate (n=1736). The manufacturer stated that the efficacy outcomes suggest dimethyl fumarate's treatment effect is maintained at 4 years (annualised relapse rate: 0.142, 95% CI 0.108 to 0.187; proportion relapsed: 36.2%, 95% CI 32.1% to 40.6%; proportion with confirmed disability progression sustained over 24 weeks: 15.4%, 95% CI 12.4% to 18.9%). The manufacturer also noted that the safety data showed no new or worsening safety outcomes.

3.40 The manufacturer presented clinical‑effectiveness data for highly active relapsing‑remitting multiple sclerosis that were not available at the time of original submission. The manufacturer commented that the patient numbers in each treatment group were small in the analysis of the highly active relapsing‑remitting multiple sclerosis subgroup, but the results were consistent with the results for the overall relapsing‑remitting multiple sclerosis population. In a pooled analysis of the DEFINE and CONFIRM trials, high disease activity was defined as:

3.41 The manufacturer provided revised cost‑effectiveness analyses as part of the additional evidence. The manufacturer presented probabilistic pairwise ICERs and fully incremental analyses for dimethyl fumarate compared with each of the comparators, incorporating the following data requested by the Appraisal Committee:

3.42 In the manufacturer's probabilistic pairwise analysis excluding non‑medical costs in the active relapsing‑remitting multiple sclerosis population, dimethyl fumarate:

3.43 The manufacturer also presented pairwise ICERs, as well as a fully incremental analysis, for a scenario using its own preferred assumptions. These included:

3.44 In the manufacturer's probabilistic pairwise analysis for the active relapsing‑remitting multiple sclerosis population that excluded non‑medical costs, but which incorporated its own preferred assumptions, dimethyl fumarate:

3.45 In response to the request in the appraisal consultation document for external validation of its economic model, the manufacturer presented cost‑effectiveness results for all of the beta interferons and for glatiramer acetate compared with no treatment. The aim of the validation was to determine how similar the ICERs from the manufacturer's economic model were to those in the NHS risk‑sharing scheme for multiple sclerosis. To more closely reflect the structural assumptions of the NHS risk‑sharing scheme economic model, the manufacturer adapted its economic model to include a 20‑year time horizon and excluded the possibility that the effectiveness of treatments wanes over time. The manufacturer's deterministic ICERs compared with no treatment were as follows:

ERG comments on the manufacturer's additional evidence

3.46 The ERG reviewed the additional evidence presented by the manufacturer and commented that it appropriately addressed the analyses requested in the appraisal consultation document. The ERG noted that adjusting the trial outcomes for baseline relapse rate made little difference to the values reflecting disability progression and relapse. The ERG stated that using the results of the manufacturer's unadjusted mixed treatment comparison was reasonable because there were too few studies in the network to estimate the effect of a covariate with precision.

3.47 The ERG noted that the manufacturer's probabilistic ICERs, which excluded non‑medical costs, were similar to those including non‑medical costs. The ERG commented that the manufacturer did not explain how it estimated the costs of each EDSS state when including or excluding non‑medical costs. For some EDSS states, the manufacturer estimated higher costs when excluding non‑medical costs than when including non‑medical costs. However, the ERG noted that these differences were small. The ERG stated that the cost of relapse (£1206) chosen by the manufacturer may be plausible but the lower cost of relapse requested in the appraisal consultation document (£607.80) also remains plausible. The ERG explored a scenario that used £607.80, which led to only a small increase in the ICERs for dimethyl fumarate compared with each treatment. The ERG considered its 'alternative' assumption for the number of neurology visits (2 visits instead of 1) in year 2 onwards, which it had been informed by a clinical adviser was plausible.

3.48 The ERG noted that treatment sequences starting with dimethyl fumarate estimate higher ICERs than sequences that start with either glatiramer acetate or Rebif. It also highlighted that a limitation of the analysis is that the effectiveness of the treatments is assumed to be the same no matter the position in the treatment pathway, but that this could be a constraint of the available data.

3.49 The ERG commented that, when the manufacturer externally validated its model, it estimated results that differed from those estimated in the NHS risk‑sharing scheme (see section 3.45). Given the information available, the ERG could not explore the reasons for these differences. The ERG acknowledged that health economic methods have evolved since the publication of the ICERs associated with the NHS risk‑sharing scheme, including, for example, the use of mixed treatment comparisons, changes in discount rates used for costs and health effects, and the use of probabilistic analyses.

3.50 Full details of all the evidence are in the evaluation report.