3 Committee discussion
The appraisal committee (section 5) considered evidence submitted by Merck, Sharpe & Dohme and a review of this submission by the evidence review group (ERG). See the committee papers for full details of the evidence.
Cytomegalovirus reactivation substantially affects the mental health and wellbeing of patients and their families
3.1 Cytomegalovirus (CMV) can become active again in about 60% to 80% of people who are seropositive for CMV and who have had an allogeneic haematopoietic stem cell transplant (HSCT). This can happen especially if the transplant involved T‑cell depletion therapy, which is common in the UK. The patient experts highlighted that CMV reactivation can have a substantial psychological effect on patients and their families, negatively affecting their mental health and wellbeing. Hospital admissions to treat CMV reactivation disrupt family and working life and are particularly stressful because of the worry and risk of further infections. Pre-emptive therapy for CMV reactivation (see section 3.2) can have serious side effects. The patient experts noted that better prevention of CMV reactivation would reduce hospital admissions and the need for toxic pre-emptive therapy, which would greatly reduce distress. The committee concluded that CMV reactivation can have a substantial psychological effect on patients and their families.
Patients and clinicians would welcome a new treatment that could prevent CMV reactivation and reduce the need for pre-emptive therapy
3.2 There are no licensed treatments available specifically for preventing CMV reactivation after an allogeneic HSCT. The current standard approach in the NHS is surveillance monitoring for viral reactivation. When viral DNA is detected, pre-emptive therapy is started with ganciclovir, valganciclovir or foscarnet, depending on the type of transplant received (T‑cell depletion or T‑cell repletion). The clinical experts stated that:
Practice varies on when to start pre-emptive therapy. Some centres would start based on a specific viral load count, but the assays for measuring viral load are not standardised.
Aciclovir is also used as prophylaxis in some centres although it may not be effective for this.
Letermovir reduces reactivation rates and the need for toxic pre-emptive therapy and improves quality of life.
The committee concluded that an effective and tolerable treatment that specifically acts to prevent CMV reactivation would benefit people who are seropositive for CMV who have had an allogeneic HSCT.
3.3 The main clinical evidence came from a phase III randomised placebo-controlled trial, PN001. This trial compared the efficacy and safety of letermovir (n=373) with placebo (n=192) in adults who were seropositive for CMV and who have had an allogeneic HSCT. Treatment continued to week 14 (about 100 days) and patients were monitored through to week 24 after transplant for the primary efficacy outcome. The primary outcome was the incidence of clinically significant CMV infection by week 24. This was assessed by the proportion of people with CMV end-organ disease or starting anti-CMV pre-emptive therapy (ganciclovir, valganciclovir or foscarnet with or without cidofovir) based on documented CMV viraemia (as measured by the central laboratory) and the clinical condition of the patient. Patients who completed the trial then entered a follow-up phase from week 24 to week 48 after transplant. The company presented results for 2 populations: the overall randomised population who had treatment (the 'all subjects as treated' population) and the main analysis population in the trial (the 'full analysis set' population). The full analysis set population excluded patients who were randomised and had treatment if they had detectable CMV DNA on day 1. To account for missing data, the company's preferred approach was to assume that if people did not complete treatment their treatment had not worked. This included people with missing data or who stopped the study early. The committee concluded that PN001 was a well conducted trial and agreed that it would consider the full analysis set population.
Only 12 patients were from the UK.
The maximum treatment duration was 100 days. The ERG considered this inappropriate because in clinical practice, some people may need longer periods of prophylaxis, for example people having treatment for active graft versus host disease or those at high risk of CMV reactivation because of T‑cell depletion. During consultation, a clinical expert explained that although the marketing authorisation allows treatment for longer than 100 days, there is no evidence that patients benefit from longer prophylaxis. The clinical experts stated that until trial evidence supports such use, they would not continue letermovir prophylaxis for longer than 100 days. The committee accepted that in clinical practice in England, the maximum letermovir treatment duration was unlikely to be longer than 100 days.
There was a delay in starting prophylaxis (mean 11.5 days, 'all subjects as treated' population; mean 10.9 days, full analysis set population) after transplant in the trial, which could potentially underestimate the efficacy and treatment duration of letermovir. During consultation, the company explained that the delay was because of concerns about the safety of starting prophylaxis immediately after HSCT. The clinical experts stated that they were now reassured about the safety of starting prophylaxis immediately after HSCT, and therefore would not expect a delay in clinical practice. Also, at the second appraisal committee meeting, the company noted that because the clinical effectiveness estimates were based on the observed treatment duration from PN001 it was not appropriate to include an additional treatment delay. The committee accepted that a delay would not be likely in clinical practice but agreed there was some uncertainty about the estimates of treatment duration. It concluded that without a better estimate of treatment duration, estimates from the trial were acceptable.
The prevalence of ciclosporin A use in people who had letermovir in the trial was 51.7% (based on the 'all subjects as treated' population). Both the ERG and the clinical experts agreed that this was much lower than seen in clinical practice. They assumed that approximately 90% to 95% of people would have ciclosporin A and the remaining patients would have tacrolimus.
The limited use of alemtuzumab (used for depleting T‑cells to avoid graft versus host disease) in the trial (4%) compared with clinical practice in the NHS (around 60% to 85%) could potentially underestimate the CMV reactivation rate and overestimate the risk of graft versus host disease. The clinical experts stated that its use in clinical practice depends on the treating centre but suggested that approximately 60% to 85% of people would have alemtuzumab.
The ERG highlighted that clinically significant CMV reactivation leading to pre-emptive therapy is defined differently in the trial than in UK practice. In the trial, a viral load threshold between 150 to 300 copies/ml was used, depending on the patient's risk of CMV infection. The clinical experts stated that, in clinical practice, the threshold varied by centre but typically would be between 400 to 700 copies/ml. During consultation, 1 clinical expert noted that CMV infection would not be determined by the threshold level of viraemia, highlighting that the clinical difference between the stated threshold ranges was negligible. The committee acknowledged that there was some ambiguity about the definition of a clinically significant CMV infection. It agreed that it was unclear if CMV infection rate, and subsequently the use of pre-emptive therapy, were over or underestimated in the trial.
During consultation 1 clinical expert commented that the PN001 results were generalisable to clinical practice in England, because the treatment duration from PN001 was realistic and the CMV infection rate was not overestimated. The committee therefore concluded that the trial results were generalisable to clinical practice, but accepted that its concerns about treatment duration, pre-emptive therapy use and the definition of a clinically significant CMV infection could make interpreting the results challenging.
3.5 In PN001, letermovir statistically significantly reduced the rate of clinically significant CMV infection at week 24 compared with placebo. The stratum-adjusted treatment difference between letermovir and placebo was −23.5 (95% confidence interval [CI] −32.5 to −14.6). The hazard ratio (HR) for time to onset of clinically significant CMV infection at week 24 was 0.29 (95% CI 0.21 to 0.42). There was also a statistically significant difference in starting pre-emptive therapy for documented viraemia by week 24 between letermovir and placebo (stratum-adjusted treatment difference −23.3 [95% CI −32.3 to −14.3]). The committee concluded that compared with placebo, letermovir is effective in reducing the incidence of clinically significant CMV infection after allogeneic HSCT and in reducing the need for pre-emptive therapy.
The all-cause mortality benefit of letermovir compared with placebo is not statistically significant at 48 weeks
3.6 In PN001, letermovir statistically significantly reduced the all-cause mortality rate at week 24 compared with placebo, with a hazard ratio of 0.57 (95% CI 0.34 to 0.96). The company also did an exploratory analysis using week 48 data, which included people who withdrew early from the trial but were confirmed to be alive after the trial had ended. The difference in all-cause mortality between letermovir and placebo was 3.8% but this was not statistically significant (HR 0.73, 95% CI 0.49 to 1.09). When people were stratified by prior CMV infection in another post hoc analysis, people on letermovir who had clinically significant CMV infection through week 24 had a lower all-cause mortality rate at week 48 when compared with people on placebo who had clinically significant CMV infection at week 24. Similar all-cause mortality rates were seen in both groups in people without clinically significant CMV infection at week 24. The committee acknowledged that these post hoc analyses could suggest that letermovir prevents additional all-cause mortality in people with prior CMV infection (CMV-related), despite not completely preventing CMV reactivation. It also noted that CMV-related mortality results were available, but the European Medicines Agency did not consider the data to be scientifically sound. During consultation, the company highlighted that although mortality was not a primary outcome in PN001, the exploratory analyses suggested letermovir could provide mortality benefit. The clinical experts stated that a mortality benefit with letermovir is plausible although it has not been proven. The committee agreed that the 48‑week post hoc analysis provided a more complete data set because it included mortality events in PN001 that occurred after week 24. It concluded that it would consider the 48‑week analysis for decision making, but acknowledged that the size of letermovir's mortality benefit is uncertain because of the limited trial follow up.
3.7 Overall, adverse events were similar between the letermovir group and the placebo group except for those leading to patients stopping treatment. There were no treatment-related deaths in either group. The most commonly reported adverse events in the 2 groups were graft versus host disease, nausea, vomiting, diarrhoea, pyrexia and rash. Cardiac disorder, hyperkalaemia, ear and labyrinth disorder and dyspnoea were more common in people on letermovir than on placebo. The ERG commented that the adverse event results were difficult to interpret because of the underlying conditions and treatments as well as the toxicity associated with various pre-emptive therapy regimens. Also, there were no safety data presented for letermovir use after 100 days. The committee was aware of the conclusions in the European public assessment report, which stated that the adverse event profile appeared similar to that of current standard care (that is, surveillance monitoring and pre-emptive therapy). The committee concluded that the safety profile of letermovir was acceptable and unlikely to be worse than current standard care.
PN001 did not show a health-related quality-of-life benefit for letermovir compared with placebo but a benefit is plausible
3.8 Health-related quality-of-life data collected at the time of randomisation and at weeks 14, 24 and 48 after transplant using EQ-5D-3L and FACT-BMT questionnaires showed no statistically significant differences between letermovir and placebo. A small possible utility benefit on graft versus host disease, rehospitalisation and opportunistic infections was seen with letermovir compared with placebo, but these benefits were not statistically tested. The company explained that health-related quality of life was an exploratory outcome and PN001 was not powered to detect statistically significant differences between treatment groups. The ERG also highlighted that, other than at randomisation, the mean values for EQ‑5D-3L and FACT-BMT scores represent a mixture of people who have had CMV reactivation and started pre-emptive therapy and those who have not. Therefore, the direct effect of letermovir on health-related quality of life was confounded. Also, the clinical experts stated that showing improvement in quality of life in a clinical trial of this nature is challenging because of differences in timing of assessments in relation to letermovir dosing and administration of other treatments. During consultation, the company explained that health-related quality of life was assessed before starting pre-emptive therapy, therefore the disutility associated with toxicities from pre-emptive therapy was not captured. The committee recalled comments from the clinical and patient experts that reducing CMV reactivation rates and the need for toxic pre-emptive therapy would improve quality of life (see section 3.1). The committee acknowledged the trial's limitations, which made interpreting the results more challenging. It agreed that although the trial did not show a health-related quality-of-life benefit for letermovir compared with placebo from preventing CMV reactivation, it concluded that such a benefit was plausible and agreed it would take this into account in its decision making.
3.9 The company's economic model had a lifetime time horizon. It consisted of a decision tree phase up to week 24 after transplant (week 48 in the scenario analysis) and a simple 2-state (alive or dead) Markov model phase covering the remaining time horizon of the model. The ERG considered that the company's model was too simplistic because it lacked explicit health states to capture differences in quality-adjusted life years (QALYs). The company's model approach does not link the rate of CMV events (the principal benefit of letermovir) with mortality. The committee was aware that this meant that nearly all the QALY benefits in the model for letermovir were derived from mortality differences. As such, the differences in the rate of CMV infection and other clinical events (for example, graft versus host disease) between the letermovir group and the placebo group and their effect on quality of life and mortality could not be fully explored. The committee agreed with the ERG that the company's model was oversimplified and acknowledged that this introduced some uncertainty about the cost-effectiveness estimates. Nevertheless, the committee recognised the difficulty in fully capturing the mortality benefits associated specifically with letermovir prophylaxis in the model because of the differences in mortality risk associated with patients' underlying conditions and the lack of available data to do this. The committee concluded that, although the model is oversimplified, it was appropriate for decision making.
3.10 In the decision tree phase of the model, the company included the cumulative probabilities of 6 different clinical events from PN001 (starting pre-emptive therapy, CMV disease, rehospitalisation, opportunistic infection, graft versus host disease and all-cause mortality). These clinical events were drawn from the 24‑week data and used 'data as observed', meaning that no adjustments were made for the 13.5% incomplete follow-ups at week 24. The ERG considered it inappropriate to use 24‑week data when 48‑week data were available for most outcomes. The committee recalled that the company had collected mortality data at week 48, which included people who withdrew early from the trial but were alive after the end of the trial. The ERG considered this data set to be more complete because only 3.2% patients were lost to follow up. During consultation, the company highlighted that the trial follow-up phase to week 48 investigated the safety of letermovir. It also noted that in PN001 letermovir prophylaxis stopped after 14 weeks. Because of this, it was not appropriate to assess the effect of prophylaxis at week 48. The ERG explained that by ignoring mortality events in PN001 between weeks 24 and 48, the company assumed an additional mortality benefit for letermovir. At the second appraisal committee meeting, a clinical expert explained that it was logical to use the 48‑week data because it was more complete. The committee recalled its consideration that a 48‑week post hoc analysis provided a more complete data set for decision making (see section 3.6), and concluded that it preferred to use the 48‑week data instead of the 24‑week 'data as observed' in its decision making.
Mortality data from the haematological malignancy research network (HMRN) is more relevant to clinical practice
3.11 In the company's model in the Markov model phase, the clinical outcome used was all-cause mortality. The mortality rate was assumed to be the same in both groups. This was based on general population mortality data from the Office for National Statistics, with a standardised mortality rate from Wingard et al. (2011) applied to account for the effect of the underlying condition. The ERG considered that the company's general approach was appropriate but that the HMRN was a more relevant source of UK data. The clinical experts at the committee meeting agreed. In the company's model the excess risk of mortality in year 2 was assumed to be equal to the excess risk in year 3. The clinical experts stated that mortality risk in year 2 was likely to be much higher than in year 3 and more in line with that reported by the HMRN (19% compared with the company's 3%). Also, the ERG highlighted that the Wingard study data were relatively old (from 1980 to 2003) and therefore their relevance to current practice was unclear. In addition, a substantial proportion (more than 40%) of the population in the Wingard study were children. During consultation, the company noted that the HMRN mortality data did not have the same level of detail about the underlying disease as the Wingard study data. However, it acknowledged that there were limitations with both data sources and agreed with the ERG's concerns about the Wingard study. The committee therefore concluded that it would consider the HMRN data in its decision making because they are more relevant to NHS clinical practice.
The ERG's approach to modelling long-term disutility associated with HSCT is preferred for decision making
3.12 In the original company submission, a scenario analysis included a disutility for the long-term effects (more than 48 weeks) of HSCT. However, the ERG did not consider this analysis to fully capture the long-term disutility associated with having HSCT because it was derived from a mix of EQ‑5D‑5L (Leunis et al. 2014) and EQ‑5D‑3L (Ara et al. 2011) values. The ERG suggested an alternative disutility based on the difference between the mean utility of patients in PN001 at 48 weeks and the mean general population utilities from Ara et al. The committee agreed that the company's approach to modelling long-term disutility associated with HSCT was inappropriate and concluded that the ERG's alternative approach was preferable.
Modelling disutility associated with graft versus host disease is included in the company's updated base case
3.13 The ERG identified that disutility associated with graft versus host disease should have been included in the company's original base-case analysis. This is because it is a serious and common complication of allogeneic HSCT and is associated with significant morbidity and mortality. During consultation, the company provided an updated base-case analysis which included the disutility associated with chronic graft versus host disease (occurring more than 100 days after HSCT), and discounting any disutility occurring beyond the first year. The company noted that changes in utility values for acute graft versus host disease (occurring within 100 days of HSCT) would already be captured in the utility values, therefore additional disutility was not needed. The committee concluded that the company's approach to modelling the disutility associated with graft versus host disease in its updated base-case analysis was acceptable and would be considered in its decision making.
There is uncertainty around the actual treatment duration for letermovir, but it is likely to be between the company's and ERG's estimates
3.14 In its original base case, the company assumed that the mean duration of treatment in the model was 69.4 days. This was based on the 'all subjects as treated' population from PN001. The ERG considered that the duration of treatment may be considerably longer and assumed mean duration of treatment to be 83 days in its original base case. This was based on the duration of therapy in the full analysis set population (72.1 days) plus an additional 10.9 days from the delayed start of prophylaxis in the trial. The clinical experts explained that, because a delay in starting letermovir prophylaxis is not expected in clinical practice, the duration of treatment is likely to be longer than 69.4 days but should not be more than 100 days (see section 3.4). During consultation, the company acknowledged that treatment duration was likely to be longer than 69.4 days and assumed a treatment duration of 72.1 days in its updated base case. The clinical experts highlighted that because of the treatment delay in PN001 there was no accurate estimate of mean treatment duration. However, they noted that a mean treatment duration of between 72.1 days and 83 days was a reasonable estimate. The committee concluded that it would consider a mean letermovir treatment duration range of 72.1 days to 83 days in its decision making.
3.15 Based on the 12 UK patients in PN001, the company's original model assumed that approximately 5% of patients would have intravenous letermovir. However, the ERG considered that the proportion of patients in PN001 who had intravenous letermovir (27%) was more representative of UK practice. The clinical experts explained that because T‑cell depletion is commonly used in current practice, they tend to use treatments that are less toxic to the gut. Therefore, they agreed with the company's assumption that only 5% of patients would have intravenous letermovir. The committee agreed that the company's assumption about intravenous letermovir use was more appropriate than the ERG's. During consultation, the company explained that the intravenous formulation of letermovir was not available in England, and provided analyses assuming 100% oral letermovir use. The committee agreed to take account of current availability and clinical expert opinion in its decision making and concluded that it would consider 0% and 5% intravenous letermovir use.
3.16 The ERG was concerned that the company's original model assumed foscarnet use was 25%, which was too high, potentially overestimating the cost of pre-emptive therapy. The ERG's clinical experts stated that only 5% to 15% of patients would have foscarnet as part of their pre-emptive therapy. However, the clinical experts at the first meeting stated that its use varied between centres and often depended on the type of transplant received. For example, people having T‑cell depletion, which is common in NHS practice, would most likely have foscarnet because of their higher risk of earlier CMV reactivation. Therefore, the clinical experts suggested that the use of foscarnet is closer to 15% to 25%, and the committee agreed that foscarnet use in the model should be between 15% and 25%. During consultation the company updated its base case and assumed foscarnet use was 20%, which the committee concluded was appropriate.
3.17 The company's updated base case was based on the ERG's original base case, but included:
an increased discount in letermovir's commercial arrangement
24‑week trial data instead of 48‑week data (see section 3.10)
a mean duration of therapy of 72.1 days (see section 3.14)
a discounted disutility associated with graft versus host disease and disease relapse beyond the first year (see section 3.13)
concomitant use of ciclosporin A of 90% and foscarnet use of 20% (see section 3.16).
The committee noted that the company's updated base-case incremental cost-effectiveness ratio (ICER) was higher than its original base-case ICER and included some of its preferred assumptions. However, the company's updated base case did not use 48‑week data from PN001 (see section 3.10) and it assumed a treatment duration (72.1 days) at the lower end of the range the committee wished to consider (see section 3.14). The company's updated deterministic base-case ICER for letermovir (including the updated commercial arrangement) compared with placebo increased from £10,904 (original base case) to £17,713 per QALY gained.
3.18 The ERG's updated base case was similar to that of the company's, but included:
a mean duration of therapy of 83 days (see section 3.14)
correction of the company's misinterpretation of the ERG model settings for some utility value inputs.
The committee noted using both the 48‑week trial data and assuming a longer mean treatment duration for letermovir increased the ICER. The ERG's updated base-case ICER for letermovir (including the updated commercial arrangement) compared with placebo decreased from £27,536 (original base case) to £24,269 per QALY gained.
3.19 Having considered the company's and ERG's updated base-case ICERs, the committee agreed that the ERG's analysis aligned more closely with its preferred assumptions. However, it recalled that it would take into account the following considerations in its decision making:
Although a health-related quality-of-life benefit from preventing CMV reactivation was not shown in PN001, a benefit with letermovir was plausible (see section 3.8). It recognised that had these benefits been included in the model the ICER would decrease.
The company's model was too simplistic in that it did not fully capture the clinical signs and symptoms of CMV and their effect on quality of life (see section 3.9). It acknowledged that the effect of this on the ICER was unclear.
The estimated treatment duration for letermovir was uncertain and the committee agreed it would consider a mean treatment duration range of 72.1 days to 83 days (see section 3.14). It understood that assuming a shorter treatment duration would reduce the ICER and that the ERG's assumed treatment duration was at the top end of this range.
Intravenous letermovir use was assumed to be 5% in the ERG's updated base case, in line with clinical expert opinion (see section 3.15). The committee acknowledged that intravenous letermovir was not currently available in England, and assuming 100% oral use would decrease the ICER.
The committee noted that when taking into account its preferred assumptions and other considerations, the ICER would be below £24,269 per QALY gained and could be lower than £20,000 per QALY gained. But it recognised that there was uncertainty about:
the size of the all-cause mortality benefit associated with letermovir (see section 3.6)
the structure of the economic model (see section 3.9) and
the health-related quality-of-life benefit of letermovir (see section 3.8).
Having considered these factors, the committee agreed that the most plausible ICER for letermovir compared with placebo is within the range normally considered to be an acceptable use of NHS resources. Therefore, the committee recommended letermovir as an option for preventing CMV reactivation and disease after an HSCT in adults who are seropositive for CMV.
3.20 The committee agreed that there is an unmet need for an effective and tolerable treatment to prevent CMV reactivation and disease after an HSCT in adults who are seropositive for CMV. However, it concluded that it had taken all potential benefits into account in its decision making (see section 3.19).