4 Evidence and interpretation

The Appraisal Committee (appendix A) considered evidence from a number of sources (appendix B).

4.1 Clinical effectiveness

4.1.1 The Assessment Group identified 3 open-label, multicentre randomised controlled trials in people with cystic fibrosis and chronic P. aeruginosa lung infection for the technologies: 2 for colistimethate sodium DPI (COLO/DPI/02/05 and COLO/DPI/02/06) and 1 for tobramycin DPI (EAGER). The manufacturer of tobramycin DPI included details of 2 randomised controlled trials in their submission: EVOLVE and EAGER. The Assessment Group did not include the EVOLVE trial in its review because the comparator was placebo, which was not in line with the scope provided by NICE.

Colistimethate sodium DPI

4.1.2 COLO/DPI/02/05 was a crossover trial comparing the safety of colistimethate sodium DPI and colistimethate sodium nebuliser solution in children and adults. The trial was carried out at 3 centres in the UK, and the population consisted of 16 people of whom 37.5% were younger than 13 years. The trial compared 1 capsule of colistimethate sodium DPI given twice daily for 4 weeks with 2 million units of colistimethate sodium nebuliser solution given twice daily for 4 weeks. The primary objective was to assess the safety (clinical tolerability and laboratory safety) of 4 weeks' treatment with colistimethate sodium DPI compared with 4 weeks' treatment with nebulised colistimethate sodium. All patients (n=16) receiving the DPI reported treatment-emergent adverse events compared with 9 out of 15 patients (60.0%) receiving the nebuliser solution. The most common treatment-related adverse events were gastrointestinal disorders (87.5% of patients) followed by cough and throat irritation (both in 81.3% of patients) in the DPI group, and cough (46.7% of patients) followed by wheezing (33.3% of patients) in the nebuliser solution group.

4.1.3 COLO/DPI/02/06 compared twice-daily doses of 125 mg colistimethate sodium DPI administered by a Turbospin device with twice-daily doses of 300 mg tobramycin nebuliser solution for nebuliser inhalation for 24 weeks. The trial was of a non-inferiority design, that is, it was intended to show that the effect of colistimethate sodium DPI was not statistically significantly worse than that of nebulised tobramycin. It was carried out in 66 centres in the European Union, Russia and the Ukraine. It included 373 people aged 6 to 56 years (mean age 21 years, standard deviation [SD]=9.49) with cystic fibrosis who had chronic pseudomonas lung infection (2 or more positive cultures in the previous 6 months) that would normally be treated with inhaled antibiotics. Other inclusion criteria were an FEV1% of between 25 and 75% of predicted and use of tobramycin nebuliser solution for inhalation (a minimum of 2 on/off cycles) before randomisation. The primary end point of the trial was the mean change in FEV1% predicted after 24 weeks of treatment. The trial was powered to detect non-inferiority, that is, a mean change in FEV1% from baseline to week 24 of no less than –3% at the lower end of the two-sided 95% confidence interval (CI) between treatment groups. Secondary outcomes included frequency of and time to exacerbations, adverse events, quality of life and physical changes. Three analyses were performed on the primary efficacy outcome: analysis of co-variance (ANCOVA), log-transformed analysis, and non-parametric analysis. Each of these 3 analyses were performed on 2 populations, the intention to treat (ITT) population (all randomised patients who received at least 1 dose of a study drug) and the per-protocol (PP) population (all randomised patients who received at least 1 dose of the study drug and who could be classified as 'efficacy evaluable'). For each population 2 sets of data were used, one where missing values were replaced using the last observation carried forward (LOCF) method and the other consisting of completers, creating in total 12 sets of results. The manufacturer stated that the primary efficacy analysis was done using LOCF data but that both ITT and PP populations were of equal importance for demonstrating non-inferiority.

4.1.4 For the primary end point of mean difference in change in FEV1% predicted after 24 weeks of treatment, the result from the ANCOVA analysis on the ITT population using LOCF data was −1.16% (95% CI −3.15% to 0.84%) suggesting that colistimethate sodium DPI was marginally less efficacious than tobramycin nebuliser solution (because the non-inferiority criterion was not met). The results of the log-transformed and non-parametric ITT population LOCF data analyses were –0.98% (95% CI −2.74% to 0.86%) and −0.56% (95% CI −2.16% to 1.00%) respectively, suggesting in both cases that colistimethate sodium DPI was non-inferior to nebulised tobramycin. For the PP population the ANCOVA, log-transformed and non-parametric analyses using LOCF data indicated that the non-inferiority hypothesis was satisfied for non-parametric analysis only (ANCOVA −1.49% [95% CI −3.79% to 0.81%], log-transformed −1.10% [95% CI –3.08% to 0.97%] and non-parametric −0.67% [95% CI −2.57% to 1.16%]).

4.1.5 Results for the secondary outcomes reported for the COLO/DPI/02/06 trial indicated that the proportion of people experiencing at least 1 protocol-defined acute exacerbation (at least 4 symptoms of worsening lung function) was higher in the colistimethate sodium DPI group than in the nebulised tobramycin group (31.1% versus 26.1%). The mean time to acute respiratory exacerbation was longer among the colistimethate sodium DPI group than the nebulised tobramycin group (63.7 days compared with 59.4 days). The mean duration of antibiotic administration for acute exacerbations was slightly less for colistimethate sodium DPI than for nebulised tobramycin (13.6 days compared with 14.4 days).

4.1.6 Adverse events were more common in the colistimethate sodium DPI group (93.6% of patients) than in the nebulised tobramycin group (89.1% of patients). A total of 27 patients (22/187 patients [11.8%] in the DPI group and 5/193 patients [2.6%] in the nebulised group) were withdrawn from the COLO/DPI/02/06 trial because of an adverse event. Adverse events that appeared to be worse in the colistimethate sodium DPI group compared with the nebulised tobramycin group were cough, throat irritation and dysgeusia. Fewer patients adhered to medication in the colistimethate sodium DPI group than in the nebulised tobramycin group (67% compared with 70% respectively adhered to over 75% of doses). The Assessment Group noted that throughout the 24 weeks, resistance to colistimethate sodium remained low (less than or equal to 1.1%) and resistance to tobramycin at 8 mg/litre breakpoint did not change significantly from baseline during the study.

4.1.7 Quality-of-life data using a preference-based measure in line with the NICE reference case were not collected in the COLO/DPI/02/06 trial. Health-related quality of life was evaluated in the trial by the Cystic Fibrosis Questionnaire Revised (CFQ-R) (a disease-specific instrument designed to measure impact on overall health, daily life, perceived wellbeing and symptoms) at several time points. No statistically significant differences in quality of life between baseline and week 24 were seen between the colistimethate sodium DPI and tobramycin nebuliser solution groups. At week 24, quality-of-life assessments were in favour of colistimethate sodium DPI in most CFQ-R domains in the ITT population, but none reached statistical significance.

Tobramycin DPI

4.1.8 The EAGER trial was designed as a non-inferiority trial in which twice-daily doses of 112 mg tobramycin DPI administered via a Podhaler device were compared with twice-daily doses of 300 mg nebulised tobramycin for 24 weeks. The study was powered to detect non-inferiority at a margin of no less than 6% of the lower one-sided 85% confidence limit of the mean difference in relative change in FEV1% between the treatment groups at 20 weeks. The study (n=533) included people aged 6 years or above (DPI group mean age 26 years [SD=11.4], nebulised group mean 25 years [SD=10.2]) with cystic fibrosis who had chronic pseudomonas lung infection (defined as a positive culture within 6 months of screening and at baseline) and whose FEV1% was between 25 and 75%. Of the study population, 55% were male. The primary end point was the incidence of adverse events; however the trial was powered to detect change in FEV1% predicted from baseline to week 20. Other outcomes included microbiological measures (such as bacterial susceptibility), body mass index changes and laboratory safety end points.

4.1.9 The ITT EAGER results were based on complete case analysis and did not incorporate imputation (that is, filling in missing data with possible plausible values). Results indicated that tobramycin DPI was associated with an improved mean FEV1% predicted compared with nebulised tobramycin at 20 weeks of +0.59% (standard error [SE] 0.92). The manufacturer reported non-inferiority (supported by least squares mean difference relative change of 1.1% [SE 1.75] which has a lower limit of the one-sided 85% confidence interval within the predicted 6% margin for non-inferiority). Only limited data were presented for lung function at 24 weeks and the Assessment Group noted that they would expect FEV1% levels to be lower at that stage (following a month without tobramycin). The Assessment Group calculated the mean FEV1% at 24 weeks to be 53.9% for the DPI group and 50.7% for the nebulised tobramycin group indicating a reduction in lung function from 20-week values (55.97% [DPI] and 55.28% [nebulised tobramycin]).

4.1.10 In the EAGER trial, pseudomonal resistance (8 mg/litre breakpoint) to tobramycin started at around 20% (the baseline value was at the end of 28 days off previous treatment) and was lower at 24 weeks in both groups (also at the end of 28 days off treatment). P. aeruginosa sputum density data showed a reduction from baseline to week 20 in the tobramycin DPI group (−1.61 log colony-forming units) compared with the nebulised group (−0.77 log colony-forming units).

4.1.11 The manufacturer did not provide details on all exacerbations from the EAGER trial; however the measure of lung disorder was used by the Assessment Group as a proxy to give an indication as to the number of participants experiencing an exacerbation. The percentage of patients experiencing lung disorders was greater in the tobramycin DPI group (33.8%) than in the nebulised tobramycin group (30.1%). However, no information was provided on the number of events experienced by patients and therefore it is not known whether some trial participants experienced multiple events. Additionally no information was provided on the time to exacerbation. Mean duration of anti-pseudomonal antibiotic treatment was also slightly shorter in the tobramycin DPI group than in the nebulised tobramycin group (30.9 days compared with 33.4 days). The number of trial participants receiving additional anti-pseudomonal treatments was higher in the tobramycin DPI group. The rate of discontinuation of treatment was higher in the tobramycin DPI group than in the nebulised tobramycin group (83/308 patients [26.9%] compared with 38/309 patients [18.2%]).

4.1.12 Adverse events were more common in the tobramycin DPI group (90.3% of patients) than in the nebulised tobramycin group (84.2% of patients). A total of 57 patients (40 [13%] in the DPI group and 17 [8%] in the nebulised group) were withdrawn from the EAGER trial because of an adverse event. The most common adverse event in the 2 groups was cough (48.4% in the DPI group and 31.1% in the nebulised group). Adverse events that appeared to be worse in the tobramycin DPI group than in the nebulised group included cough and dysphonia. The EAGER trial did not define the assessment of adherence but stated that it was 'generally high' with over 90% adherence in both groups.

4.1.13 Quality-of-life data using a preference-based measure in line with the NICE reference case were not collected in the EAGER trial. The manufacturer reported data from a treatment satisfaction questionnaire for medication results that showed higher values in the tobramycin DPI group than in the nebulised group. Least squares mean difference averages indicated an improvement over visits for effectiveness (9.36 [SE 1.46]), for side effects (−0.5 [SE 1.22]), for convenience (24.35 [SE 1.55]) and for global satisfaction (5.20 [SE 1.66]) for tobramycin DPI compared with nebulised tobramycin.

4.1.14 The Assessment Group commented that the quality of the included studies for both colistimethate sodium DPI and tobramycin DPI was generally poor to moderate. None of the included trials scored well on all risk of bias items, particularly on the issues of blinding and non-adherence to the current European Medicines Agency (EMA) research guidelines. The Assessment Group judged that this could lead to selection and reporting bias for subjective outcomes such as adverse events, inaccuracies and imprecision in the results, and might limit the generalisability of the studies. The Assessment Group judged that follow-up was not long enough to detect slowing of the rate of decline in respiratory function, according to current EMA research guidelines, nor for any assessment of mortality. The Assessment Group also highlighted that because FEV1% is a surrogate outcome, the EMA recommend that it should be considered alongside microbiological outcomes and 'harder' clinically relevant outcomes such as frequency of exacerbations and antibiotic use. Both tobramycin DPI and colistimethate sodium DPI seemed to result in more exacerbations and more people experiencing exacerbations than nebulised tobramycin, but slightly less time on antibiotics. The incidence of adverse events was similar between groups within the trials except for cough, which had a higher incidence in both DPI groups. More patients in the DPI groups withdrew because of adverse events in both EAGER and COLO/DPI/02/06. The Assessment Group judged that it was not possible to determine whether the changes seen in the colistimethate sodium DPI group were significantly different to the changes seen in the tobramycin DPI group because of different trial designs, a lack of data at 24 weeks, different population analyses of results and uncertain comparability of patient characteristics at baseline. The Assessment Group was unable to draw definite conclusions as to the relative efficacy of any intervention except where there was direct evidence for the dry powder formulations compared with nebulised tobramycin.

Mixed treatment comparison (tobramycin DPI)

4.1.15 The manufacturer of tobramycin DPI performed a literature review followed by a network meta-analysis using data from 7 studies to assess the clinical effectiveness of tobramycin DPI given by Podhaler relative to TOBI (tobramycin nebuliser solution), Bramitob (tobramycin nebuliser solution made by a different manufacturer), nebulised colistimethate sodium, aztreonam nebuliser solution and placebo. Colistimethate sodium DPI was not included in the analysis because study results for this technology were not in the public domain. The manufacturer explained that in the network meta-analysis there were underlying differences in the included trials in terms of trial populations, outcomes used and study design. The Assessment Group did not carry out an evaluation or provide specific comments on the manufacturer's network meta-analysis because it judged that given the available evidence, such an analysis was not feasible. The results of the network meta-analysis found no statistically significant differences between comparators in terms of efficacy at 4 weeks (measured by FEV1% predicted).

4.2 Cost effectiveness

4.2.1 The cost-effectiveness evidence consisted of: a model by the manufacturer (Forest) for colistimethate sodium DPI; an analysis of the manufacturer's model by the Assessment Group; a de novo model produced by the Assessment Group; and Assessment Group economic analyses in response to the original and revised patient access schemes submitted by the manufacturer of colistimethate sodium DPI and the patient access scheme submitted by the manufacturer of tobramycin DPI. The manufacturer of tobramycin DPI proposed that a cost-minimisation analysis should be undertaken, because their network meta-analysis had found that the included anti-pseudomonal treatments had similar efficacy.

4.2.2 The Assessment Group performed a systematic review of published literature and identified 3 economic evaluations: a cost–consequence analysis of home intravenous treatment in people with cystic fibrosis (Wolter et al. 1997); a cost-effectiveness analysis comparing hospital and home care in people with cystic fibrosis (Thornton et al. 2005); and a cost–consequence analysis of inhaled tobramycin nebuliser solution in people with cystic fibrosis (Iles et al. 2003). None of these 3 studies relate to either colistimethate sodium DPI or tobramycin DPI. The Assessment Group judged that these studies provided some information about the costs and outcomes of the comparator therapies and explained some of the key methodological problems surrounding the economic evaluation of treatments for cystic fibrosis, for example, short-term lung function improvements rather than quality-adjusted life year (QALY) gains.

Manufacturer's model on the cost effectiveness of colistimethate sodium DPI

4.2.3 Forest used a cohort-based decision analysis to compare colistimethate sodium DPI with nebulised tobramycin. The population modelled were people aged 6 years or older with documented cystic fibrosis who had chronic pseudomonas lung infection (the same population as the COLO/DPI/02/06 trial). The model had a 24-week time horizon and took a UK NHS perspective. The main assumptions of the model were that FEV1% determines year 1 or 2 mortality risk, utility is fixed at 0.68, there is no reduction in quality of life during an exacerbation or for an adverse event and all patients have a fixed maximum life expectancy of 37.4 years.

4.2.4 The COLO/DPI/02/06 trial did not collect health-related quality-of-life data using a preference-based instrument. Utility estimates in the manufacturer's model were obtained from the CFQ used in the COLO/DPI/02/06 study, mapped to EQ-5D tariff values using regression equations with coefficients derived from patient-level data from a published study in people with cystic fibrosis (Eidt-Koch et al. 2009). In this study, data were collected in 2006 across 4 cystic fibrosis centres in Germany. A cohort of 96 patients with cystic fibrosis completed both the German version of the CFQ and the EQ-5D-Y. Patients included in this study were generally young (the mean age was approximately 13 years, range 8 to 17 years) and mean FEV1% predicted was generally high in both the child and adolescent groups (93.6% and 90.7% respectively).The responses to the EQ-5D-Y were valued using the EQ-5D tariff. This mapping exercise produced a single utility value of 0.68 for people with cystic fibrosis with chronic P. aeruginosa lung infection.

4.2.5 Costs and QALYs based on treatment, life expectancy and exacerbations were then estimated by a cohort-based decision analysis incorporating an estimated mortality risk and this single utility value of 0.68 for people with cystic fibrosis. Predicted mortality differences between colistimethate sodium DPI and nebulised tobramycin were estimated by regression equations for mortality at 1 and 2 years using reported data from a retrospective analysis of the risk of mortality by Kerem et al. (1992). This study used the patient characteristics of FEV1% predicted, forced vital capacity, partial pressures of oxygen and carbon dioxide, sex, weight and height to predict mortality at 1 and 2 years. The manufacturer fitted several polynomial regression equations to the Kerem et al. data for mortality risk at 1 or 2 years by FEV1% group. The manufacturer then applied the best fit regression equation to patient-level data from COLO/DPI/02/06 to calculate the 1- and 2-year mortality risks for the intervention and comparator groups based on FEV1% predicted only.

4.2.6 The manufacturer's model included costs for acquisition of medication and costs associated with exacerbations. Acquisition costs for nebulised tobramycin were taken from BNF edition 61. The model assumed a cost per dose of tobramycin of £21.20 (annual cost of £7738, excluding VAT), corresponding to a regimen in which 2 doses of nebulised tobramycin are used each day, and each 28-day treatment period is followed by 28 days without nebulised tobramycin. The manufacturer's economic analysis priced colistimethate sodium DPI at parity with nebulised tobramycin. Exacerbations were costed from NHS reference costs but as there were no reference costs for cystic fibrosis exacerbations, the reference cost for an asthma admission with major comorbidities and/or complications without intubation was used. The numbers of exacerbations for people receiving colistimethate sodium DPI and nebulised tobramycin were calculated from patient-level data in the pivotal COLO/DPI/02/06 trial. The manufacturer reported results in terms of incremental net benefit with the results indicating that colistimethate sodium DPI dominated nebulised tobramycin, that is, it was more effective and cost less than nebulised tobramycin.

Assessment Group's analysis of the Forest model

4.2.7 The Assessment Group highlighted that there were limitations with the manufacturer's model and that it deviated from the NICE reference case in many ways: no discounting was carried out for costs because of the short time horizon; there were inconsistent time horizons for costs and health outcomes; results were presented as an incremental net benefit rather than an incremental cost per QALY gained; and no probabilistic sensitivity analysis was carried out. The Assessment Group also highlighted that the short study duration of the COLO/DPI/02/06 trial meant a high degree of censoring of mortality estimates. The Assessment Group also expressed substantial concerns in relation to the model using changes in FEV1% predicted measured in the short term to predict long-term mortality benefits. The Assessment Group highlighted the shortcomings of using particular sources of evidence (Kerem et al.) to derive survival estimates, the fixed life expectancy of 37.4 years used in the model, the use of the particular mapping method to produce utility estimates, the potential biases in the modelling of exacerbation rates and omission of relevant costs and health impacts.

4.2.8 The Assessment Group carried out a literature search and concluded that FEV1% alone is unlikely to represent a valid independent predictor of survival. It judged that the survival estimated in the manufacturer's model derived from the Kerem et al. data lacked validity because of the potential for confounding from other prognostic factors. The Assessment Group noted that the use of the regression equation to link mortality to lung function was unnecessary and unjustified because it should have been possible to directly apply the Kerem et al. mortality probabilities to the categorical FEV1% predicted bands from the COLO/DPI/02/06 trial.

4.2.9 The Assessment Group carried out a re-analysis of the manufacturer's model to try and correct for some of the problems identified with the model. It acknowledged that it could not fully resolve the problems regarding the time horizon, the health impact of adverse events or the uncertainty surrounding the QALY benefits for colistimethate sodium DPI. The results of the revised analysis found that if colistimethate sodium DPI is priced lower than nebulised tobramycin it may dominate because of lower costs from avoided exacerbations and incremental QALY gains. If the price is higher than nebulised tobramycin, the incremental cost per QALY gained ranges from £42,872 to £485,550 depending on assumptions about time horizons and drug acquisition costs.

Assessment Group de novo model

4.2.10 The Assessment Group developed a de novo probabilistic state transition model to compare colistimethate sodium DPI with nebulised tobramycin in people with cystic fibrosis who had chronic P. aeruginosa lung infection. It did not initially include tobramycin DPI in the model because patient-level FEV1 data were not available from the manufacturer at the time the report was produced (this information was subsequently provided by the manufacturer when the patient access scheme for tobramycin DPI was considered) and information on the price of tobramycin DPI was not available until February 2012. Other comparator treatments listed in the scope were not included in the analysis because of lack of data. The model had a lifetime time horizon and a UK NHS perspective. The cycle length was 24 weeks and treatment duration was assumed to be equivalent between the competing treatments. The model assumed that treatments were administered in line with their marketing authorisation. Mean survival for patients in the model was estimated using data from Dodge et al. 2009. This study reported survival data up to the end of 2003 for all people with cystic fibrosis born in the UK between 1968 and 1992 collated by active enquiry of cystic fibrosis clinics and other hospital consultants. The model had 5 health states: FEV1 70–99%, FEV1 40–69%, FEV1 less than 40%, post lung transplant and death. The 24-week probabilities for transition between health states were based on patient-level data from the COL/DPI/02/06 trial. The Assessment Group judged that some people may switch between colistimethate sodium and tobramycin at some point in their lives. However, this feature was not included in the model because clinical-effectiveness data on the effect of treatment switching were not available.

4.2.11 Different levels of health-related quality of life were assumed for each health state. Total QALYs were calculated as the total time in each health state weighted by the respective utility for that health state, less any QALY losses resulting from exacerbations. Utility estimates were obtained from the Bradley (2010) study. No mortality gain from improved FEV1% predicted was incorporated into the model. A further assumption was that there was no quality of life reduction with adverse events.

4.2.12 Costs in each treatment group included drug acquisition costs and the costs of managing exacerbations (either in hospital or at home). Potential cost savings associated with reduced maintenance of nebulisers were also included in the economic analysis for the DPI group. Costs associated with follow-up and concomitant medications were assumed to be equivalent between treatment groups.

4.2.13 The Assessment Group produced a set of analyses that included 5 different pricing scenarios for colistimethate sodium, because at that time the list price for colistimethate sodium DPI was not available. The results of these analyses were superseded by additional analyses which included the confirmed list price and the original and revised patient access schemes (see sections 4.2.15 to 4.2.19). The analyses resulted in an incremental QALY loss (0.13) over a patient's lifetime associated with colistimethate sodium DPI compared with nebulised tobramycin. If colistimethate sodium DPI is priced at £15.98 or above per dose, it is dominated by nebulised tobramycin. If colistimethate sodium DPI is priced at £10.60 per dose, the model suggests an incremental cost-effectiveness ratio (ICER) of £23,788 saved per QALY lost. However, the Assessment Group noted that this ICER lies in the south-west quadrant of the cost-effectiveness plane (that is, less effective and less expensive) and represents a QALY loss and cost savings for colistimethate sodium DPI compared with nebulised tobramycin.

4.2.14 The Assessment Group stated that although there is considerable uncertainty surrounding the extrapolation of the COLO/DPI/02/06 trial data, the conclusions of the analysis appear robust. This is because the uncertainty imposed by long-term extrapolation of short-term results has little bearing on the model conclusions because colistimethate sodium DPI remains dominated if its price is set higher than nebulised tobramycin. The Assessment Group highlighted that the deterministic sensitivity analysis (performed on utilities, transition probabilities, utility decrement from exacerbation, cost of hospitalisation and point estimates for parameters) showed that the choice of utility value had the most substantial effect on the cost-effectiveness estimate. The Assessment Group also re-ran the model using a 24-week time horizon. The results indicated an expected tiny decrease (0.002) in QALYs for colistimethate sodium DPI compared with nebulised tobramycin. If priced at £9.11 per dose or £10.60 per dose, colistimethate sodium DPI is expected to be less expensive than nebulised tobramycin and results in ICERs of £276,814 and £49,596 saved per QALY lost respectively. As in the long-term model these positive ICERs are the result of cost savings and QALY reductions. If colistimethate sodium DPI is priced at £15.98 or above per dose, it is dominated by nebulised tobramycin, as in the long-term model.

Colistimethate sodium DPI patient access schemes

4.2.15 The Assessment Group also carried out additional analyses in response to the patient access schemes submitted by the manufacturers of colistimethate sodium (sections 4.2.15 to 4.2.19) and tobramycin DPI (sections 4.2.20 to 4.2.25). With regard to the original colistimethate sodium DPI scheme, only the price of colistimethate sodium DPI was amended; all other assumptions and parameters in the model remained unchanged (sections 4.2.10–4.2.12). The Assessment Group explained that when the list price was modelled (this having been finalised and provided at this stage), colistimethate sodium DPI was dominated by nebulised tobramycin. When the patient access scheme discount was incorporated, the results demonstrated that colistimethate sodium DPI was less effective and less expensive than nebulised tobramycin, the ICER being £52,672 saved per QALY lost.

4.2.16 The Assessment Group also carried out a probabilistic sensitivity analysis. The probability of colistimethate sodium DPI (at list price) being more cost effective than nebulised tobramycin (at list price) at a threshold of £20,000 and £30,000 per QALY gained was zero. The results also suggested that when the original patient access scheme is included in the analysis, the probability of colistimethate sodium DPI being more cost effective than nebulised tobramycin is approximately 0.79 at a threshold of £20,000 saved per QALY lost, and 0.66 at a threshold of £30,000 saved per QALY lost.

4.2.17 Additionally the Assessment Group carried out a sensitivity analysis using the Commercial Medicines Unit (CMU) Electronic Marketing Information Tool (E-MIT) price rather than the BNF list price for nebulised tobramycin. In September 2012, the estimated price for nebulised tobramycin paid by the NHS was £970.12 rather than the BNF list price of £1187. The results of this analysis suggest that on the basis of the original patient access scheme price for colistimethate sodium DPI and the CMU E-MIT price for nebulised tobramycin, colistimethate sodium DPI is dominated by nebulised tobramycin.

4.2.18 The Assessment Group concluded that at its list price, colistimethate sodium DPI is expected to be dominated by nebulised tobramycin (at list price). When the original patient access scheme is included in the analysis, the incremental cost effectiveness of colistimethate sodium DPI compared with nebulised tobramycin (at list price) is expected be around £52,700 saved per QALY lost (that is, colistimethate sodium DPI is less effective but also less expensive than nebulised tobramycin). When the CMU E-MIT price of nebulised tobramycin is compared with both list and original patient access scheme prices of colistimethate sodium DPI, colistimethate sodium DPI is dominated by nebulised tobramycin.

4.2.19 The Assessment Group also carried out additional analyses in response to the revised patient access scheme submitted by the manufacturer of colistimethate sodium as part of its response to the appraisal consultation document (ACD) consultation. Only the price of colistimethate sodium DPI was amended in the model; all other assumptions and parameters in the model remained unchanged (sections 4.2.10–4.2.12). When the revised patient access scheme discount was incorporated, the incremental QALY was −0.13, and the incremental cost was −£37,946 compared to nebulised tobramycin (list price). These results demonstrated that colistimethate sodium DPI was less effective and less expensive than nebulised tobramycin, the ICER being £288,563 saved per QALY lost. The probabilistic sensitivity analysis associated with these analyses estimated that with the revised patient access scheme, colistimethate sodium DPI had a probability of 1.0 of being cost effective at the £20,000 and £30,000 per QALY gained thresholds. The Assessment Group again carried out a sensitivity analysis using the CMU E-MIT price rather than the BNF list price for nebulised tobramycin sodium DPI. The results of this analysis suggest that on the basis of the revised patient access scheme price for colistimethate sodium DPI and the CMU E-MIT price for nebulised tobramycin, the resulting ICER was £154,916 saved per QALY lost (incremental cost and QALY of −£20,371 and −0.13 respectively). However, as all these analyses included only a comparison with nebulised tobramycin the cost effectiveness of colistimethate sodium DPI relative to any other comparator is uncertain.

Tobramycin DPI patient access scheme

4.2.20 In response to an agreed patient access scheme for tobramycin DPI, the Assessment Group carried out additional analyses evaluating the cost effectiveness of tobramycin DPI compared with nebulised tobramycin. This analysis used the Assessment Group de novo model (used for colistimethate sodium DPI described above), but transition probabilities between health states and exacerbation parameters were based on those observed in the EAGER trial. The manufacturer provided individual patient-level FEV1% data together with aggregated data on exacerbations from the EAGER trial, which had been previously unavailable to the Assessment Group. All other base-case assumptions in the model were unchanged (see sections 4.2.10–4.2.12).

4.2.21 The Assessment Group carried out analyses using 3 sets of FEV1 data from the EAGER trial: 1) pre-dose FEV1% at week 0 to FEV1% at week 24; 2) pre-dose FEV1% at week 0 to pre-dose FEV1% at week 20; 3) post-dose FEV1% at week 0 to post-dose FEV1% at week 20. The Assessment Group highlighted that there was uncertainty around which of these analyses should be considered the most reliable because of the effect of tobramycin on FEV1% soon after administration (rather than off treatment 4 weeks later). The Assessment Group judged that the analysis 'Pre-dose FEV1% at week 0 to FEV1% at week 24' (analysis 1) was the most appropriate and should be considered to be the base-case analysis, but noted that this was not the ITT population because it only included trial participants for whom baseline and week 24 lung function values had been recorded. The Assessment Group highlighted that there was a notable amount of missing data with higher rates of attrition in the tobramycin DPI arm than in the nebulised tobramycin arm and therefore the data may reflect a 'best case scenario' whereby only participants who had a response to the drug are captured in either group.

4.2.22 In the Assessment Group model, the incremental QALY gain for tobramycin DPI compared with nebulised tobramycin was 0.34 (pre-dose FEV1% at week 0 to FEV1% at week 24) and the incremental costs were £42,453. The cost-effectiveness results for the base-case analysis of the list prices of tobramycin DPI compared to nebulised tobramycin gave an ICER of £123,563 per QALY gained (without the patient access scheme applied). With the patient access scheme applied, tobramycin DPI dominated nebulised tobramycin (at list price) with incremental cost savings of £19,275.

4.2.23 The Assessment Group also carried out a one-way sensitivity analysis on the following parameters: deterministic point estimates of parameters on utility values, transition probabilities, utility decrement for exacerbations and costs of hospitalisation. These analyses indicated that tobramycin DPI dominated nebulised tobramycin when the patient access scheme was included in analysis 1 (pre-dose FEV1% at week 0 to FEV1% at week 24) except for the scenario in which the transition probabilities for nebulised tobramycin were set as equal to those for tobramycin DPI.

4.2.24 The Assessment Group also carried out probabilistic sensitivity analyses. The probability of tobramycin DPI (at list price) being more cost effective than nebulised tobramycin (at list price) at thresholds of £20,000 and £30,000 per QALY gained was zero for all 3 ways of interpreting the FEV1 data described in section 4.2.21. The probability of tobramycin DPI (with the inclusion of the patient access scheme) being more cost effective than nebulised tobramycin (at list price) at thresholds of £20,000 and £30,000 per QALY gained was 1.00 for all of the analyses: pre-dose FEV1% at week 0 to FEV1% at week 24; pre-dose FEV1% at week 0 to pre-dose FEV1% at week 20; and post-dose FEV1% at week 0 to post-dose FEV1% at week 20.

4.2.25 The Assessment Group also carried out an additional sensitivity analysis using the price of nebulised tobramycin based on estimates from the CMU E-MIT (£970.12) rather than the BNF list price of £1187. The results found that with the introduction of the patient access scheme, tobramycin DPI consistently dominated nebulised tobramycin irrespective of which FEV1% data were used. Probabilistic sensitivity analyses indicated that the probability of tobramycin DPI being more cost effective than nebulised tobramycin was 1.0 at a threshold of £20,000 per QALY gained and 0.99 at a threshold of £30,000 per QALY gained, for the preferred analysis based on pre-dose FEV1% at week 0 to FEV1% at week 24. The Assessment Group concluded that the introduction of the patient access scheme could be expected to result in tobramycin DPI consistently dominating nebulised tobramycin irrespective of which FEV1% data are used. However, the Assessment Group pointed out that uncertainties remained:

  • Because of absence of sufficient evidence relating to any other comparator, the economic analysis of tobramycin DPI is restricted solely to an economic comparison with nebulised tobramycin.

  • Data on minor and major exacerbations were not collected in the EAGER trial. The Assessment Group explained that lung disorder may represent a reasonable proxy for pulmonary or cystic fibrosis exacerbations and hence these data were used to estimate the probability of any exacerbation (minor or major) in each treatment group. The mean probability of exacerbation for tobramycin DPI was estimated to be 0.34 and the mean probability of exacerbation with nebulised tobramycin was 0.30. The manufacturer provided trial data estimates of the number of patients receiving any new antibiotic, total days used, and the number of patients who needed both additional antibiotic treatment and hospitalisation in each treatment group. The estimates of the number of patients who needed both additional antibiotic treatment and hospitalisation, combined with the estimated number of exacerbation events from Konstan et al., were used to produce an estimate of the pooled probability that an exacerbation event was major. The Assessment Group explained that this estimate is very similar in terms of number of exacerbations to that derived from the COLO/DPI/02/06 trial but the validity of this estimate remains questionable.

  • Much of the incremental benefit is driven by the small QALY gains observed within the trial period which are inflated considerably by extrapolating over the patient's remaining lifetime. Over the patient's lifetime this incremental QALY gain is expected to range from 0.04 QALYs (post-dose 0–20 weeks) to 0.34 QALYs (pre-dose 0–24 weeks).

  • Missing data and high attrition rates in the EAGER trial for tobramycin DPI mean that transition data may reflect a best-case scenario in which only responders are captured in either group and therefore cost effectiveness may be less than that estimated.

4.3 Consideration of the evidence

4.3.1 The Appraisal Committee reviewed the data available on the clinical and cost effectiveness of colistimethate sodium DPI and tobramycin DPI for treating pseudomonas lung infection in cystic fibrosis, having considered evidence on the nature of chronic pseudomonas lung infection in people with cystic fibrosis and the value placed on the benefits of colistimethate sodium DPI and tobramycin DPI by people with the condition, those who represent them, and clinical specialists. It also took into account the effective use of NHS resources.

4.3.2 The Committee considered the current treatment pathway for people with cystic fibrosis with chronic P. aeruginosa lung infection. The Committee heard from clinical specialists that treatment is generally patient driven and that the most important outcomes for the patient that influence treatment decisions are the person's quality of life, treatment burden, maintaining good lung function and reducing the incidence of exacerbations. The Committee heard from clinical specialists that first-line treatment for chronic P. aeruginosa lung infection routinely starts with nebulised colistimethate sodium (unless it is contraindicated), this choice being largely based on cost. If there is no response, an unacceptable adverse event profile, an excessive number of acute exacerbations or a loss of lung function, then treatment is switched routinely to nebulised tobramycin. Tobramycin is administered in cycles of 28 days on treatment, followed by 28 days off treatment, in accordance with the marketing authorisation. The Committee noted that this was in line with the national guidelines from the Cystic Fibrosis Trust. The clinical specialists advised that having 28 days off treatment was not favoured by some people with cystic fibrosis and some clinicians, and that either nebulised colistimethate sodium during the 28-day off period or continuous half-dose nebulised tobramycin were used in some patients. The Committee noted that the use of colistimethate sodium only in the tobramycin 28-day off period and the use of continuous half-dose tobramycin by some people on nebulised tobramycin were outside the respective marketing authorisations, and heard from the clinical specialists that there was no clinical-effectiveness evidence for such approaches to treatment. The Committee therefore understood that the usual treatment pathway was for nebulised colistimethate sodium to be used first and then nebulised tobramycin second.

4.3.3 The Committee discussed the comments received during consultation on the ACD regarding the current treatment pathway. Some indicated that in the majority of adult specialist centres in the UK there has been a move away from offering nebulised colistimethate sodium as initial treatment followed by tobramycin, towards alternating therapy between tobramycin and colistimethate sodium on a monthly basis. Other comments indicated that it was still usual for nebulised colistimethate sodium to be used first and then patients were switched to nebulised tobramycin if there were problems with nebulised colistimethate sodium or if it did not work well enough. The Committee heard from the Assessment Group that information from their clinical experts suggested that less than 25% of people with cystic fibrosis and chronic P. aeruginosa lung infection would receive an alternating therapy regimen. The Committee therefore concluded that some people with cystic fibrosis and chronic pseudomonas lung infection may receive alternating tobramycin and colistimethate sodium treatment in clinical practice. The Committee noted the increased cost of such alternating antibiotic regimens and that it had not been presented with any evidence as to the clinical effectiveness of this approach by the Assessment Group or the manufacturers or during consultation. It concluded that because there was no evidence of the clinical effectiveness of using these antibiotics in an alternating regimen, it could not consider this issue further.

4.3.4 The Committee discussed the appropriate comparators for colistimethate sodium and tobramycin DPIs. It heard from the manufacturer of colistimethate sodium DPI that the EMA had indicated that the most appropriate comparator at the time of the study design for its pivotal trial would be nebulised tobramycin because this was the only licensed comparator in all of the study site countries. The Committee agreed that given the current clinical pathway, ideally it would have liked to have seen effectiveness evidence comparing colistimethate sodium DPI with nebulised colistimethate sodium and also whether there was evidence of any clinical benefit of colistimethate sodium DPI in people being switched from nebulised colistimethate sodium because of lack of efficacy. Taking into account the sequence of inhaled antibiotics currently used in the treatment pathway in the UK (see sections 4.3.2 and 4.3.3) and the clinical specialists' opinion that clinicians would switch from one antibiotic to another (whatever the preparation), the Committee concluded that the most appropriate comparator for colistimethate sodium DPI would be nebulised colistimethate sodium and the most appropriate comparator for tobramycin DPI would be nebulised tobramycin.

4.3.5 The Committee considered the treatment burden associated with cystic fibrosis. The Committee heard from patient experts how time-consuming all the treatments can be, taking up to 4 hours each day and imposing on other daily activities. The Committee noted that most people with cystic fibrosis use nebulisers for administration of other daily treatments as well as for antibiotics for pseudomonas lung infection. The Committee heard from patient experts how using a nebuliser also involves preparation of the treatment and cleaning of equipment, both of which add to the treatment burden. The Committee recognised that the strict routine and amount of time spent receiving treatment have a significant impact on the daily activities of people with cystic fibrosis and their families. The Committee concluded that reducing the time that people with cystic fibrosis spend receiving treatment would be beneficial in improving the quality of life of people with cystic fibrosis and their families.

4.3.6 The Committee discussed the impact of long-term treatment with nebulised antibiotics in people with cystic fibrosis. The Committee heard from clinical specialists and patient experts that adherence to medication was variable and did not just relate to nebulisers but also to inhalers and oral treatments. The Committee heard from clinical specialists that in clinical practice adherence to nebulised treatments was approximately 30–60%. The patient experts commented that out of all of their treatments, the ones they were least likely to adhere to were those administered with a nebuliser. The Committee also heard from the patient experts that older nebulisers are not easily portable, take up a lot of space in the home and involve greater costs to the patient in electricity usage. Additionally, the clinical specialists commented that newer, smaller and faster nebulisers are in common use and these greatly decrease the time of treatment but still involve drug preparation and cleaning. The Committee noted that nebulised colistimethate sodium and tobramycin are not licensed for use with these faster nebulisers but nevertheless accepted that it is usual practice for these types of fast nebulisers to be used, and this was confirmed by the clinical specialists. The Committee concluded that in clinical practice (rather than in clinical trials), people with cystic fibrosis may be more likely to adhere to a dry powder for inhalation treatment than a nebulised treatment in view of the speed and convenience of drug delivery.

Clinical effectiveness

4.3.7 The Committee discussed the clinical-effectiveness evidence for colistimethate sodium and tobramycin DPIs. The Committee noted that there was no clinical trial comparing the effectiveness of colistimethate sodium DPI with that of the preferred comparator, nebulised colistimethate sodium. The Committee discussed the clinical-effectiveness evidence from the COLO/DPI/02/06 trial which compared colistimethate sodium DPI with nebulised tobramycin and the EAGER trial which compared tobramycin DPI with nebulised tobramycin. The Committee noted that both trials were non-inferiority in design and therefore only assessed whether the interventions were not worse than nebulised tobramycin, where non-inferiority was accepted if the lower confidence limit for the difference between treatments in FEV1% was above a fixed percentage (the 95% confidence limit with a difference of –3% was chosen in COLO/DPI/02/06 and the 85% confidence interval with a difference of –6% in EAGER). The Committee noted the different confidence levels and margins of non-inferiority selected for the 2 key trials which it found surprising given it would have expected these to be based on similar clinical considerations. It was also aware that in the case of COLO/DPI/02/06, logarithmic transformations and a non-parametric approach using the median rather than mean values reduced the impact of extreme values and hence made it easier to achieve non-inferiority. The Committee concluded that it only had evidence exploring whether either dry powder formulation was not worse than nebulised tobramycin and no evidence to prove that either was more effective than or equivalent to nebulised tobramycin.

4.3.8 The Committee discussed the quality of the 2 key trials. It noted the Assessment Group's critique of the trials, in particular the fact that the manufacturers had not commented in their submissions on the quality of these trials in light of the current EMA research guidelines for the development of medicinal products for the treatment of cystic fibrosis. The Committee supported the Assessment Group's comments on the methodological limitations of both trials, such as lack of blinding, and agreed that this could have introduced selection and reporting bias for subjective outcomes such as adverse events and might limit the generalisability of the findings. The Committee concluded that the evidence base for assessing the clinical effectiveness of colistimethate sodium and tobramycin DPIs was of, at best, modest quality but that it was the best available.

4.3.9 The Committee discussed the results for lung function measurements in the 2 key trials and the results of these analyses in the ITT population. The Committee was aware that the primary outcome for the COLO/DPI/02/06 trial was respiratory function as measured by FEV1% predicted at 24 weeks. The Committee noted that the study was powered to detect a minimum change in the difference in FEV1% of −3% (based on the lower bound of a 95% confidence interval for the difference in means). The Committee noted that the non-inferiority criterion was not met in the pre-defined ANCOVA analysis for the ITT population but it acknowledged that in the logarithmic and non-parametric analyses the non-inferiority criterion had been met in the ITT population for the LOCF analyses. The Committee was additionally aware that the manufacturer had stated that both LOCF and completer analyses were of equal importance and that the non-inferiority criterion was only met for the ITT population in the non-parametric analysis using the completer dataset. The Committee noted that the EAGER trial was powered to detect non-inferiority at a margin of no less than 6% difference in the mean FEV1% between tobramycin DPI and nebulised tobramycin at the lower end of the one-sided 85% confidence interval and that results showed that tobramycin DPI was not statistically significantly worse than nebulised tobramycin with respect to change in mean FEV1% at 20 weeks. The Committee heard from the clinical specialists that FEV1% tends to fluctuate over the short term and the clinical specialists therefore agreed that the 24-week follow-up period of the trials was not sufficiently long enough to assess the impact of either colistimethate sodium DPI or tobramycin DPI on FEV1% predicted compared with nebulised tobramycin in the longer term. The Committee noted that both trials had resulted in relatively small changes in FEV1% levels at 24 weeks. Additionally, the Committee heard from the clinical specialists that small changes in FEV1% levels over a short term were not considered clinically meaningful in determining whether chronic P. aeruginosa lung infection was being effectively managed in clinical practice. The Committee discussed the nature of non-inferiority trials compared with trials designed to show equivalence and the short-term nature of both of the DPI versus nebulised tobramycin trials. Whilst it would have much preferred trials that were designed for equivalence and had continued for at least the 12 months specified in the current EMA research guidelines for such agents because of the importance of the clinical outputs for use in the cost-effectiveness analysis, it accepted that the evidence presented in terms of FEV1% was the best available and it had to make its judgements accordingly. Additionally the Committee noted that the results of COLO/DPI/02/06 trial indicated it had failed its primary non-inferiority end point in some analyses, but that it had a stricter definition of non-inferiority than the EAGER trial, which also had a primary end point of 20 not 24 weeks. The Committee concluded that the COLO/DPI/02/06 and EAGER trials may have demonstrated that colistimethate sodium DPI and tobramycin DPI were non-inferior to nebulised tobramycin with respect to change in FEV1% within the populations tested and in the manner conducted within each trial, but remained concerned with the uncertain clinical relevance of these findings given the short-term nature of these trials.

4.3.10 The Committee discussed microbiological outcomes and the adverse events reported in the 2 key trials. The Committee noted that the COLO/DPI/02/06 and EAGER trials had both reported improvements in mean sputum density of P. aeruginosa with colistimethate sodium DPI and tobramycin DPI compared to nebulised tobramycin (see sections 4.1.6 and 4.1.10). However, the Committee heard from the clinical specialists that microbiological measurements such as sputum density were not generally carried out in clinical practice. The Committee also observed that both the COLO/DPI/02/06 and EAGER trials reported a higher incidence of adverse events, particularly cough, with both dry powder formulations (see sections 4.1.6 and 4.1.12). Additionally the Committee noted there were higher levels of withdrawal because of adverse events in the dry powder formulation groups of both trials. The Committee was uncertain as to the implications of the adverse event results and it was unsure whether the adverse events associated with either tobramycin DPI or colistimethate sodium DPI were significantly different from those associated with nebulised tobramycin.

4.3.11 The Committee considered which outcomes were the most clinically relevant for assessing the effectiveness of a drug for chronic P. aeruginosa lung infection in people with cystic fibrosis. The Committee noted the opinion of clinical specialists that treatment decisions are often informed by the incidence of exacerbations and the quality of life of the person with cystic fibrosis (see section 4.3.2) rather than short-term changes in FEV1%. The Committee understood that the COLO/DPI/02/06 trial had not identified any statistically significant improvement in health-related quality of life with colistimethate sodium DPI compared with nebulised tobramycin (see section 4.1.7). It noted that no health-related quality-of-life data had been collected in the EAGER trial. The Committee heard from clinical specialists that the number of exacerbations was routinely measured in clinical practice to determine whether a particular treatment was working. The Committee noted that the Assessment Group estimated the probability of people experiencing any exacerbation was slightly lower for colistimethate sodium DPI than for nebulised tobramycin in the COLO/DPI/02/06 trial, but that the rate of protocol-defined exacerbations was higher in the colistimethate sodium DPI group than in the nebulised tobramycin group. It also noted the Assessment Group's comments that exacerbation data were not specifically defined and collected in the EAGER trial and that the term 'lung disorder' might be a reasonable proxy for exacerbation of chronic infection. The Committee understood that the Assessment Group had estimated that the probability of any exacerbation was higher with tobramycin DPI than with nebulised tobramycin. The Committee also heard from the patient experts that avoiding exacerbations was considered the most important outcome for people with cystic fibrosis as it meant feeling better and having a better quality of life. The Committee was aware from patient experts that exacerbations can lead to periods of hospitalisation and the need for intravenous drugs which can have a potentially negative impact on quality of life. The Committee acknowledged the importance of exacerbations to patients and the NHS and the fact that the trial evidence for exacerbations was difficult to interpret because of the way it had been reported and the fact it was over such a short time frame. The Committee concluded that the differences observed in the trials may be small and over a short time horizon but they were the only evidence that is available. The Committee concluded that it was uncertain as to how it should interpret the exacerbation results. Because there were limited data on quality of life and uncertainty around the evidence on exacerbations, the Committee could not draw definitive conclusions as to whether either dry powder offered any clinical benefit over nebulised tobramycin for clinically relevant outcomes.

4.3.12 The Committee discussed the results of the network meta-analysis by the manufacturer of tobramycin DPI (see section 4.1.15). The Committee noted that both the manufacturer and the Assessment Group had acknowledged the limitations of this analysis because of lack of available trials and comparable populations and outcome datasets. The Committee noted that the network meta-analysis had not included colistimethate sodium DPI, but accepted the manufacturer's explanation that this was because trials of colistimethate sodium DPI were not available as none had been published at the time of the submission. The Committee also heard from the manufacturer of colistimethate sodium DPI that it was not aware of any trials other than the small safety trial COLO/DPI/02/05, which compared colistimethate sodium DPI and nebulised colistimethate sodium. The Committee also noted that the manufacturer of tobramycin DPI considered that the results of the analysis comparing tobramycin DPI with other alternative formulations of nebulised tobramycin (such as Bramitob) should be treated with caution because of the underlying differences in study populations. The Committee agreed that because of the uncertainties surrounding the network meta-analysis it would not consider the results further. The Committee again concluded that it could only draw conclusions based on the evidence described in the 2 key trials and that it had no clinical evidence comparing colistimethate sodium DPI with the appropriate comparator, nebulised colistimethate sodium.

4.3.13 The Committee discussed the additional benefits of the mode of delivery of the dry powder formulations over nebulised alternatives. The Committee noted that both technologies aimed to give people with cystic fibrosis and chronic P. aeruginosa lung infection quality-of-life benefits in terms of ease of use and convenience. The Committee considered that it may be plausible that the dry powder formulations could be expected to result in clinical benefits over nebulised solutions because they may increase levels of adherence to treatment. The Committee noted that in the COLO/DPI/02/06 trial levels of adherence to treatment were greater for nebulised tobramycin and it was unclear how adherence was measured in the EAGER trial. However the Committee heard from the patient experts and clinical specialists that people with cystic fibrosis are a heterogeneous population and therefore levels of adherence vary according to individual factors. The Committee heard from clinical specialists that levels of adherence to treatment would be different in clinical practice to those observed under trial conditions. Therefore the Committee agreed that it should not infer too much from these results when evaluating the likely benefit of the technologies in terms of improving adherence to treatment. The Committee acknowledged that both nebulised colistimethate sodium and nebulised tobramycin were embedded as treatment options in current clinical practice and thus judged to be clinically effective in treating chronic P. aeruginosa lung infection. The Committee therefore accepted that a change in the mode of delivery of these drugs would be unlikely to adversely affect their clinical effectiveness compared with nebulised formulations of the drugs, although it only had evidence for the comparison of tobramycin DPI with nebulised tobramycin. The Committee again recognised the limitations of the trial evidence in that this was restricted to a comparison of either DPI with nebulised tobramycin. Despite the limitations in the identified evidence and uncertainties in the interpretation of the trial outcomes, the Committee concluded that on balance it could see an additional benefit for people with cystic fibrosis and chronic pseudomonas lung infection of having the choice of a dry powder formulation of an anti-pseudomonal drug as well as its appropriate nebulised comparator.

Cost effectiveness

4.3.14 The Committee discussed the available cost-effectiveness evidence for colistimethate sodium DPI and tobramycin DPI. It noted that there was no cost-effectiveness evidence comparing colistimethate sodium DPI with nebulised colistimethate sodium. The Committee discussed the model provided by the manufacturer of colistimethate sodium DPI which compared colistimethate sodium DPI with nebulised tobramycin and the critique of the model by the Assessment Group. The Committee noted that the Assessment Group had identified key limitations in the submitted economic model. The Committee had particular concerns about the inconsistent time horizons used in the model for costs and health outcomes and the validity of using mortality benefits associated with 24 weeks of treatment and extrapolating over a lifetime. The Committee noted that the clinical specialists felt that relatively small changes in FEV1% would not be seen as a predictor of 1- or 2-year mortality risk in clinical practice. The Committee also noted that the data used in the regression equations for estimating mortality risk were based on a 20-year-old paper, and the prognosis for people with cystic fibrosis had substantially improved during that time. Additionally, the Committee had concerns about biases in the modelling of exacerbations, which were a major driver of cost savings in the model. The Committee did not accept the validity of the assumption that all patients would have a fixed life expectancy of 37.4 years. The Committee heard from the clinical specialists that it was not unusual to see people aged between 40 and 60 years with cystic fibrosis. The Committee were also concerned about the lack of any preference-based instrument to measure health-related quality of life and the limitation of the method used to map CFQ-R to EQ-5D. Given these uncertainties and others raised by the Assessment Group, the Committee concluded that the manufacturer's (Forest's) model lacked credibility and therefore they would not consider it or its results plausible.

4.3.15 The Committee discussed the Assessment Group's de novo model which compared colistimethate sodium DPI with nebulised tobramycin and tobramycin DPI with nebulised tobramycin. The Committee noted that the model had a lifetime time horizon. The Committee agreed that the use of a lifetime horizon was appropriate, but acknowledged the limitation of extrapolating short-term trial results over long time horizons. Additionally the Committee noted other limitations of the Assessment Group model, including the fact that it did not recognise that, in the current treatment pathway, some people would move from one drug to another (for example from colistimethate sodium to tobramycin). The Committee also noted that treatment duration is assumed to be equivalent between the 2 treatments. The Committee agreed that it was also plausible that some people on nebulised tobramycin would receive some form of treatment on a continuous basis (either as continuous nebulised reduced-dose tobramycin or as nebulised colistimethate sodium in off-months from tobramycin) but there was no evidence on which to base any cost-effectiveness estimate. However, the Committee also accepted that other assumptions in the model such as no difference in the rate of adverse events between colistimethate sodium DPI and nebulised tobramycin may bias in terms of favouring colistimethate sodium DPI. It was also aware that no disutility associated with adverse events was included in the model and was aware of the increased rates of cough in the DPI arms of the COLO/DPI/02/06 and EAGER trials and that this could be a factor affecting adherence. The Committee acknowledged comments received during consultation that there are inherent difficulties in quantifying the relationship between FEV1 and quality of life. The Committee concluded, however, that despite these limitations the Assessment Group's de novo model was the best available framework for assessing the cost effectiveness of colistimethate sodium DPI compared with nebulised tobramycin and of tobramycin DPI compared with nebulised tobramycin.

4.3.16 The Committee discussed the results from the Assessment Group's de novo model comparing colistimethate sodium with nebulised tobramycin, again noting this was not the appropriate comparator for colistimethate sodium DPI. The Committee noted the approved patient access scheme for colistimethate sodium DPI and based its decisions on the cost-effectiveness results from analyses incorporating the revised patient access scheme price. The Committee considered the sensitivity analysis using the E-MIT price of nebulised tobramycin and recognised that analyses using the list price for nebulised tobramycin may overestimate the cost effectiveness of colistimethate sodium DPI in terms of NHS practice. The Committee discussed the implications of considering a treatment that was less effective and less costly, and the need therefore to have considerable confidence in the clinical-effectiveness results. The Committee concluded that there was a great deal of uncertainty in the modelling of colistimethate sodium DPI and that the most plausible ICERs indicated that with the revised patient access scheme colistimethate sodium DPI resulted in a small QALY loss and a cost saving compared with nebulised tobramycin.

4.3.17 The Committee discussed all the uncertainties associated with the clinical- and cost-effectiveness evidence for colistimethate sodium DPI. It acknowledged the difficulties in performing high-quality research in a disease with a heterogenous clinical profile and where treatment options are ultimately based on clinician and patient preferences. However, the Committee was aware of the statements from clinical specialists about current treatment for people with cystic fibrosis and chronic P. aeruginosa lung infection and the treatment pathway of switching from one antibiotic to another. The Committee again confirmed that the most appropriate comparator for colistimethate sodium DPI was nebulised colistimethate sodium and that it would have wished to see the results of such a comparison. The Committee noted again that the manufacturer of colistimethate sodium DPI had confirmed that there was no trial that compared the effectiveness of colistimethate sodium DPI with nebulised colistimethate sodium. The Committee was therefore unable to reach a conclusion on the clinical and cost effectiveness of colistimethate sodium DPI compared with nebulised colistimethate sodium.

4.3.18 The Committee discussed the cost-effectiveness evidence for tobramycin DPI. It noted that there was no economic analysis in the manufacturer's submission for tobramycin DPI and that the manufacturer felt that a cost-minimisation approach was appropriate. The Committee did not agree that cost minimisation was appropriate in this instance because the evidence had not proved equivalence between tobramycin DPI and nebulised tobramycin and that cost minimisation would not incorporate any estimate of uncertainty; additionally the Committee was aware that the NICE reference case stated a preference for cost–utility analysis. The Committee therefore based its discussions on the cost-effectiveness analysis carried out by the Assessment Group incorporating the patient access scheme. The Committee noted that the results of this analysis had shown that tobramycin DPI consistently dominated nebulised tobramycin when the patient access scheme was included (incremental cost of –£42,500 and incremental QALY gain of 0.34 in the base case). The Committee also noted the results of the probabilistic sensitivity analysis and one-way sensitivity analysis which indicated that tobramycin DPI consistently dominated nebulised tobramycin. Additionally, the Committee was aware that the analyses were based on the inclusion of aggregate lung disorder data as a proxy measure for exacerbations because data on major and minor exacerbations had not been collected in the EAGER trial. The Committee recognised that the model assumed that the intervention and comparator are used for 28 days followed by 28 days without use of any drug and that this may not be reflective of clinical practice. The Committee accepted that there was no clinical evidence for such an approach and neither the clinical specialists nor the manufacturers knew of any. The Committee acknowledged that the small QALY gain for tobramycin DPI was uncertain because quality-of-life data were not collected in the EAGER trial. However, it agreed it was reasonable to assume that there would be some QALY gain for tobramycin DPI over nebulised tobramycin in clinical practice in view of the reported benefits to patients in terms of ease of use and convenience although it acknowledged that the number of withdrawals from the EAGER trial did not indicate this relationship. The Committee therefore agreed that despite the limitations of all the clinical- and cost-effectiveness evidence and hence the uncertainty inherent in the Assessment Group model, it was reasonable to conclude that tobramycin DPI was a cost-effective use of NHS resources. Therefore the Committee concluded that it could recommend tobramycin DPI (with the associated patient access scheme applied) as a treatment option for chronic P. aeruginosa lung infection in people with cystic fibrosis who would otherwise have been treated with nebulised tobramycin; that is, when colistimethate sodium is contraindicated, is not tolerated, or has not produced an adequate clinical response. This is provided that the patient access scheme is operational in primary, secondary and tertiary settings of care.

4.3.19 The Committee considered whether there were any groups of people for whom the COLO/DPI/02/06 trial (in which the comparator for colistimethate sodium DPI was nebulised tobramycin) provided clinical- or cost-effectiveness evidence for the use of colistimethate sodium. It recognised that the majority of people currently switching to nebulised tobramycin from nebulised colistimethate sodium do so either because of a lack of efficacy of nebulised colistimethate sodium or because colistimethate sodium is contraindicated. The Committee recognised that a potential switch from nebulised colistimethate sodium to colistimethate sodium DPI in these two groups would therefore be inappropriate. However, the Committee considered that there was a group of people who could benefit or were benefiting from nebulised colistimethate sodium but were unable to tolerate it twice daily in its nebulised form. The current treatment option for these people would be nebulised tobramycin and thus the COLO/DPI/02/06 trial did give evidence for the use of colistimethate sodium DPI rather than nebulised tobramycin in such a group of people. The Committee noted the small QALY loss for colistimethate sodium DPI compared with nebulised tobramycin but also the substantial cost saving (£38,000 with the list price for nebulised tobramycin). The Committee further observed that adherence might be greater with the use of a dry powder inhaler in such a population. The Committee also acknowledged comments received during consultation that some people would be sensitive to nebulised aminoglycosides (such as tobramycin) in terms of side effects or be otherwise intolerant of such therapy. However the Committee agreed that its understanding of the treatment pathway was that this group receive colistimethate sodium as first-line treatment, tobramycin as second-line treatment and then an alternative treatment to tobramycin if tobramycin treatment fails or is otherwise contraindicated. The Committee therefore decided that colistimethate sodium DPI did not have a role to play in the treatment of chronic P. aeruginosa in people who were sensitive (in terms of toxicity) to tobramycin or intolerant of tobramycin. It therefore concluded that it could only recommend colistimethate sodium DPI as a treatment option for people who would clinically benefit from continued colistimethate sodium but cannot tolerate it in its nebulised form. This is provided that the patient access scheme is operational in primary, secondary and tertiary settings of care.

4.3.20 The Committee discussed whether NICE's duties under the equalities legislation required it to alter or add to its preliminary recommendations in any way. The Committee noted a potential equalities issue in that cystic fibrosis mostly affects people of white European origin; however, the Committee recognised that this reflected the epidemiology of cystic fibrosis rather than being an equalities concern. The Committee was also aware that some patients or carers may have difficulty manipulating an inhaler for dry powder inhalation, but noted that the same may apply to other modes of treatment delivery. The Committee concluded that its recommendations would not affect any of the groups whose interests are protected by the legislation and that there was no need to alter or add to its recommendations.

Summary of Appraisal Committee's key conclusions

TA276

Appraisal title: Colistimethate sodium and tobramycin dry powders for inhalation for treating pseudomonas lung infection in cystic fibrosis

Section

Key conclusions

The Committee concluded that it could only draw conclusions based on the evidence described in the 2 key trials and that it had no clinical evidence comparing colistimethate sodium DPI with the appropriate comparator, nebulised colistimethate sodium.

4.3.12

The Committee agreed that despite the limitations of all the clinical- and cost-effectiveness evidence and hence the uncertainty inherent in the Assessment Group model, it was reasonable to conclude that tobramycin DPI was a cost-effective use of NHS resources. The Committee recommended tobramycin DPI as a treatment option only when nebulised tobramycin is considered an appropriate treatment; that is, when colistimethate sodium is contraindicated, is not tolerated, or has not produced an adequate clinical response. This is provided that the patient access scheme is operational in primary, secondary and tertiary settings of care.

4.3.18

Given the available evidence and the current treatment pathway, the Committee could only recommend colistimethate sodium DPI as a treatment option for people who would clinically benefit from continued colistimethate sodium but cannot tolerate it in its nebulised form. This is provided that the patient access scheme is operational in primary, secondary and tertiary settings of care.

4.3.19

Current practice

Clinical need of patients, including the availability of alternative treatments

The most important outcomes for the patient that influence treatment decisions are the person's quality of life, treatment burden, maintaining good lung function and reducing the incidence of exacerbations.

4.3.2

Current treatment options include the use of inhaled antibiotics effective against P. aeruginosa (such as nebulised colistimethate sodium or tobramycin) and oral or intravenous antibiotics to eradicate initial or intermittent P. aeruginosa colonisation or acute exacerbations of chronic infection. Azithromycin may be given in combination with these antibiotics to act on the biofilms.

2.5

First-line treatment for chronic P. aeruginosa lung infection routinely starts with nebulised colistimethate sodium (unless it is contraindicated), this choice being largely based on cost. If there is no response, an unacceptable adverse event profile, an excessive number of acute exacerbations or a loss of lung function, then treatment is switched routinely to nebulised tobramycin.

4.3.2

The technology

Proposed benefits of the technology

How innovative is the technology in its potential to make a significant and substantial impact on health-related benefits?

The Committee recognised that the strict routine and amount of time spent receiving treatment have a significant impact on the daily activities of people with cystic fibrosis and their families. The Committee concluded that reducing the time that people with cystic fibrosis spend receiving treatment would be beneficial in improving the quality of life of people with cystic fibrosis and their families.

4.3.5

The Committee concluded that in clinical practice (rather than in clinical trials), people with cystic fibrosis may be more likely to adhere to a dry powder for inhalation treatment than a nebulised treatment in view of the speed and convenience of drug delivery.

4.3.6

What is the position of the treatment in the pathway of care for the condition?

The Committee heard from the manufacturer of colistimethate sodium DPI that the EMA had indicated that the most appropriate comparator at the time of the study design for its pivotal trial would be nebulised tobramycin because this was the only licensed comparator in all of the study site countries. The Committee agreed that given the current clinical pathway, ideally it would have liked to have seen effectiveness evidence comparing colistimethate sodium DPI with nebulised colistimethate sodium and also whether there was evidence of any clinical benefit of colistimethate sodium DPI in people being switched from nebulised colistimethate sodium because of lack of efficacy. Given the current treatment pathway in the UK, the Committee concluded that the most appropriate comparator for colistimethate sodium DPI would be nebulised colistimethate sodium and the most appropriate comparator for tobramycin DPI would be nebulised tobramycin.

4.3.4

Adverse reactions

The Committee was unsure whether the adverse events associated with either tobramycin DPI or colistimethate sodium DPI were significantly different from those associated with nebulised tobramycin.

4.3.10

Evidence for clinical effectiveness

Availability, nature and quality of evidence

The Committee discussed the quality of the 2 key trials that compared colistimethate sodium DPI with nebulised tobramycin and tobramycin DPI with nebulised tobramycin. It noted the Assessment Group's critique of the trials, in particular the fact that the manufacturers had not commented in their submissions on the quality of the trials in light of the current EMA research guidelines for the development of medicinal products for the treatment of cystic fibrosis. The Committee supported the Assessment Group's comments on the methodological limitations of both trials, such as a lack of blinding, and agreed that this could have introduced selection and reporting bias for subjective outcomes such as adverse events and might limit the generalisability of the findings. The Committee concluded that the evidence base for assessing the clinical effectiveness of colistimethate sodium and tobramycin DPIs was of, at best, modest quality but that it was the best available.

4.3.8

The Committee noted that there was no clinical trial comparing the effectiveness of colistimethate sodium DPI with that of the preferred comparator, nebulised colistimethate sodium.

4.3.7

The Committee noted that both trials were non-inferiority in design and therefore only assessed whether the interventions were not worse than nebulised tobramycin.

4.3.7

Relevance to general clinical practice in the NHS

The Committee acknowledged that both nebulised colistimethate sodium and nebulised tobramycin were embedded as treatment options in current clinical practice and thus judged to be clinically effective in treating chronic P. aeruginosa lung infection. The Committee therefore accepted that a change in the mode of delivery of these drugs would be unlikely to adversely affect their clinical effectiveness compared with nebulised formulations of the drugs.

4.3.13

Uncertainties generated by the evidence

There is uncertainty about the clinical relevance of the findings of the 2 key trials given the short-term nature of these trials.

4.3.9

The Committee concluded that it was uncertain as to how it should interpret the exacerbation results. Because there were limited data on quality of life and uncertainty around the evidence for exacerbations, the Committee could not draw definitive conclusions as to whether either dry powder offered any clinical benefit over nebulised tobramycin for clinically relevant outcomes.

4.3.11

Whilst the Committee would have much preferred trials that were designed for equivalence and had continued for at least the 12 months specified in the current EMA research guidelines for such agents, it accepted that the evidence presented in terms of FEV1% was the best available and it had to make its judgements accordingly.

4.3.9

Are there any clinically relevant subgroups for which there is evidence of differential effectiveness?

Not applicable

Estimate of the size of the clinical effectiveness including strength of supporting evidence

For the primary end point of mean difference in change in FEV1% predicted after 24 weeks of treatment, the result from the ANCOVA analysis on the ITT population using LOCF data was −1.16% (95% CI −3.15% to 0.84%) suggesting that colistimethate sodium DPI was marginally less efficacious than tobramycin nebuliser solution (because the non-inferiority criterion was not met). The results of the log-transformed and non-parametric ITT population LOCF data analyses were −0.98% (95% CI −2.74% to 0.86%) and −0.56% (95% CI −2.16% to 1.00%) respectively, suggesting in both cases that colistimethate sodium DPI was non-inferior to nebulised tobramycin. For the PP population the ANCOVA, log-transformed and non-parametric analyses using LOCF data indicated that the non-inferiority hypothesis was satisfied for non-parametric analysis only (ANCOVA −1.49% [95% CI −3.79% to 0.81%], log-transformed −1.10% [95% CI
−3.08% to 0.97%] and non-parametric −0.67% [95% CI −2.57% to 1.16%]).

4.1.4

Tobramycin DPI was associated with an improved mean FEV1% predicted compared with nebulised tobramycin at 20 weeks of +0.59% (SE 0.92). The manufacturer reported non-inferiority (supported by least squares mean difference relative change of 1.1% [SE 1.75] which has a lower limit of the one-sided 85% confidence interval within the predicted 6% margin for non-inferiority).

4.1.9

The Committee concluded that it only had evidence exploring whether either dry powder formulation was no worse than nebulised tobramycin and no evidence to prove that either was more effective than or equivalent to nebulised tobramycin.

4.3.7

The Committee concluded that the COLO/DPI/02/06 and EAGER trials may have demonstrated that colistimethate sodium DPI and tobramycin DPI were non-inferior to nebulised tobramycin with respect to change in FEV1% within the populations tested and in the manner conducted within each trial, but remained concerned with the uncertain clinical relevance of these findings given the short-term nature of these trials.

4.3.9

Evidence for cost effectiveness

Availability and nature of evidence

The manufacturer of colistimethate sodium DPI used a cohort-based decision analysis to compare colistimethate sodium DPI with nebulised tobramycin in people aged 6 years or older with documented cystic fibrosis who had chronic pseudomonas lung infection.

4.2.3

The Committee concluded that the manufacturer's (Forest's) model lacked credibility and therefore they would not consider it or its results plausible.

4.3.14

The Committee discussed the Assessment Group's de novo model which compared colistimethate sodium DPI with nebulised tobramycin and tobramycin DPI with nebulised tobramycin. The Committee noted that the model had a lifetime time horizon. The Committee agreed that the use of a lifetime horizon was appropriate, but acknowledged the limitation of extrapolating short-term trial results over long time horizons. Additionally the Committee noted other limitations of the Assessment Group model, including the fact that it did not recognise that, in the current treatment pathway, some people would move from one drug to another (for example from colistimethate sodium to tobramycin).

4.3.15

The Committee concluded, however, that despite these limitations the Assessment Group's de novo model was the best available framework for assessing the cost effectiveness of colistimethate sodium DPI compared with nebulised tobramycin and of tobramycin DPI compared with nebulised tobramycin.

4.3.15

The Assessment Group also carried out additional analyses in response to the patient access schemes submitted by the manufacturers of colistimethate sodium DPI and tobramycin DPI.

4.2.15–4.2.25

Uncertainties around and plausibility of assumptions and inputs in the economic model

Colistimethate sodium DPI: The Committee had particular concerns about the inconsistent time horizons used in the Forest model for costs and health outcomes and the validity of using mortality benefits associated with 24 weeks of treatment and extrapolating over a lifetime.

4.3.14

The Committee had particular concerns about limitations of the Assessment Group's model. The Committee agreed that the use of a lifetime horizon was appropriate, but acknowledged the limitation of extrapolating short-term trial results over long time horizons. Additionally it noted that the model did not recognise that, in the current treatment pathway, some people would move from one drug to another (for example from colistimethate sodium to tobramycin). The Committee also noted that treatment duration is assumed to be equivalent between the 2 treatments. It agreed that it was also plausible that some people on nebulised tobramycin would receive some form of treatment on a continuous basis (either as continuous nebulised reduced-dose tobramycin or as nebulised colistimethate sodium in off-months from tobramycin) but there was no evidence on which to base any cost-effectiveness estimate.

4.3.15

Tobramycin DPI: The Committee was aware that for the Assessment Group model for tobramycin DPI uncertainty was generated because the analyses were based on the inclusion of aggregate lung disorder data as a proxy measure for exacerbations because data on major and minor exacerbations had not been collected in the EAGER trial.

4.3.18

The QALY gain for tobramycin DPI was uncertain because quality-of-life data were not collected in the EAGER trial.

4.3.18

Incorporation of health-related quality-of-life benefits and utility values

Have any potential significant and substantial health-related benefits been identified that were not included in the economic model, and how have they been considered?

The Committee discussed the additional benefits of the mode of delivery of the dry powder formulations over nebulised alternatives. The Committee noted that both technologies aimed to give people with cystic fibrosis and chronic P. aeruginosa lung infection quality-of-life benefits in terms of ease of use and convenience.

4.3.13

The Committee acknowledged that the small QALY gain for tobramycin DPI was uncertain because quality-of-life data were not collected in the EAGER trial. However, it agreed it was reasonable to assume that there would be some QALY gain for tobramycin DPI over nebulised tobramycin in clinical practice in view of the reported benefits to patients in terms of ease of use and convenience although it acknowledged that the number of withdrawals from the EAGER trial did not indicate this relationship.

4.3.18

The Committee noted the small QALY loss for colistimethate sodium DPI compared with nebulised tobramycin but also the substantial saving (£38,000 with the list price for nebulised tobramycin). The Committee further observed that adherence might be greater with the use of a dry powder inhaler in such a population.

4.3.19

Are there specific groups of people for whom the technology is particularly cost effective?

Not applicable

What are the key drivers of cost effectiveness?

The key drivers of cost effectiveness were the cost of interventions and comparators when the most recent patient access schemes were included, and the small QALY gains for tobramycin DPI compared with nebulised tobramycin and for colistimethate sodium compared with nebulised tobramycin.

4.2.24, 4.2.25

Most likely cost-effectiveness estimate (given as an ICER)

Tobramycin DPI consistently dominated nebulised tobramycin with inclusion of the patient access scheme, that is, there was a cost saving and QALY gain for tobramycin DPI compared to nebulised tobramycin.

4.3.18

The Committee noted the small QALY loss for colistimethate sodium DPI compared with nebulised tobramycin but also the substantial cost saving (£38,000 with the list price for nebulised tobramycin).

4.3.19

Additional factors taken into account

Patient access schemes (PPRS)

A patient access scheme has been agreed with the Department of Health for colistimethate sodium DPI, details of which are confidential.

3.3

A patient access scheme has been agreed with the Department of Health for tobramycin DPI, details of which are confidential.

3.6

The Committee noted the approved patient access scheme for colistimethate sodium DPI and based its decisions on the cost-effectiveness results from analyses involving the patient access scheme price.

4.3.16

The Committee based its discussions on the cost-effectiveness analysis of tobramycin DPI carried out by the Assessment Group incorporating the patient access scheme.

4.3.18

End-of-life considerations

Not applicable

Equalities considerations and social value judgements

The Committee discussed NICE's duties under the equalities legislation and concluded that its recommendations would not affect any of the groups whose interests are protected by the legislation and that there was no need to alter or add to its recommendations.

4.3.20

  • National Institute for Health and Care Excellence (NICE)