3 Committee discussion
The appraisal committee (section 4) considered evidence submitted by Dompé and a review of this submission by the evidence review group (ERG). See the committee papers for full details of the evidence.
3.1 Neurotrophic keratitis is a rare degenerative condition which is classed as an orphan disease. The clinical experts noted that it is a difficult condition to treat: there is no standard care pathway and many treatments are ineffective in healing the cornea. They noted that when the disease progresses, there is high risk of permanent vision loss as a result of fibrotic scars. The experts explained that disfigurement caused by corneal scarring and surgical procedures such as tarsorrhaphy (medical or surgical eyelid closure) can significantly impair quality of life. The committee agreed that people with neurotrophic keratitis have an unmet clinical need and would welcome any new treatment that improves outcome and reduces the need for surgery.
3.2 The underlying cause of neurotrophic keratitis varies, and clinicians are faced with a heterogeneous patient population that is difficult to treat. The clinical experts highlighted that there is no standard care pathway and choice of treatment depends on the severity of the disease, clinician preference, patient need and availability. Current treatments are palliative in nature, such as preservative-free artificial tears and prophylactic antibiotics, and they are routinely used along with other treatments. Because there is no standard treatment that works for all patients, more than 1 treatment is normally prescribed in clinical practice. The committee noted that autologous serum-derived eye drops may take 6 to 8 weeks to prepare, and they are not available in many centres in England. The clinical experts stated that although tarsorrhaphy is not a popular treatment option with patients (because it can cause disfigurement), it is an effective and inexpensive procedure that is often used when all other options have failed. The clinical experts also highlighted that in some centres surgical tarsorrhaphy has been replaced by botulinum toxin-induced protective ptosis. The committee understood that cenegermin would be used early in the treatment pathway but that topical treatments may still be used concomitantly in clinical practice. It concluded that there is no standard care for neurotrophic keratitis but that cenegermin would be used as a potential early option.
3.3 The company's clinical evidence came from 2 trials, REPARO and study 0214, which were unpublished at the time of the company's submission. Both trials are double-blinded, randomised, multicentre, vehicle-controlled, parallel group studies comparing cenegermin with vehicle. Vehicle acts a proxy for artificial tears, which is the only comparator identified by the company (owing to lack of clinical evidence for other comparators listed in the final NICE scope). Treatment lasted for 8 weeks, in line with the marketing authorisation. The clinical experts highlighted that the vehicle treatment could not be considered a placebo because it will have some therapeutic benefit. The committee heard that REPARO was done in Europe and included people with unilateral stage 2 or 3 neurotrophic keratitis, refractory to 1 or more previous conventional non-surgical treatments (n=156). Study 0214 was done in the US; it included a similar population to REPARO, but also included people with bilateral disease (although only the worse-affected eye was studied; n=48). The committee noted that the trials had low patient numbers but this was to be expected considering the rarity of the disease. However, in both trials the controlled follow-up period was short (8 weeks), there was little long-term follow-up and withdrawal rates were high (up to 37.5%). Further uncertainty came from only a small proportion of patients having the licensed methionine-containing formulation of cenegermin (34 patients across both studies). The committee concluded that the results of both trials were associated with significant uncertainty.
3.4 Corneal healing (defined as having less than 0.5 mm epithelial defect) at week 8 was a co-primary end point of study 0214 and a secondary end point of REPARO. The difference in the percentage of patients achieving corneal healing between the cenegermin and vehicle arms at 8 weeks was 30.9% (97.06% confidence interval [CI] 10.60 to 51.13; p=0.002) in REPARO and 40.4% (95% CI 14.2 to 66.6; p=0.006) in study 0214. Similar results were also seen using a stricter definition of corneal healing (that is, no residual fluorescein staining in the area of the corneal lesion [0 mm] and no persistent staining elsewhere in the cornea), which was a co-primary end point in study 0214 and a post-hoc analysis in REPARO. The primary analysis approach of both studies at 8 weeks was determined by a central reading centre and last observation carried forward (LOCF) methodology was used to account for missing data. The ERG raised concerns about the use of the LOCF method because of the associated biases; it stated that the multiple imputation approach would have been a more appropriate method to account for missing data in the trials. However, the committee noted that the different approaches did not lead to different conclusions about the comparative effectiveness of cenegermin and vehicle. The committee also noted that, in both studies, there was no statistically significant difference between the treatment arms at week 8 in secondary outcomes (specifically, percentage of patients achieving complete corneal clearing, time to onset of disease deterioration, and change from baseline in mean best corrected distance visual acuity score). The committee concluded that cenegermin is more clinically effective than vehicle in terms of corneal healing at 8 weeks.
3.5 The committee noted that tarsorrhaphy was an exclusion criteria for both REPARO and study 0214, but heard from the clinical experts that it is frequently used in current management of neurotrophic keratitis. The experts noted that tarsorrhaphy is unpopular with patients and ideally cenegermin would be given earlier in the treatment pathway. The ERG also noted that although differences in the baseline characteristics exist between the populations in the 2 trials, the patient populations are generalisable to NHS clinical practice. The committee concluded that the REPARO and study 0214 trial populations are generalisable to clinical practice in England.
3.6 The clinical experts explained that there is no robust clinical evidence to suggest that cenegermin could effectively 'cure' neurotrophic keratitis or prevent recurrence. The committee recalled that people who were healed at week 8 but no longer healed at 32 or 56 weeks of the extended follow-ups of REPARO and study 0214 were considered to have had a recurrence of neurotrophic keratitis. Recurrence rates at 32 weeks in the 2 trials varied from 0% to 3% in REPARO and 0% to 14% in study 0214, depending on the arm to which people were originally randomised. At 56 weeks, recurrence rates were 3% to 5% in REPARO. The committee noted that these analyses were exploratory and based on a small number of patients for whom response data were available, so no firm conclusions could be drawn. The committee concluded that the recurrence rate and need for further treatment with cenegermin after 8 weeks is uncertain because of a lack of long-term data.
Both the original and revised models contained errors and produce uncertain estimates of cost effectiveness
3.7 The committee considered the company's original model to be structurally flawed and its results not robust. After consultation, the company submitted a revised model structure. This model allowed people having treatments from the standard care basket to move from the non-healing to healed states, which was not possible in the original model. The probability of healing or not healing after standard care was taken from a survey of 12 clinical experts, the responses to which varied widely in their estimates of effectiveness. In the survey, the clinical experts provided estimates for: artificial tears, contact lenses, tarsorrhaphy, autologous serum eye drops, amniotic membrane transplantation, conjunctival flap and corneal transplant. The ERG acknowledged the improved model structure but questioned these estimates, because they were generally considered to be the same as or better than healing rates after cenegermin. The ERG noted that if treatments in the standard care basket were more effective than cenegermin, cenegermin would not be a cost-effective treatment. Alternatively, the survey estimates of corneal healing may not be accurate, which would produce unreliable cost-effectiveness results. The ERG also considered that the treatment effects were incorrectly implemented in the revised model; people may have several treatments, so the probability of complete healing could be more than 1. The company acknowledged the uncertainty but explained that the survey data were the only data available that could be used in the revised model. The committee also noted anomalies in the cost-effectiveness results using the revised model when varying the effectiveness of treatments in the standard care basket between 0% and 100%. It questioned the reliability of the model results and whether alternative methodologies could have been used instead of a naïve indirect comparison. The committee acknowledged the challenges of modelling a complex disease area with no established treatment pathway and minimal clinical evidence. Nevertheless, it had concerns about the robustness and reliability of the cost-effectiveness estimates from both the company's original and revised models, which would need to be accounted for in its decision-making.
3.8 The committee recalled that, as a result of the structure of the company's model, people in the sustained healing state do not experience a recurrence of disease after 5 years and will remain healed until death. Clinical advice received by the company suggested that the recurrence rate reduces with time and tends to plateau after 5 years. The committee recalled the clinical expert statement and agreed that there is no clinical evidence to support the assumption that people who are completely healed at 5 years will remain healed for the rest of their lifetime, and that their disease is effectively cured. The committee concluded that it is inappropriate to extrapolate the treatment effect of cenegermin over a lifetime.
3.9 In its response to consultation, the company reported analyses assuming that surgery is only done in the first year and people in the non-healing state had 2 visits to a specialist per month. The clinical experts had previously explained that some people with acute disease may need frequent visits to a specialist, but that 2 or 3 visits each year would be more likely once the condition is stabilised (that is, maintenance treatment is reached). The committee considered 2 visits per month to be too high because it equates to 24 specialist visits a year or around 450 over a lifetime; the clinical experts explained that this seemed very unlikely. The committee noted that the costs included in the model were overestimated because artificial tears, autologous serum eye drops and contact lenses were assumed to continue for a lifetime after healing. The ERG explained that in both the original and revised model submitted after consultation, people who move to the healed or sustained healing states after having treatments in the standard care basket were assumed to then have artificial tears for life. The committee understood that in the trial, artificial tears were stopped 8 weeks after healing. To explore this inconsistency, the ERG did an exploratory scenario analysis using what it considered to be more clinically plausible assumptions:
autologous serum eye drops stopped after 1 year in the healed state
artificial tears stopped after 1 year in the sustained healing and healed states
all costs were the same in the healed and non-healed states in the first year.
This scenario substantially increased the cost-effectiveness estimates for cenegermin compared with artificial tears. The ERG also identified 5 errors in the company's revised model that related to the costs of autologous serum eye drops, contact lenses, specialist visits and surgical treatments, and the weighting of costs. The ERG's correction of these errors further increased the cost-effectiveness estimates. The committee concluded that costs and resource use were overestimated in the model.
3.10 The ERG explained that in the company's revised model, the costs and utilities of recurrence were applied incorrectly and should have been modelled differently. This is because people with recurrence between cycles 1 to 13 have different costs and utilities to those with recurrence after cycle 14 (when there are no surgical treatments after 1 year). The ERG ultimately considered the model too simple to accurately estimate the cost effectiveness of cenegermin compared with artificial tears. The committee concluded that the costs and utilities of recurrence were not accurately captured in the model, limiting the robustness of any cost-effectiveness estimates.
3.11 The committee noted that the only difference between the healed and non-healed health states in the company's original model was a disutility applied for tarsorrhaphy. The ERG queried the accuracy of this disutility value, because there was little supportive evidence and it considered that most people would only have the procedure once in their lifetime. In response to consultation, the company applied the disutility for tarsorrhaphy in the first year only. The ERG explained that although this addressed the inaccuracy in how utility values were previously applied, it did not address the underlying uncertainty in the values themselves. Because of this, the quality-adjusted life year (QALY) gain for cenegermin is likely to be overestimated. The committee concluded that there was considerable uncertainty associated with the utility values used in both the company's original and revised models.
The modelled benefits for cenegermin are likely to be overestimated and the cost-effectiveness estimates are uncertain
3.12 The committee recalled the uncertainty in the evidence (see section 3.6) and that neither the company's original nor revised model produced a robust estimate of cost effectiveness compared with artificial tears that reflects clinical practice in England. Based on the evidence presented, the committee concluded that the modelled benefits for cenegermin are likely to have been overestimated. This, plus the unknown longer-term corneal healing effects, meant it was not possible to identify a robust cost-effectiveness estimate for cenegermin compared with artificial tears.
3.13 The company considered cenegermin to be innovative because it is the only treatment that has shown an improvement in outcomes when used to treat neurotrophic keratitis, a severe condition. The committee acknowledged that there is a large unmet need for people with neurotrophic keratitis and that cenegermin would be beneficial for patients, but it had not been presented with evidence of any additional benefits that were not captured in the QALY calculations.
3.14 The committee considered that cenegermin is more clinically effective than vehicle in terms of corneal healing at 8 weeks. However, the longer-term effects are not known because there were no available data. The committee considered that there was substantial uncertainty in the cost-effectiveness evidence and that the estimates of cost-effectiveness were very uncertain. There were errors in costs, implausible assumptions and uncertainty in utility values and therefore the modelled benefits for cenegermin are likely to be overestimated. This, plus the unknown longer-term corneal healing effects, meant it was not possible to identify a robust incremental cost-effectiveness ratio (ICER) for cenegermin compared with artificial tears (exact ICERs cannot be reported here because of a confidential patient access scheme). Although it was not presented with a robust ICER that reflects clinical practice in England, the committee considered that based on the evidence presented the most likely ICER for cenegermin compared with artificial tears would be higher than the range that NICE normally considers to be an acceptable use of NHS resources (that is, £20,000 to £30,000 per QALY gained). Because of this, it concluded that cenegermin cannot be recommended within its marketing authorisation to treat moderate or severe neurotrophic keratitis.