The company's model assumed that people could either remain in the post-op controlled health state, or move to the uncontrolled state if their symptoms returned, at a rate of 42.8% per year. This estimate was derived from Benson et al. (2023), which found that 17% of people needed a third surgery within 2.4 years. The company's clinical experts advised that people typically wait around 2 years for revision surgery, and that symptoms return quickly in 40% to 50% of people. Adjusting for the waiting time and working backward from the study data produced the 42.8% figure. The company's model also assumed people in the uncontrolled health state could move back to the post-op controlled health state. At the first committee meeting, the rate was 15.1% per year (calculated from Benson et al.), which the EAG corrected to 14.8% after identifying a calculation error. This probability reflects the annual chance of revision surgery for everyone in the uncontrolled health state. This includes those on medical treatment, those whose symptoms had returned after surgery, and those ineligible or unwilling to have surgery. The EAG's clinical experts noted that rates of loss of control and revision surgery varied. In a scenario analysis, the EAG reduced the waiting time for surgery from 2 years to 1 year, which reduced the transition probability for moving from the post-op controlled to the uncontrolled health state to 12.1%. This increased the ICER considerably. The clinical experts confirmed that a 2-year waiting time was likely, given the referral time to an ENT specialist (about 12 months), surgery waiting list (at least 12 months) and potential wait for a CT scan. They also said that a rate of 42.8% per year for symptoms to return after surgery was reasonable. SINUS UK provided evidence from a survey of 51 people that showed that about half had their symptoms return within 6 months of surgery. The clinical experts also noted that the polyps regrow within 6 months in about a third of people. The EAG said it had concerns about how the transition probabilities had been calculated for the full loop of this part of the model; that is, from the post-op controlled to uncontrolled health state, and from the uncontrolled to post-op controlled health state after revision surgery. It said it was not clear how or why the data had been chosen to calculate it.
After draft guidance consultation, the company provided further detail on its calculations. It maintained its transition rate of 42.8% for the post-op controlled to uncontrolled health state. But, it updated the annual rate of revision surgery to 7.1% from 14.8% using an updated extrapolation of the mean rate of surgery from Benson et al. The EAG corrected the company's transition rate from the post-op controlled to uncontrolled health state to 37.2% (from 42.8%) because it said the company had incorrectly converted the probability in its calculation to an annual probability. It was unable to verify the company's calculation for the transition probabilities. But it provided an alternative analysis of the Kaplan–Meier data from Benson et al., based on 14.5% having a third surgery 3 years after the second and assuming a 2-year waiting time. The EAG's transition rates were 15% from the post-op controlled to uncontrolled health state and 39.4% from the uncontrolled to post-op controlled health state. The clinical expert said that the company's transition rate of 42.8% for loss of control was in line with evidence from the Netherlands that, in clinical practice, symptoms return within 12 months of surgery in around 40% of people. They said that 14.5% of people having a third surgery 3 years after the second was plausible and reflects clinical practice. The company said that its 7.1% transition rate was for the whole uncontrolled health state, including people not waiting for surgery. It said that the EAG's transition rate was implausible because it assumed that everyone with uncontrolled severe CRS with nasal polyps was on the waiting list for surgery, whereas many would not be. The clinical expert explained that the annual rate of revision surgery is low, but they would not expect it to be as low as 7%. Patient experts said that it was reasonable to assume that not everyone in the uncontrolled health state would go on to have surgery. They explained that some people who had already had 2 surgeries would choose not to have another, and said the company's figures were more realistic. They added that the EAG's estimates did not look plausible because they suggested the probability of having surgery was higher than the probability of symptoms returning.
The committee discussed the calculations used to derive the transition rates from the post-op controlled to uncontrolled health state and from the uncontrolled to post-op controlled health state. It took into account the face validity of the figures, as discussed by clinical and patient experts. The committee considered that both the company's and EAG's estimates for the transition probabilities were extremely uncertain and did not align with clinical expectations, so were not appropriate for decision making. It thought that the company's transition probabilities were more in line with the views of the clinical and patient experts but were not mathematically correct. This was because the company did not use valid methods to estimate event rates. It did not account for the time at risk for people who did not have the event in the denominators of the event rate calculations. In addition, the committee noted that applying the company's assumptions did not result in a figure of 14.5% having a third surgery at 3 years, from Benson et al. The committee thought that the EAG's approach was mathematically correct, and resulted in 14.5% having a third surgery at 3 years. But the transition probabilities it derived were less closely aligned with clinical and patient experts' opinions. The committee thought that the most reliable data presented was the estimate of 14.5% of people having a third surgery 3 years after the second, from Benson et al. This was because it was based on real-world evidence from UK patients. It considered that any calculation of the transition probabilities needed to be based on this figure. The committee was aware of a randomised controlled trial that compared endoscopic sinus surgery plus medical treatment with medical treatment alone in people with CRS with nasal polyps (Lourijsen et al. 2022). This paper reported that 12 months after surgery, 46.0% of people had uncontrolled CRS despite surgery. It also used a minimal clinically important difference of 9 points on the SNOT-22 scale, in line with the committee-accepted response criteria (see section 3.8). The committee considered that it was more appropriate to use the figure of 46.0% from Lourijsen et al. for the transition probability from the post-op controlled to uncontrolled health state. This was because it was evidence based. It also removed the need to assume a 2-year waiting list for surgery, which had previously been required in the EAG and company approaches, and which increased uncertainty because it substantially changed the result depending upon how it was applied. The committee noted that the rate of loss of control in the study was not constant and the rate of 46.0% was likely to be an overestimate, and may favour dupilumab. The committee noted it was possible to calculate a transition probability from the uncontrolled to post-op controlled health state by using both the:
-
14.5% figure from Benson et al. for the proportion of people having surgery at 3 years
-
46.0% figure from Lourijsen et al. for the proportion of people who transition from the post-op controlled to uncontrolled health state in the year after surgery.
This resulted in an estimate of 13.1% for the proportion of people transitioning from the uncontrolled to post-op controlled health state. The committee noted the resulting transition probabilities aligned with the 14.5% figure from Benson et al., and with clinical and patient expert feedback that the probability of having surgery was lower than the probability of symptoms returning after surgery.
The committee noted the high level of uncertainty associated with the transition probabilities used in the model. It noted that the rate of loss of control from Lourijsen et al. was not constant and it would have preferred to account for the time varying nature of the probability. Based on the transition probabilities presented, it concluded that the following transitions were most appropriate for decision making:
-
46.0% from the post-op controlled to uncontrolled health state
-
13.1% from the uncontrolled to post-op controlled health state.
The committee chose these because they were based on evidence from a randomised controlled trial in people with CRS with nasal polyps, aligned with data from Benson et al., and reflected the experience of the clinical and patient experts. However, the committee emphasised that these transition probabilities were still associated with considerable uncertainty and limitations in the available evidence. It noted that the estimates were based on the best available data and expert opinion, but that more robust, prospective data would have been preferred to inform these parameters. The committee concluded that the approach taken in this case was not optimal, and that future evaluations should seek to use more clearly validated and transparently derived transition probabilities wherever possible.