3 The manufacturer's submission

The Appraisal Committee considered evidence submitted by the manufacturer of rivaroxaban and a review of this submission by the Evidence Review Group (ERG).

3.1 The main clinical effectiveness evidence came from one multicentre, double-blind randomised controlled trial. The ROCKET-AF trial ('Rivaroxaban once daily oral direct factor Xa inhibitor compared with vitamin K antagonism for prevention of stroke and embolism trial in atrial fibrillation') compared rivaroxaban with dose-adjusted warfarin. The manufacturer also compared rivaroxaban with aspirin and dabigatran etexilate (110 mg or 150 mg twice a day) using a network meta-analysis in people for whom anticoagulation therapy was considered suitable. The ROCKET-AF trial was designed as double-blind, double dummy trial comparing rivaroxaban (20 mg or 15 mg once a day) with warfarin (target INR of 2.0 to 3.0) for the prevention of stroke and thromboembolic events in people with non-valvular atrial fibrillation at risk of future thromboembolic events. People were randomly allocated to one of the two treatment groups with equal probability (1:1 allocation ratio). The study took place in 45 countries, including the UK, and a total of 14,264 people were enrolled across the two treatment arms (rivaroxaban n = 7131, warfarin n = 7133). Treatment continued until approximately 405 adjudicated primary efficacy end point events had occurred in the per-protocol population on treatment. As a result, the time on treatment varied from patient to patient depending on when they enrolled in the trial. The median duration of treatment was 590 days.

3.2 The primary efficacy end point in ROCKET-AF was a composite of stroke (ischaemic and haemorrhagic stroke) and non-central nervous system systemic embolism. The primary safety end point was defined as a composite of major bleeding and clinically relevant non-major bleeding. To show non-inferiority in preventing stroke and non-central nervous system embolism, the upper boundary of the confidence interval of the hazard ratio (HR) for rivaroxaban compared with warfarin had to be less than 1.46. Once non-inferiority was demonstrated for the primary outcome, further analyses investigated superiority of rivaroxaban over warfarin.

3.3 More than 50% of people in the trial received treatment for at least 18 months. The median age of study participants was 73 years and 60.3% were men. The majority of the trial population (62.4%) had previously received warfarin therapy and 36.5% had previously received aspirin. Risk of stroke at baseline was classified according to CHADS2 score, which is used to predict the risk of stroke in people with atrial fibrillation. The trial entry criteria included a history of stroke, transient ischaemic attack or systemic embolism, or a CHADS2 score of 2 or more. The mean CHADS2 score was 3.48 for the rivaroxaban group and 3.46 for the warfarin group; 99.98% of the trial population had a baseline CHADS2 score of 2 or more and 86.95% a baseline CHADS2 score of 3 or more. Three participants (0.02% of the trial population) had a baseline CHADS2 score of 1. In the warfarin group the mean time in therapeutic range for the INR range of 2.0 to 3.0 was 55% (58% median). Some variability was observed in time in therapeutic range by region: north America had the highest overall INR control, followed by western Europe, Latin America, Asia Pacific, and eastern Europe.

3.4 Three analyses were defined in the manufacturer's submission for the efficacy analysis: the intention-to-treat set (all patients randomised), the safety-on-treatment set (all intention-to-treat patients who had taken at least one dose of the study drug and were followed for events) and the per-protocol set (all intention-to-treat patients excluding those who had major pre-defined protocol deviations). The primary non-inferiority analysis of the ROCKET-AF trial was conducted on the per-protocol and the safety-on-treatment population data sets. The superiority analyses were conducted on the safety-on-treatment population data sets. In addition to these analyses, sensitivity analyses were performed to assess non-inferiority and superiority in the intention-to-treat population. The primary safety analysis was conducted on the safety-on-treatment population data.

3.5 Pre-planned subgroup analyses were conducted. These were by region, prior use of vitamin K antagonists (such as warfarin), and history of stroke, transient ischaemic attack, and non-central nervous system systemic embolism. Other subgroups included prior chronic aspirin use, sex, age, family origin, renal function, body mass index, weight, CHADS2 score, congestive heart failure, hypertension, diabetes, type of atrial fibrillation, proton pump inhibitor use at baseline, and prior myocardial infarction. Results were summarised by subgroup based on data from the safety-on-treatment and intention-to-treat populations.

3.6 The non-inferiority of rivaroxaban compared with warfarin was demonstrated for the primary outcome (composite of stroke and non-central nervous system systemic embolism) in both the per-protocol and safety-on-treatment populations. The results for the per-protocol population were HR 0.79 (95% confidence interval [CI] 0.66 to 0.96) and HR 0.79 (95% CI 0.65 to 0.95) for the safety-on treatment population. Superiority of rivaroxaban over warfarin was also demonstrated in the safety-on-treatment population, but was not demonstrated for this outcome in the sensitivity analysis using the intention-to-treat population data set (HR 0.88, 95% CI 0.75 to 1.03).

3.7 For the primary safety end point of major or non-major clinically relevant bleeding, the results from the safety-on-treatment population data for ROCKET-AF suggest a comparable safety profile for rivaroxaban and warfarin, with no statistically significant difference between the two treatments (HR 1.03, 95% CI 0.96 to 1.11). Bleeding sites for the primary safety end point differed between treatment groups. Rivaroxaban was more often associated with bleeding at sites throughout the gastrointestinal tract (3.15% compared with 2.16%, p < 0.001) but intracranial haemorrhage rates were significantly lower with rivaroxaban than with warfarin (0.5% compared with 0.7%, p = 0.02). Following a request from the ERG the manufacturer provided subgroup analyses for the safety-on-treatment and intention-to-treat populations in people who had previously used vitamin K antagonists, people who had not previously used vitamin K antagonists, people with a time in therapeutic range below 60%, and those with a time in therapeutic range above 60%. In the safety-on-treatment population, superiority of rivaroxaban compared with warfarin was demonstrated for the primary outcome (composite of stroke and non-central nervous system systemic embolism) in people who had not previously used vitamin K antagonists (HR 0.72, 95% CI 0.53 to 0.97) but not in people who had previously used vitamin K antagonists (HR 0.84, 95% CI 0.66 to 1.08).

3.8 The manufacturer undertook a Bayesian network meta-analysis comparison of rivaroxaban with warfarin, aspirin, no treatment and dabigatran etexilate. The clinical evidence for the rivaroxaban with warfarin comparison was taken from the ROCKET-AF trial. Evidence for the other comparators was obtained from studies found by a systematic literature search. The manufacturer identified 18 studies for inclusion in the network meta-analysis:

  • one comparing rivaroxaban with warfarin

  • seven comparing aspirin with placebo or control

  • eight comparing warfarin with aspirin

  • one comparing a vitamin K antagonist with clopidogrel plus aspirin

  • one comparing dabigatran etexilate with warfarin (the RE-LY study).

    The manufacturer reported network meta-analysis results for the outcomes using the ROCKET-AF safety-on-treatment population data set. At the request of the ERG, the manufacturer also provided the results for the outcomes using the ROCKET-AF intention-to-treat population dataset. The efficacy estimates from this network meta-analysis using the ROCKET-AF safety-on-treatment population data set were used in the manufacturer's cost-effectiveness analyses.

3.9 The ERG undertook an exploratory network meta-analysis comparing rivaroxaban with dabigatran etexilate, aspirin, placebo, and adjusted standard dose warfarin. It included data from eight of the 18 studies from the manufacturer's network meta-analysis:

  • one comparing dabigatran etexilate with warfarin

  • one comparing rivaroxaban with warfarin

  • three comparing aspirin with warfarin

  • three comparing warfarin with placebo.

    The ERG judged that including only these eight trials would reduce the amount of heterogeneity in the network. Only dosing strategies that the ERG considered to be comparable were included (that is, rivaroxaban 20 mg per day, dabigatran etexilate 150 mg twice a day, aspirin 300 mg per day, and adjusted dose warfarin aiming at a target INR range between 2 and 3). Dabigatran etexilate at a dose of 110 mg twice daily was not included in the analysis because the economic model could not accommodate a dosing regimen in which the dose is reduced from 150 mg twice daily to 110 mg twice daily when the patient reaches the age of 80 years. A fixed-effect model was used because of the high degree of homogeneity between the included trials. The efficacy estimates from this network meta-analysis were used in the ERG's cost-effectiveness analyses.

3.10 The manufacturer developed a Markov model that compares rivaroxaban (20 mg once a day) with warfarin (adjusted dose warfarin at 4.5 mg once a day, target INR 2.5, range 2.0 to 3.0), aspirin (150 mg once a day), dabigatran etexilate (110 mg to 150 mg twice a day) and no treatment. The population in the model is the same as the ROCKET-AF safety-on-treatment population. The model has a lifetime time horizon and a UK NHS perspective.

3.11 The model included the following health states:

  • anticoagulant initiation

  • stable atrial fibrillation (on or off therapy)

  • minor stroke (on or off therapy)

  • major stroke (on or off therapy)

  • post minor stroke (on therapy)

  • post major stroke (on therapy)

  • minor bleed (on or off therapy)

  • major bleed (on or off therapy)

  • intracranial bleed (on or off therapy)

  • post intracranial bleed (on or off therapy)

  • systemic embolism (on or off therapy)

  • myocardial infarction (on or off therapy)

  • post myocardial infarction (on or off therapy)

  • death.

    The ROCKET-AF trial results for the safety-on-treatment population were used to inform the efficacy estimates for rivaroxaban compared with warfarin, rivaroxaban compared with warfarin in people whose atrial fibrillation is poorly controlled on warfarin, and the vitamin K antagonist-naive model populations. The characteristics of the population for the analyses of rivaroxaban compared with aspirin, dabigatran etexilate and no treatment were based on the patient characteristics of a UK GP practice-based survey (Gallagher et al. 2008). Efficacy estimates for rivaroxaban compared with dabigatran etexilate and aspirin were obtained from the manufacturer's network meta-analysis.

3.12 The manufacturer classified all model events as either transient or permanent depending on associated long-term costs and consequences:

  • Systemic embolism, minor extracranial bleeds and major extracranial bleeds were assumed to have no lasting clinical or economic consequences and as such were considered transient events in the model.

  • Minor stroke, major stroke, intracranial bleeding and myocardial infarction were considered by the manufacturer to be permanent events, in the sense that they have lasting clinical and economic consequences. Consequently, the manufacturer developed post-event health states to account for the different risks, costs and utilities associated with surviving a permanent event.

3.13 The manufacturer highlighted that increasing age was an important risk factor for ischaemic stroke and systemic embolism, and adjusted the baseline risk of these events to account for patients aging as they move through the model. Risks were calculated using the Framingham risk equations. In the model, a weighted average relative risk (weighted by the proportion of patients in each risk group at initiation) is calculated for each age group and applied to the baseline risk as patients enter that age group. The risks of extracranial bleeding, intracranial bleeding and myocardial infarction were assumed to be independent of time and, therefore, were not adjusted for.

3.14 The baseline risk of each event was adjusted according to the treatment regimen the patient was receiving. Patients may stop their primary therapy and switch to a pre-specified secondary therapy at any time, although the risk adjustment applied for the remainder of that cycle is that of the primary therapy. The probabilities of treatment discontinuation for warfarin and rivaroxaban were based on data from the ROCKET-AF trial. The manufacturer assumed that treatment-discontinuation rates for aspirin, dabigatran etexilate and placebo were equivalent to that for rivaroxaban, given the similarity of administration between these interventions.

3.15 The health-state utility values and treatment-related utility values in atrial fibrillation were obtained from published sources identified by a systematic literature search. The ROCKET-AF trial did not include a generic measure of health-related quality of life (such as the EQ-5D) that could be used to estimate utilities in the model. The estimates of resources and costs were obtained from NHS reference costs for 2009/10 and systematic literature searching. The manufacturer's model categorised monitoring costs into the following distinct phases: initiation, maintenance, and re-initiation. The manufacturer's model calculated the quarterly cost of initiation, maintenance and re-initiation by taking a weighted average of each cost; the costs of the individual phases were weighted by the proportion of people treated in primary and secondary care, as indicated by the manufacturer's survey. The resulting annual cost estimates for warfarin monitoring were £663 (£448 in primary care and £215 in secondary care) for the first year and £525 (£359 in primary care and £166 in secondary care) for subsequent years.

3.16 The manufacturer's base-case analysis of rivaroxaban compared with warfarin used only statistically significant data from the ROCKET-AF safety-on-treatment population. The analysis produced an incremental cost of £740 and a QALY gain of 0.039 resulting in an incremental cost-effectiveness ratio (ICER) of £18,883 per quality-adjusted life year (QALY) gained. The manufacturer also presented the results of four subgroup analyses:

  • For rivaroxaban compared with warfarin in people whose INR is poorly controlled on warfarin, rivaroxaban dominated (was more effective and cost less) warfarin.

  • For rivaroxaban compared with warfarin in people who had not previously received warfarin, the ICER was £15,494 per QALY gained.

  • For rivaroxaban compared with aspirin, the ICER was £2083 per QALY gained.

  • For rivaroxaban compared with dabigatran etexilate, rivaroxaban dominated dabigatran etexilate.

    The manufacturer carried out univariate sensitivity analysis on the base case, scenario analyses and subgroup analyses. The main drivers of the model results were consistent across analyses, with the cost of warfarin monitoring in primary care having a major impact on all ROCKET-AF-based analyses. The probabilistic sensitivity analyses indicated that, using the base case, rivaroxaban had a 75% probability of being cost effective at a maximum acceptable ICER of £20,000 per QALY gained and an 88% probability at £30,000 per QALY gained. The manufacturer's scenario analysis of rivaroxaban compared with warfarin used all point estimates from the ROCKET-AF safety-on-treatment population regardless of their statistical significance (that is, both statistically significant and non-statistically significant point estimates were used). The resulting ICER was £8732 per QALY gained.

3.17 In the ERG's view, a Markov model was an appropriate choice for modelling the chronic condition of atrial fibrillation. The ERG noted that the manufacturer chose a cycle length of 3 months and that only one event per 3-month cycle was possible because of the nature of the model. The manufacturer acknowledged that, in reality, people may experience more than one event in 3 months, but clinical opinion was that the probability of this would be low. The ERG agreed that assuming one event per model cycle was necessary and reasonable. However, the ERG noted that the manufacturer's model also suspends the risk of further events in the subsequent model cycle. The ERG considered that this additional suspension of risk was likely to bias the analysis against the more effective treatment because the overall event rate would be lower, and the potential to demonstrate clinical and economic benefits would also be lower.

3.18 The ERG identified the following limitations to the model's structural assumptions and parameter sources:

  • not separating out the number of hospital visits needed by people who were within and outside recommended INR control

  • not adjusting risk of bleeding by age; not adjusting utility by age

  • the source of myocardial infarction risk for people treated with aspirin

  • out of date source of post-myocardial infarction mortality risk

  • double counting of re-initiation costs of warfarin monitoring

  • suspending the risk of further events for the subsequent model cycle after an event

  • excluding transient ischaemic attack as a potential event.

3.19 The ERG presented an exploratory analysis in which, if possible, adjustments were made to account for the limitations identified (see section 3.18). The analysis for rivaroxaban compared with warfarin produced an incremental cost of £1134 and a QALY gain of 0.034, resulting in an ICER of £33,758 per QALY gained. Similarly, for warfarin-naive people, after incorporating the ERG's model adjustments, the ICER for rivaroxaban compared with warfarin increased from £15,494 to £29,894 per QALY gained. However, rivaroxaban remained dominant in people whose INR was poorly controlled on warfarin after the ERG's model adjustments were incorporated. The structure of the manufacturer's model meant it wasn't possible to remove risk suspension or add transient ischaemic attack as a potential event. Consequently, the ERG was unable to fully quantify the impact of these limitations on the ICERs. However, the ERG considered that suspending risk and excluding transient ischaemic attack as an event would favour rivaroxaban (that is, the removal of these limitations would decrease the ICER for rivaroxaban compared with warfarin), because warfarin is generally less effective than rivaroxaban (based on the safety-on-treatment population of ROCKET-AF).

3.20 In the ERG's view the manufacturer's base-case model is driven by the cost of anticoagulation monitoring rather than the differential effectiveness of rivaroxaban and warfarin. The ROCKET-AF trial showed that, for most outcomes, there was no statistically significant difference between rivaroxaban and warfarin. The ERG highlighted that when the cost of anticoagulation monitoring was separated out by INR range the ICER substantially increased from £18,883 per QALY gained to £27,281 per QALY gained. In addition, the ERG's scenario analysis using the alternative anticoagulation monitoring costs of £242 per person (discussed by the Committee in the ongoing appraisal of dabigatran etexilate for the prevention of stroke and systemic embolism in atrial fibrillation) increased the ICER to £62,568 per QALY gained.

3.21 The ERG was concerned that the trials included in the network meta-analysis presented by the manufacturer to compare rivaroxaban with aspirin and dabigatran etexilate were heterogeneous. The high levels of heterogeneity were not shown when the ERG conducted its own network meta-analysis restricting the network to the comparators specified in the final scope. When the ERG applied the treatment effects estimated by its network meta-analysis to the manufacturer's model, and a full incremental analysis of rivaroxaban, dabigatran etexilate, warfarin and aspirin was conducted, an ICER of £34,680 per QALY gained was obtained for dabigatran etexilate compared with rivaroxaban, whereas rivaroxaban had dominated dabigatran etexilate in the manufacturer's analysis. The ERG applied further adjustments to account for the following limitations:

  • the absence of a post-systemic embolism health state

  • not adjusting bleeding risk by age

  • not adjusting utility by age

  • out of date source of post-myocardial infarction mortality risk

  • assuming equivalent discontinuation rates.

    This reduced the ICER to £12,701 per QALY gained for dabigatran etexilate compared with rivaroxaban. Exploratory analysis assuming an equivalent ability of rivaroxaban and dabigatran etexilate to prevent myocardial infarction further decreased the ICER to £3578 per QALY gained for dabigatran etexilate compared with rivaroxaban.

3.22 The ERG noted the presence of potential biases in the model, with limitations of risk suspension and the absence of transient ischaemic attack and dyspepsia as adverse reactions. Removing risk suspension is likely to favour dabigatran etexilate whereas including transient ischaemic attack and dyspepsia is likely to reduce the ICER for rivaroxaban compared with dabigatran etexilate. Furthermore, the ERG noted that there is a large amount of uncertainty in the model and that the model is highly sensitive to even small changes to the discontinuation rates. Therefore, the ERG concluded that the results of the probabilistic sensitivity analysis should be taken into account when considering its alternative ICER for dabigatran etexilate compared with rivaroxaban. The probabilistic sensitivity analysis indicated that dabigatran etexilate was dominant in 45% of the 1000 runs and dominated in 35% of runs.

Manufacturer's additional analyses

3.23 The manufacturer provided additional analyses in response to the Appraisal Committee's request for further clarification on the cost effectiveness of rivaroxaban presented in the appraisal consultation document. The manufacturer provided a revised cost-effectiveness analysis incorporating the following amendments requested by the Appraisal Committee:

  • data from the General Practice Research Database to provide event rates according to baseline level of stroke risk and the distribution of patients with different CHADS2 scores from the study of Gallagher et al. (2008)

  • all the efficacy point estimates from the safety-on-treatment population of the ROCKET-AF trial

  • revised event rate in the warfarin arm to reflect the time in therapeutic range achieved in trial centres in western Europe (60.62%)

  • fixed annual warfarin INR monitoring cost of £242 per person in the sensitivity analysis only.

3.24 In addition to the amendments requested by the Appraisal Committee, the manufacturer also amended the model to include the following:

  • Revised annual warfarin INR monitoring cost of £580. The manufacturer used the same unit costs as the original model; however, the number of visits needed for the re-initiation was reduced from seven to five per 3-month cycle. The costs associated with warfarin monitoring in primary care in the updated model were £175.50 for initiation of warfarin (calculated as a weighted average of patients who had, and had not received previous warfarin), £135 for maintenance on warfarin and £135 for re-initiation of warfarin.

  • Case fatality rates of 90 days instead of the 30-day rates in the original model.

  • Updated 'real world' discontinuation rates. For warfarin these came from the General Practice Research Database. For rivaroxaban they were calculated by applying relative risks from the General Practice Research Database to discontinuation rates from the ROCKET-AF trial.

  • Treatment-related disutility applied to warfarin of 0.05. This was obtained from a study evaluating how patients with atrial fibrillation (attending GP- and hospital-led clinics) value different health outcomes. The disutility figures for warfarin were weighted by the UK distribution of primary and secondary care anticoagulation management.

  • Updated results for rivaroxaban compared with aspirin based on an additional indirect comparison. This comparison used only trials comparing rivaroxaban with warfarin and warfarin with aspirin, to reduce the network heterogeneity.

3.25 The manufacturer presented the following cost-effectiveness results when all the amendments in sections 3.23 and 3.24 were applied, with an annual warfarin INR monitoring cost of £580 (that is, excluding the Appraisal Committee's request to incorporate a fixed annual warfarin INR monitoring cost of £242 per person):

  • for rivaroxaban versus warfarin in the licensed population (the population with one or more risk factors for stroke), an incremental cost of £705, an incremental QALY of 0.2459 resulting in an ICER of £2869 per QALY gained

  • for rivaroxaban versus warfarin in the population whose INR is poorly controlled on warfarin, rivaroxaban dominated warfarin

  • for rivaroxaban versus warfarin in the population for whom warfarin is considered unsuitable, the ICER was £9170 per QALY gained.

3.26 The manufacturer presented the following cost-effectiveness results when all the amendments detailed in sections 3.23 and 3.24 were applied, with an annual warfarin INR monitoring cost of £242 per person (that is, including all of the Appraisal Committee's requests):

  • for rivaroxaban versus warfarin in the licensed population (the population with one or more risk factors for stroke), an incremental cost of £2220, a QALY gain of 0.2459 resulting in an ICER of £9031 per QALY gained

  • for rivaroxaban versus warfarin in the population whose INR is poorly controlled on warfarin, the ICER was £4350 per QALY gained.

3.27 The ERG provided a critique and exploratory analysis of the manufacturer's additional analyses. The ERG agreed that the manufacturer had adequately provided a model cohort representative of people with atrial fibrillation in the UK, and that the analysis was based on all efficacy point estimates from the ROCKET-AF trial. The ERG also agreed that it was reasonable to use a discontinuation rate of five and a 90-day case fatality rate in the model. However the ERG noted that the Committee's request to evaluate the effect of low time in therapeutic range was addressed as an amendment to the base case rather than the subgroup analysis requested. The ERG noted that varying the risk of stroke and systemic embolism according to level of INR control resulted in an increase in the ICER of £3742 per QALY gained.

3.28 In the ERG's view, the manufacturer's inclusion of a disutility for warfarin in the model was not appropriate. The ERG noted that the manufacturer had not provided any justification for the assumption that no disutility is associated with rivaroxaban, aspirin or dabigatran etexilate. The ERG pointed out that there is evidence of disutility associated with other oral anticoagulants such as dabigatran etexilate and therefore it is unreasonable to assume there is no disutility associated with rivaroxaban. The ERG found that removing the disutility for warfarin has a substantial impact on the ICER, increasing it from £2869 to £10,764 per QALY gained.

3.29 The ERG re-ran the manufacturer's updated cost-effectiveness analysis. The ERG noted that in both the original and updated models the manufacturer had categorised systemic embolism as a temporary event. The ERG judged that in order to adequately approximate a post-systemic embolism health state (which would account for the increased risk of stroke following a systemic embolism) it was appropriate to amend the model so that after a systemic embolism patients move into the post-minor stroke health state. The ERG also noted that the manufacturer's updated model continues to use the out of date source of post-myocardial infarction mortality risk. The ERG judged that it was more appropriate to use an updated mortality risk post myocardial infarction that took account of current use of statins. The ERG's analysis included a fixed annual warfarin INR monitoring cost of £242 per person as requested by the Appraisal Committee (see section 3.23) but did not include any disutility associated with warfarin or any other treatment (see section 3.24). The ERG's revised analysis for rivaroxaban compared with warfarin produced an incremental cost of £1815, an incremental QALY of 0.061 and an ICER of £29,537 per QALY gained.

  • National Institute for Health and Care Excellence (NICE)