5 Outcomes

The Diagnostics Advisory Committee (section 11) considered evidence from a number of sources (section 12).

How outcomes were assessed

5.1 The assessment consisted of a systematic review of the evidence on test performance and clinical-effectiveness data for faecal calprotectin testing. The outcome measures included in the assessment were:

  • referral rates

  • numbers of colonoscopies with or without faecal calprotectin testing

  • proportion of colonoscopies with no abnormal findings

  • duration from onset of symptoms to definite diagnosis of inflammatory bowel disease (IBD) – late diagnosis of Crohn's disease

  • costs

  • adverse events such as complications of colonoscopy

  • quality of life and hence quality‑adjusted life years (QALYs).

5.2 The External Assessment Group did a systematic review of the evidence on cost effectiveness for faecal calprotectin testing and constructed a de novo economic model. The outcomes of interest for the economic evaluation were the morbidity and mortality associated with inflammatory and non‑inflammatory diseases of the bowel and their treatment, and particularly of people with IBD incorrectly diagnosed as irritable bowel syndrome (IBS). Given the chronic nature of the conditions and their potential impact on a person's quality of life, the main outcome of interest was health-related quality of life, including the impact of adverse effects associated with colonoscopy. The de novo economic model followed a linked evidence approach in which intermediate outcomes (results of faecal calprotectin testing) were linked to treatment outcomes and hence QALY gains. Costs and QALYs were assigned to each of the strategies assessed in the model.

5.3 Although the scope allowed for the assessment of faecal calprotectin testing for both adults and children in both primary and secondary care, the External Assessment Group modelled 2 specific scenarios: faecal calprotectin testing for distinguishing between IBD and IBS in an adult population in primary care; and faecal calprotectin testing for distinguishing between IBD and non‑IBD in a paediatric population in secondary care. The External Assessment Group believed these scenarios reflect the most likely use of faecal calprotectin testing in clinical practice.

Clinical effectiveness

Previous systematic reviews

5.4 Five previously conducted systematic reviews of faecal calprotectin testing were quality assessed and summarised by the External Assessment Group.

5.5 In summary, reviews conducted recently (published in 2010 or later) and judged to be medium or high quality by the External Assessment Group concluded that faecal calprotectin testing is a useful tool. For example, the Centre for Evidence‑based Purchasing (2010) review focusing on faecal calprotectin for distinguishing between IBS and IBD concluded that faecal calprotectin performs well in distinguishing organic bowel disease from functional bowel disease (organic disease includes IBD and functional disease includes IBS). Sensitivity and specificity were over 80% in most of these studies (at a 50 micrograms/g cut‑off) and, when calculated, most positive and negative predictive values were 70–90%.

Diagnostic accuracy of faecal calprotectin – the comparisons

5.6 The External Assessment Group summarised the ability of faecal calprotectin testing in 4 sets of comparisons:

  • organic compared with non‑organic

  • IBS compared with IBD (most appropriate comparison for adults)

  • organic compared with IBS

  • IBD compared with non‑IBD (most appropriate comparison for paediatrics).

5.7 Organic disease includes inflammatory diseases. 'Organic' disease is formally defined as a condition in which there is an observable and measurable disease process (for example, inflammation).

5.8 The External Assessment Group suggested that comparisons 2 and 4 represent the most likely use of faecal calprotectin testing in clinical practice and therefore that the economic analysis should focus on the cost effectiveness of faecal calprotectin testing within these applications of the test. Diagnostic accuracy data for comparisons 2 and 4 are summarised below. Faecal calprotectin testing is used in symptomatic patients to distinguish between 2 different types of disease. Diagnostic sensitivity refers to the proportion of patients whose test is positive in the presence of an inflammatory disease of the bowel (such as IBD); diagnostic specificity refers to the proportion of patients whose test is negative in the absence of inflammatory disease of the bowel. Patients whose test is negative may be found to have IBS.

5.9 Nearly all of the evidence came from studies in secondary care, with few data from primary care. Data from a pilot project supported by the NHS Technology Adoption Centre were available to the External Assessment Group and were used in the economic analysis. These data are also summarised below.

IBS compared with IBD

5.10 Seven studies gave results that compared IBS and IBD, at 8 cut‑off levels ranging from 8–150 micrograms/g, all in adults in secondary care. All studies assessed enzyme‑linked immunosorbent assay (ELISA) tests, and one also assessed the performance of the point‑of‑care test (POCT) CalDetect. As expected, low cut‑offs gave high sensitivity for IBD but poor specificity. Sensitivity was consistently high (usually 100% at levels under 50 micrograms/g; ranging from 83–100% at a cut‑off of 50 micrograms/g), but specificity was more varied (51–100%).

5.11 Many of the studies had a small sample size. The largest study was by Li et al. (2006), which employed a sample of 240 people. Studies were of mixed quality. Schroder et al. (2007) and Schoepfer et al. (2008) were assessed as having the least risk of bias.

5.12 Five studies reported data for faecal calprotectin testing with ELISA with a cut‑off of 50 micrograms/g. This allowed for the meta‑analysis of the studies to provide an overall combined estimate of sensitivity and specificity. The combined estimates for ELISA tests, at a 50 micrograms/g cut‑off, were a sensitivity of 93% and a specificity of 94%. The meta‑analysis estimates were informed by a pool of 596 people, of which 40% were from the Li et al. (2006) study. The mean age of people in these studies, when reported, ranged from 40–52 years in people with IBS and 34–45 years in people with IBD. However, the age of people in the Schoepfer et al. (2008) study went as high as 78 years.

5.13 The only study using a POCT was Otten et al. (2008), which assessed the CalDetect test in a sample of 114 people. Otten et al. showed that the test performed well at a cut‑off of 15 micrograms/g, with a sensitivity of 100% and a specificity of 95%. At a cut‑off of 60 micrograms/g, although specificity improved slightly to 98%, sensitivity was only 61%, which the External Assessment Group considered to be unlikely to be acceptable in clinical practice given the importance of not missing people with IBD. The average age of people in the Otten et al. study was 52 years in people with IBS and 45 years in people with IBD.

5.14 The cost effectiveness of faecal calprotectin testing for distinguishing between IBD and IBS in an adult population in primary care was assessed in the economic evaluation conducted by the External Assessment Group.

IBD compared with non-IBD

5.15 Eleven studies reported IBD compared with non‑IBD, at 8 cut‑off levels. Eight studies were conducted in paediatrics and 3 in adults. All used ELISA tests, and one (Damms and Bischoff 2008) also assessed the Prevista POCT (not identified in the scope for the assessment).

5.16 The studies showed consistently high sensitivity at lower cut‑offs, nearly all over 90%, with most at the 50 micrograms/g cut‑off having sensitivities of 100%. Specificity was more varied, ranging from 44–93% at a 50 micrograms/g cut‑off. Most of these results were in paediatric groups. Most studies reported results at only 1 cut‑off, but 1 study reported 5 cut‑offs and another 4, both in paediatric populations. Studies were of mixed quality with Canini et al. (2006), Diamanti et al. (2010), Fagerberg et al. (2005), Henderson et al. (2012) and van de Vijver et al. (2012) assessed as having the least risk of bias compared with the other studies.

5.17 Six separate estimates of sensitivity and specificity were available at a cut‑off of 50 micrograms/g and another 6 estimates at 100 micrograms/g, which allowed the individual estimates to be meta‑analysed into combined overall estimates of sensitivity and specificity for ELISA tests. The overall pooled results for IBD compared with non‑IBD showed very high sensitivity of 99% but moderate specificity of 74% at a cut‑off of 50 micrograms/g. These estimates were informed by a pool of 531 people with most of these studies including people up to the age of 18 years. At a cut‑off of 100 micrograms/g, sensitivity was found to fall to 94% but specificity to improve to 82%. These estimates were informed by a pool of 656 people; however, the upper age limit varied in these studies. Two studies recruited people up to the age of approximately 15 years, 2 studies up to the age of 18 years and 1 study up to an age of 20 years. The age limit was not reported in the sixth study.

5.18 The cost effectiveness of faecal calprotectin testing for distinguishing between IBD and non‑IBD in a paediatric population in secondary care was assessed in the economic evaluation conducted by the External Assessment Group.

Primary care pilot data on faecal calprotectin testing

5.19 Implementation projects for faecal calprotectin testing in 2 North East Clinical Commissioning Groups in Northumberland and Durham Dales during 2011/12 were undertaken by the NHS Technology Adoption Centre. The Durham Dales data were available to the External Assessment Group and were used to inform the economic analysis, which also allowed exploration of what might happen if faecal calprotectin testing is introduced in primary care.

5.20 The Durham Dales project provided data on GP referrals following the introduction of faecal calprotectin testing in primary care. GPs made diagnostic decisions based on clinical assessment and knowledge of the faecal calprotectin test result. They referred patients who they thought might have IBD, and managed those who they thought had IBS in primary care.

5.21 A final consultant diagnosis was made, based on faecal calprotectin test results and clinical data including colonoscopy. The clinical data came from GP and outpatient data, when patients were referred, or just from GP data, when patients were not referred. Patients diagnosed as having IBS and not referred for specialist investigation did not have colonoscopy, so it was not possible to completely exclude patients with false negative results (partial verification bias). The Durham Dales data could not be used to inform the estimates of test accuracy for the CalDetect test (used in the implementation project) in the main economic analysis in primary care because of the partial verification bias.

5.22 Using the Durham Dales data of 111 patients who were followed up, the External Assessment Group used a prevalence of IBD of 6.3% in primary care and, in the absence of faecal calprotectin testing, a sensitivity of GP current practice of 100%, and 79% specificity in the model. The data also showed that GPs referred about 25% (29/111) of patients who presented with symptoms. The External Assessment Group created a scenario analysis that arbitrarily assumed that, if faecal calprotectin testing becomes available, GPs will test twice as many patients (50%) than they would have referred in the absence of faecal calprotectin testing.

5.23 Using the North-European data from Shivananda et al. (2006), a ratio of ulcerative colitis to Crohn's disease of 3:2 (incidence of ulcerative colitis 12.9 in 15–44 age group, based on 539 cases, and of Crohn's disease 8.7, based on 365 cases) would be expected in this adult population.

Ranges of faecal calprotectin values and choice of test

Ranges

5.24 The distribution of faecal calprotectin values is highly skewed and a wide range can be observed. Low levels may be seen in people with IBD and raised levels may be seen in people with IBS/non‑IBD (for example, people with infectious gastroenteritis or food poisoning).

5.25 In some studies, the ranges did not overlap, but in others they did. For example, in El‑Badry et al. (2010), the value of faecal calprotectin in people with IBD ranged from 98–637 micrograms/g, which did not overlap with the value of faecal calprotectin in people with IBS (14–65 micrograms/g). In all other studies, the range of faecal calprotectin in patients with IBD overlapped with the range of faecal calprotectin in patients with IBS. In some studies, such as Li et al. (2006) and Schroder et al. (2007), the range of faecal calprotectin levels in people with IBD was wide, with the lowest value being 15 micrograms/g and the highest being 2574 micrograms/g.

5.26 The range of results in studies comparing IBD and non‑IBD in children was similar to that found in studies comparing IBD and IBS in adults. In some studies (Canini et al. 2006; Diamanti et al. 2010; Sidler and Leach 2008), the ranges of faecal calprotectin levels overlapped in children and faecal calprotectin levels were high.

5.27 The External Assessment Group noted that faecal calprotectin levels were often raised in conditions other than IBD, such as larger colorectal adenomas and some colorectal cancers. The accuracy of faecal calprotectin testing is lower in these other conditions when compared with IBD.

Choice of test

5.28 The External Assessment Group summarised several studies that evaluated the comparative performance of faecal calprotectin tests in particular situations. For example, some studies assessed the performance of the tests for distinguishing IBS from IBD and others assessed the tests in distinguishing organic from non‑organic disease.

5.29 Overall, the External Assessment Group concluded that there are limited data comparing the performance of different faecal calprotectin tests. Of the studies conducted, they concluded that none suggested any considerable differences between the various faecal calprotectin tests.

Clinical outcomes

5.30 Modelling was used to estimate clinical outcomes and QALYs. Please refer to the economic analysis below.

Economic analysis

Review of existing economic analyses

5.31 Seven references were identified in the systematic review of economic analyses. Although previous economic analyses have typically concluded that faecal calprotectin testing is cost saving compared with diagnostic pathway costs without it, several issues were highlighted in the critique of the literature, which need further consideration. These included: the use of a small sample size to inform the analysis (Hornung and Anwar 2011); assumptions about test accuracy and no consideration of false negative results (Mindemark and Larsson 2012); the analysis considering colonoscopy but not faecal calprotectin testing (Goldfarb et al. 2004 and Dubinsky et al. 2002 – also, this analysis was conducted in the US context); studies that were conducted in England but in primary care only (York Health Economics Consortium [YHEC] economic report for the Centre for Evidence‑based Purchasing review, 2010); and some studies that were available only in abstract/poster format, which did not allow for a full critique of the analysis (Mascialino et al. 2012 and 2013).

5.32 The External Assessment Group constructed a de novo economic model to address the decision problem for this evaluation.

Cost-effectiveness model constructed by the External Assessment Group

5.33 The External Assessment Group constructed a full cost‑effectiveness model. The External Assessment Group model was informed by the model used in NICE clinical guideline 152 on Crohn's disease, the modelling for NICE clinical guideline 166 on ulcerative colitis, the modelling for NICE clinical guideline 61 on irritable bowel syndrome in adults and the YHEC model. In particular, these models were used to inform induction therapy and remission patterns in people with IBD and IBS.

Model structure

5.34 The model uses a linked-evidence approach to combine the outcomes of diagnostic strategies with the management (induction therapy and remission patterns) of patients' conditions, to allow the estimation of clinical outcomes and QALYs. The model assesses multiple diagnostic strategies and allows for multiple test sequences to be considered (for example, an initial ELISA test followed by colonoscopy). The outcomes from the diagnostic pathway are linked to the care pathway following diagnosis. Patients with true positive results for Crohn's disease and ulcerative colitis are considered separately from one another because patients in these groups follow different and complicated induction and remission pathways post diagnosis. Both patients with true negative and with false negative results follow the care pathway for IBS, with those patients whose disease does not respond to dietary changes after advice and subsequent medical treatment for IBS being retested for IBD. Patients with false positive results (incorrectly diagnosed as having IBD) are eventually correctly diagnosed as having IBS, given that it is assumed all patients with false positive results are referred for specialist investigation and undergo a colonoscopy (assumed 100% specificity). The model employs a weekly cycle and adopts a 10‑year time horizon.

Model aim

5.35 Although the scope allowed for the assessment of faecal calprotectin testing for both adults and children in both primary and secondary care, the External Assessment Group modelled 2 specific populations: an adult population in primary care, with faecal calprotectin test accuracies for IBD compared with IBS, and a paediatric population in secondary care, with faecal calprotectin test accuracies for IBD compared with non‑IBD. The External Assessment Group believed these populations reflect the most likely use of faecal calprotectin testing in clinical practice.

5.36 The main aim of the model was to assess the impact of faecal calprotectin testing when added to current clinical practice compared with current practice alone on the differentiation of IBD and IBS in primary care. This model was then adjusted to reflect the differing test performances and costs in the paediatric population to provide an approximation of the cost effectiveness of faecal calprotectin testing for distinguishing between IBD and non‑IBD. However, the External Assessment Group highlighted the limitation of this approach because the main model structure does not fully account for the non‑IBD case mix in the paediatric population (prevalence of IBS in the non‑IBD group is lower than that seen in adults).

Tests assessed in the modelling

5.37 The use of an ELISA faecal calprotectin testing service was evaluated in the base case for both of the primary and secondary care scenarios. The POCT CalDetect was evaluated in the base case primary care scenario.

Health-related quality of life (HRQoL)

5.38 The base case applied the quality‑of‑life decrements from remission to active disease of 0.280 for Crohn's disease and 0.200 for ulcerative colitis from Stark et al. (2010). But sensitivity analyses applying the quality‑of‑life decrements from mild‑to‑moderate disease of 0.075 for Crohn's disease, as drawn from Gregor et al. (1997), and of 0.165 for ulcerative colitis, as drawn from Poole et al. (2010), were also explored. The utility decrements for IBS were less important for modelling purposes, given that the 100% specificity assumed for colonoscopy meant that there were no patients with false positive results by the end of the test sequence. For the base case, the 0.071 increment for response to treatment estimated within NICE clinical guideline 61 was applied. The 0.662 baseline HRQoL that this increment was applied to was taken from Brazier et al. (2004). A sensitivity analysis using values from Spiegel et al. (2009) was also considered: 0.780 for response to treatment and 0.730 for no response to treatment, but the algorithm used to construct the EQ‑5D utilities was not clear. The baseline HRQoL value for IBS has an impact because of the small mortality rate associated with colonoscopy, with this impact enduring for the 10‑year time horizon of the model.

Adverse effects associated with colonoscopy

5.39 Because of data constraints, the cost impacts were limited to modelling the relatively rare (less than 0.5%) serious adverse events of bleeds and perforations. The quality‑of‑life impacts were limited to the mortality associated with perforations. While perforations are rare, resulting in a very low mortality rate, the QALY impact of this persisted for the duration of the model.

5.40 There is evidence from the literature that colonoscopies result in minor adverse events among a reasonable proportion of patients; for example, de Jonge et al. (2012) suggested that around 40% of those investigated with colonoscopy have some effects persisting for 30 days after the colonoscopy. In common with NICE clinical guideline 118 on colonoscopic surveillance for the prevention of colorectal cancer, these minor adverse events were not taken into account in the modelling principally because of a lack of quality‑of‑life data.

Costs

5.41 The costs included in the model were the costs of the different tests, treatment costs (including induction therapy and maintenance therapy costs for people in remission), NHS resource costs (for example, staff time) and costs of adverse effects associated with colonoscopy.

5.42 The per person costs of an ELISA test and POCT CalDetect were estimated to be £22.79 (based on an assumption of 40 patient samples per 96 well‑plate, costed at the list price, plus an average 11–12 minutes of staff time at grade 6/7) and £24.03 (test list price plus cost of 15 minutes of GP practice‑nurse time) respectively.

5.43 Colonoscopy was estimated to cost £741.68 per person. This estimate was based on a weighted average of the NHS reference cost for outpatient and day cases without biopsy (procedures payment by results code FZ51Z/FZ54Z), or with biopsy (procedures payment by results code FZ52Z/FZ55Z) for colonoscopy or, when used, sigmoidoscopy. The cost included an outpatient gastroenterology appointment (£164) and costs of adverse effects (an average of £12 per colonoscopy).

Primary care analysis (IBS compared with IBD in adults) – key model characteristics and results

5.44 The base case considered the cost effectiveness of GP testing compared with GP testing plus faecal calprotectin testing in the adult population for distinguishing IBS from IBD.

Patient characteristics

5.45 For the primary care adult population, the model adopted a baseline age of 25 years for those presenting with symptoms, as used in NICE clinical guideline 152 on Crohn's disease. Consistent with the modelling in this guideline, the proportion of females was taken to be 50% for both Crohn's disease and ulcerative colitis. It appears that a higher proportion of people with IBS are female; in the Brazier et al. (2004) sample, 86% were female, although the External Assessment Group suggested this estimate may be towards the upper end. For IBS, the base case adopted a proportion of females of 75%. These estimates only affect the all‑population mortality risks. Because these risks are low during mid-adulthood, for both women and men, the average age and proportion of model inputs for women will have had a minimal impact on the results.

5.46 The base-case prevalence of IBD (6.3%) was drawn from the Durham Dales data, while the prevalence of ulcerative colitis among patients with IBD (a ratio of ulcerative colitis to Crohn's disease of 3:2) was drawn from Shivananda et al. (1996).

Strategies assessed

5.47 The strategies assessed were:

  • GP current practice (clinical assessment with no faecal calprotectin testing)

  • GP current practice plus the POCT CalDetect using a cut‑off of 15 micrograms/g

  • GP current practice plus ELISA testing using a cut‑off of 50 micrograms/g.

    The External Assessment Group opted to use the lower 15 micrograms/g cut‑off from Otten et al. (2008) because the data for the 60 micrograms/g cut‑off suggested only a slight gain in terms of a better specificity, 97.8% compared with 94.5%, but considerable loss in terms of a worse sensitivity, 60.9% compared with 100.0%. Test accuracy data used in the model are summarised in table 2 below.

5.48 The External Assessment Group assumed, given lack of evidence to the contrary, that the accuracy of ELISA testing is the same as would be obtained from ELISA testing in conjunction with GP current practice.

5.49 The delay between referral and colonoscopy was assumed to be 4 weeks and the time to retesting among those with negative tests but not responding to IBS therapy was assumed to be 12 weeks, both estimates being based on expert opinion. The latter estimate may have been optimistic because a sequence of unsuccessful treatments might be tried for people with IBS. This was explored in sensitivity analyses.

5.50 The modelling assumed that all people who test positive or have an indeterminate result are referred to secondary care and all of these people receive a colonoscopy (indeterminate results are treated as if the results are determinate). Because of a lack of data, the External Assessment Group was not able to incorporate the impact of a gastroenterologist's assessment on the number of people who will go on to receive a colonoscopy (which may also include the use of faecal calprotectin testing) in the base case; however, this is explored in sensitivity analysis.

Table 2 Primary care analysis – base-case test accuracy data 1

Test

GP current practice

CalDetect (POCT)

ELISA

Colonoscopy

Cut-off

15 micrograms/g

50 micrograms/g

Sensitivity

100%

100.0%

(95% CI 85–100%)

93.0%

(95% CI 85–98%)

95.0%

Specificity

79%

94.5%

(95% CI 88–98%)

94.0%

(95% CI 76–100%)

100.0%

Test accuracy data source

Primary care data from the NHS Technology Adoption Centre project

Secondary care data from Otten et al. (2008)

External Assessment Group meta-analysis of secondary care data

Expert opinion

Abbreviations: CI, confidence interval; ELISA, enzyme‑linked immunosorbent assay; POCT, point‑of‑care test

1 Confidence intervals used in the probabilistic sensitivity analysis, when reported, are given in brackets

Base-case cost-effectiveness results – primary care

5.51 Without faecal calprotectin testing, GP current practice is highly sensitive in terms of referring people with IBD and is as good as, if not better, than faecal calprotectin testing. Of the 6.3% of people with IBD in the total population, all were identified by the GP current practice strategy and the POCT CalDetect strategy. Colonoscopy would correctly identify 6.0% of the 6.3% referred as patients with true positive results (because of its 95% sensitivity), resulting in a total of 0.3% of patients with false negative results. ELISA testing is slightly worse, identifying 5.9% of the 6.3% (because of its lower sensitivity when compared with current practice and the POCT), with 0.4% of patients being classified as having false negative results. Of the 5.9% referred for colonoscopy, 5.6% of patients would be identified as having true positive results, with 0.3% being classified as having false negative results, resulting in a total of 0.7% of patients with false negative results. Therefore, a slightly larger number of people will have IBD but will be incorrectly diagnosed as having IBS when using an ELISA testing strategy when compared with current practice strategy and a POCT CalDetect strategy (0.7% compared with 0.3%).

5.52 Within the total patient population, GP current practice incorrectly identified 19.8% of patients as having false positive results (people thought to have IBD but who actually have IBS) and requiring referral for colonoscopy. The rates of patients with false positive results incorrectly referred for colonoscopy for POCT CalDetect and ELISA were much lower, at 5.1% and 5.6% respectively. Therefore, without faecal calprotectin testing, many of the patients with false positive results would go on to have a colonoscopy, which has a risk (although low) of serious complications such as perforation. Such events are too rare to significantly affect costs, but they do have some QALY impact. This is also true for the more common minor adverse effects of colonoscopy (which were not explicitly considered in the model because of a lack of data).

5.53 Taking the diagnostic performance of the different testing strategies summarised above into account, the resulting total per patient costs and QALYs are provided in table 3.

Table 3 Primary care – base-case results (per patient)

Strategies

QALYs

Test costs

Other costs

Total costs

GP current practice (no faecal calprotectin testing)

Crohn's disease

0.1832

£22

£493

£515

Ulcerative colitis

0.2771

£32

£144

£176

IBS

5.7682

£202

£2404

£2606

Total

6.2285

£257

£3041

£3297

Current practice plus POCT CalDetect (15 micrograms/g cut‑off)

Crohn's disease

0.1832

£23

£493

£516

Ulcerative colitis

0.2771

£33

£144

£177

IBS

5.7691

£114

£2408

£2522

Total

6.2293

£170

£3044

£3214

Current practice plus ELISA (50 micrograms/g cut‑off)

Crohn's disease

0.1831

£23

£492

£515

Ulcerative colitis

0.2770

£34

£143

£177

IBS

5.7690

£116

£2407

£2524

Total

6.2291

£173

£3042

£3215

Abbreviations: ELISA, enzyme-linked immunosorbent assay; IBS, irritable bowel syndrome; POCT, point‑of‑care test; QALYs, quality‑adjusted life years

5.54 The faecal calprotectin tests were estimated to result in similar average cost savings compared with GP current practice: £83 for the POCT CalDetect and £82 for ELISA per patient. This was mainly because of the lower number of referrals and colonoscopies for false positive results. Average QALY gains of around 0.0007 QALYs were also accrued, although these were limited because the low prevalence of IBD and the similar high sensitivities of the tests resulted in relatively few false negative results. Therefore, the faecal calprotectin testing strategies dominated current practice (provided greater benefit at reduced cost). Some of the QALY differences accrued were from the very slightly lower mortality associated with the lower number of colonoscopies. The POCT CalDetect and ELISA strategies were estimated to be broadly equivalent in terms of costs and QALYs, with only minor differences between them.

Sensitivity analysis

5.55 A range of sensitivity analyses were conducted to explore the impact of varying the main model parameters. These included: varying the prevalence of IBD between 5–25% (6.3% used in the base case); changing the source of utility values; adjusting the costs of colonoscopy (no outpatient appointment cost) and removing any associated mortality; varying the number of patients whose condition did not respond to IBS medication; varying the time it takes for patients with false negative results to re‑present to the clinician (8, 16 and 24 weeks; 12 weeks was used in the base case); and exploring the impact of varying the specificity of the consultant's diagnosis at an outpatient clinic assessment in people referred with a false positive diagnosis. Scenario analyses were also undertaken using different sources of test accuracy for faecal calprotectin and alternative assumptions surrounding the uptake of faecal calprotectin testing (assuming 50% of patients are tested as opposed to the 25% used in the base case) in primary care.

5.56 The sensitivity and scenario analyses appeared to broadly affect the 3 strategies in a similar way and suggested that the results of the base case were reasonably robust. The most notable impact was from assuming 50% of patients are tested in primary care, as opposed to 25%, which reduced the cost savings with faecal calprotectin testing.

Secondary care (IBD compared with non‑IBD in children) – key model characteristics and results

5.57 The base case considered the cost effectiveness of faecal calprotectin testing before colonoscopy compared with direct referral for colonoscopy in the secondary care paediatric population for distinguishing IBD from non‑IBD.

Patient characteristics

5.58 For the secondary care paediatric population, the proportions of females included were 38% (35/91) for patients with IBD and 44% (44/99) for patients without IBD; these were drawn from Henderson et al. (2012). An average age of 16 years was assumed because, for the adult modelling, this had minimal impact on results.

5.59 In the base case, a 48% (91/190) prevalence of IBD and a 75% (62/83) prevalence of Crohn's disease among patients with IBD were drawn from Henderson et al. (2012).

Strategies assessed

5.60 The strategies assessed were:

  • direct referral for colonoscopy

  • ELISA testing when used at the 50 micrograms/g cut‑off followed by colonoscopy

  • ELISA testing when used at the 100 micrograms/g cut‑off followed by colonoscopy.

    Test accuracy data used in the model are summarised in table 4.

Table 4 Secondary care scenario – base‑case test accuracy data 1

Test

ELISA

ELISA

Colonoscopy

Cut-off

50 micrograms/g

100 micrograms/g

Sensitivity

99.0%

(95% CI: 95–100%)

94.0%

(95% CI: 87–99%)

95.0%

Specificity

74.0%

(95% CI: 59–85%)

82.0%

(95% CI: 68–92%)

100.0%

Test accuracy data source

External Assessment Group meta-analysis of secondary care data

External Assessment Group meta-analysis of secondary care data)

Expert opinion

Abbreviations: CI, confidence interval; ELISA, enzyme‑linked immunosorbent assay

1 Confidence intervals used in the probabilistic sensitivity analysis, when reported, are given in brackets

Base-case cost-effectiveness results – secondary care

5.61 The base-case prevalence of IBD of 47.9% increased the importance of test sensitivities compared with the primary care setting, and so the effect of false negative results on the modelling outputs. Within the total patient population, ELISA with the 50 micrograms/g cut‑off led to 47.4% of patients with true positive results being referred for colonoscopy, while ELISA with the 100 micrograms/g cut‑off led to 45.0% of patients with true positive results being referred for colonoscopy. Colonoscopy was assumed to have a sensitivity of 95%. So, if all (47.9%) patients were referred immediately for colonoscopy, 45.5% would be diagnosed with IBD. With ELISA with the 50 micrograms/g cut‑off, 45.0% of the 47.4% of patients referred for colonoscopy were diagnosed as having IBD, while 42.8% of the 45.0% of patients referred for colonoscopy after ELISA with the 100 micrograms/g cut‑off were diagnosed as having IBD; a net difference between the cut‑offs of 2.2%.

5.62 Despite the higher IBD prevalence in the secondary care population, the main test differences still lay in the number of unnecessary colonoscopies. Without faecal calprotectin testing, all 52.1% of patients without IBD received a colonoscopy, compared with 13.5% for ELISA with the 50 micrograms/g cut‑off and only 9.4% for ELISA with the 100 micrograms/g cut‑off.

5.63 Taking the diagnostic performance of the different testing strategies summarised above into account, the resulting total per patient costs and QALYs are provided in table 5.

Table 5 Secondary care: base-case results (per patient)

Strategies

QALYs

Tests

Other

Total

Direct referral for colonoscopy

Crohn's disease

2.5773

£244

£6938

£7183

Ulcerative colitis

0.8942

£83

£463

£546

Non-IBD

3.2094

£338

£629

£967

Total

6.6809

£665

£8031

£8696

ELISA 50 micrograms/g before colonoscopy

Crohn's disease

2.5767

£254

£6934

£7188

Ulcerative colitis

0.8941

£86

£463

£549

Non-IBD

3.2117

£120

£634

£754

Total

6.6824

£460

£8031

£8491

ELISA 100 micrograms/g before colonoscopy

Crohn's disease

2.5757

£256

£6921

£7177

Ulcerative colitis

0.8938

£87

£462

£549

Non-IBD

3.2119

£95

£634

£729

Total

6.6814

£438

£8018

£8456

Abbreviations: ELISA, enzyme-linked immunosorbent assay; IBD, inflammatory bowel disease; QALYs, quality‑adjusted life years

5.64 Prior testing using ELISA was estimated to dominate (provided greater benefit at reduced cost) the strategy of sending all patients directly for colonoscopy. Compared with referring all patients directly for colonoscopy, ELISA used at the 50 micrograms/g cut‑off was estimated to save £205 per patient, while ELISA used at the 100 micrograms/g cut‑off was estimated to save £240 per patient. QALY gains of around 0.001 QALYs were estimated for ELISA compared with direct referral for colonoscopy, these being slightly larger for ELISA with the 50 micrograms/g cut‑off because of its better sensitivity. The additional average cost of £35 and additional average QALYs of 0.0001 for ELISA with the 50 micrograms/g cut‑off compared with ELISA with the 100 micrograms/g cut‑off resulted in an incremental cost‑effectiveness ratio (ICER) of £35,000 per QALY gained.

Sensitivity analysis

5.65 A range of sensitivity analyses were conducted to explore the impact of varying the main model parameters. These included: varying the prevalence of IBD to 40% and 60% (48% used in the base case); changing the source of utility values; removing any associated mortality of colonoscopy; varying the time it takes for patients with false negative results to re‑present to the clinician (8, 16 and 24 weeks; 12 weeks was used in the base case); and changing the annualised net cost of false negative results to £376 (£188 was used in the base case).

5.66 As for primary care, most of the changes appeared to broadly affect the 3 strategies in a similar manner. The main difference arose from varying the prevalence of IBD, which tended to reduce the cost savings from faecal calprotectin testing because of the rise in prevalence, as would be anticipated. The source of utilities also had an impact on the anticipated net gain from ELISA with the 50 micrograms/g cut‑off compared with ELISA with the 100 micrograms/g cut‑off, the ICER for which increased to £117,000 per QALY gained. However, the External Assessment Group thought that this may have overstated the effect, given the prevalence of Crohn's disease within the presenting population and the perhaps rather small quality‑of‑life decrement sourced from Gregor et al. (1997).

  • National Institute for Health and Care Excellence (NICE)