4 Evidence and interpretation

The Appraisal Committee (appendix A) considered evidence from a number of sources (appendix B).

4.1 Clinical effectiveness

4.1.1 The independent systematic review identified 19 randomised controlled trials (RCTs) that were relevant to the appraisal and the results from these were included in the reviewed data. An unpublished meta-analysis based on individual patient data (IPD) from a subset of patients (n = 1536) from four RCTs was also considered. This independent meta-analysis was submitted by a manufacturer consultee before publication on an academic-in-confidence basis, and the results are not presented in this document. Neither the assessment report nor the consultee submissions made a distinction between laparoscopic and laparoscopically assisted surgery. No data were identified comparing hand-port-assisted laparoscopic surgery with open surgery.

4.1.2 When compared with open surgery, laparoscopic surgery was associated with a statistically significant longer operating time (weighted mean difference [WMD] 40 minutes, 95% confidence interval [CI] 32 to 48 minutes, based on three RCTs) and shorter hospital stay (WMD 2.6 days, 95% CI 2.0 to 3.1 days, based on four RCTs). The results with laparoscopic resection also suggested a trend towards a decreased number of lymph nodes retrieved (WMD 0.4, 95% CI 1.4 to 0.6 nodes, based on three RCTs), an increased risk of anastomotic leakage (pooled relative risk [RR] 1.13, 95% CI 0.74 to 1.73, based on eight RCTs), and a decreased risk of operative and 30-day mortality (based on three RCTs) compared with open resection, although these differences did not reach statistical significance.

4.1.3 Seven RCTs and the IPD meta-analysis reported overall survival. Raw data were available from six RCTs and contributed to a meta-analysis that did not show a statistically significant difference in overall survival between laparoscopic and open resection (pooled RR 1.03, 95% CI 0.98 to 1.09). However, these RCTs had widely differing follow-up periods that ranged from 1 to 108 months, and proportion of events rather than time-to-event data were analysed. Three-year survival outcomes from the seventh RCT (the CLASICC trial) have not been published and only very limited information about these results was available.

4.1.4 Five RCTs and the IPD meta-analysis reported disease-free survival. Raw data were available from four RCTs – meta-analysis of these data did not show a statistically significant difference between laparoscopic and open surgery (pooled RR 1.01, 95% CI 0.95 to 1.07). Long-term survival outcomes in the fifth RCT (the CLASICC trial) have not been published and only very limited information about these results was available.

4.1.5 Seven RCTs and the IPD meta-analysis contained relevant information on tumour recurrence. Two of the RCTs reported zero event rates in both surgery groups. In a meta-analysis of the remaining five studies, there was no statistically significant difference between the two types of surgery (pooled RR 0.92, 95% CI 0.74 to 1.14). Eight RCTs contained information on port-site recurrence. There were only three reported events.

4.1.6 Some patients who were originally randomised to undergo laparoscopic surgery were converted intra-operatively to open resection. Eleven RCTs reported conversion rates: the mean overall rate was 20%. Three RCTs recorded separate outcome data for converted patients who appeared to have higher blood loss, require a longer hospital stay and have a greater risk of tumour recurrence than patients who underwent the laparoscopic or open procedure as planned.

4.1.7 Anastomotic leakage was the only outcome for which there were sufficient data to conduct a stratified meta-analysis by location of cancer (that is, to establish differences in clinical effectiveness for cancers of the colon and rectum). The increased risk of anastomotic leakage with laparoscopic resection compared with open resection was similar for colon and rectal cancers (pooled RR for colon cancer 1.27, 95% CI 0.70 to 2.31, four studies; pooled RR for rectal cancer 1.25, 95% CI 0.63 to 2.46, two studies).

4.1.8 Only two RCTs reported subgroup analyses by stage of cancer for overall survival. Both reported that there was no statistically significant difference in overall survival between patients undergoing laparoscopic surgery and those undergoing open surgery for cancer stages I, II or III.

4.1.9 Submissions from manufacturer and professional consultees contended that long-term clinical outcomes between open and laparoscopic colorectal surgery are equivalent, while short-term clinical outcomes favour the laparoscopic approach.

4.2 Cost effectiveness

4.2.1 The Assessment Group conducted a systematic review of economic evaluations published from 2000 to 2005 and performed an independent economic evaluation. The consultees did not submit any formal economic evaluation of the technology. Instead, key issues were identified and highlighted in the submissions.

4.2.2 The Assessment Group identified five relevant primary studies. Two were UK studies: an unpublished draft paper on the short-term economic evaluation of a subset of patients in the CLASICC trial, and a small study in the context of an enhanced recovery programme. When compared with open surgery, the mean cost for laparoscopic surgery was higher in all of the studies except one. There was considerable variation in the reported differences in mean costs of laparoscopic and open surgery in the studies.

4.2.3 The principal arguments used by a manufacturer in its submission were as follows: (a) the conversion rate of laparoscopic to open surgery and the length of hospital stay are the two key drivers of total cost; (b) laparoscopic surgery shortens hospital stay; (c) conversion rates can be lowered to under 10% through appropriate training, mentoring and case selection, and; (d) with the control of conversion rates, the cost of laparoscopic surgery should be similar to or lower than that of open surgery. On the basis of these arguments, the manufacturer concluded that as there is no difference in long-term clinical outcomes between laparoscopic and open surgery, and short-term outcomes favour laparoscopic surgery, laparoscopic surgery should therefore be a cost-effective alternative for patients within the NHS.

4.2.4 The assessment report cautioned that while it is likely that the total cost of laparoscopic surgery decreases as the conversion rate is lowered, direct evidence is limited. In addition, it is not clear how a reduction in conversion rate would affect the cost difference between laparoscopic and open surgery.

4.2.5 The Assessment Group conducted its own economic evaluation using a balance-sheet approach in addition to a modelling approach. Laparoscopic surgery was associated with higher estimated cost than open surgery with an estimated difference of £265 (95% CI –£3829 to £4405). Assuming that the long-term outcomes are equivalent, a judgment is then required as to whether the short-term benefits associated with earlier recovery merit the extra cost of laparoscopic resection. Difference in length of hospital stay was identified as one of the key determinants of this cost difference. Threshold analysis suggested that the cost difference would decrease to zero if laparoscopic surgery decreased the average length of hospital stay by just over 4 days when compared with open surgery. However, this magnitude of difference was rarely observed in any of the studies included in the systematic review. In addition, if the difference in length of stay between the two types of surgery decreases to as little as 1 day (for example, in an enhanced recovery programme), the incremental cost of laparoscopic surgery compared with the open procedure would increase to over £500.

4.2.6 The Assessment Group used a Markov model to estimate the long-term costs and benefits in a hypothetical cohort of 65-year-old patients with colorectal cancer undergoing surgical resection of tumour. Laparoscopic surgery was dominated (that is, it was associated with higher costs but was no more effective) by open surgery in the base–case analysis and in almost all of the sensitivity analyses.

4.2.7 The Assessment Group acknowledged that these results did not capture the quality-of-life benefits that might be associated with earlier recovery, for which little data were available. The Group concluded that, taking £30,000 as a theoretical value for the maximum acceptable cost of an additional quality-adjusted life-year (QALY) and the estimated mean incremental cost for laparoscopic surgery as £263 (base–case analysis) and £290 (equal mortality and disease-free survival), then in order for laparoscopic surgery to be considered cost effective, the QALY gain associated with laparoscopic surgery would have to be 0.009 in the base–case and 0.010 in the case of equal overall and disease-free survival.

4.3 Consideration of the evidence

4.3.1 The Committee noted that more evidence has become available since NICE issued the original guidance (NICE technology appraisal no. 17) in 2000. The Committee reviewed the new data available on the clinical and cost effectiveness of laparoscopic surgery for the treatment of colorectal cancer, having considered evidence on the nature of the condition and the value placed on the benefits of laparoscopic surgery by people with surgically resectable colorectal cancer, those who represent them, and clinical experts. It was also mindful of the need to take account of the effective use of NHS resources.

4.3.2 The Committee considered the evidence that laparoscopic surgery is associated with a longer operating time and a shorter hospital stay. The evidence from RCTs did not show a difference between laparoscopic and open surgery in terms of tumour recurrence, disease-free or overall survival at 3 years. Professional experts at the Appraisal Committee meeting reported that the consensus among clinicians is that there is no difference in long-term outcomes between laparoscopic and open colorectal surgery provided that the laparoscopic procedure is performed by adequately trained surgeons. The Committee was therefore persuaded that laparoscopic colorectal surgery and open colorectal surgery are likely to have similar long-term outcomes with appropriate patient selection and when performed by surgeons with the appropriate experience and skills.

4.3.3 The Committee was also persuaded that there are important differences between the laparoscopic and open approaches regarding both the length of hospital stay for patients and their ability to return to normal activities after the operation. These differences favoured laparoscopic procedures. The Committee considered that although there was little direct evidence of quality-of-life benefits associated with the laparoscopic procedure over the open procedure, it was likely that such benefits exist and are significant in the short term, at least for the first 6 weeks after the operation. On this basis, the Committee concluded that the quality-of-life benefits would be sufficient to make the laparoscopic procedure cost effective and an appropriate use of resources for the NHS providing it was undertaken by surgical teams who are fully trained and experienced in performing the procedure.

4.3.4 The Committee was aware that, on average, 20% of individuals scheduled for laparoscopic surgery were converted to open surgery in clinical trials, and there was some evidence that these individuals had poorer outcomes than those who had laparoscopic or open surgery as planned. The Committee heard from the professional experts that poorer outcomes in converted patients tend to result from the individual's condition, which influences the decision to convert, rather than as a direct result of the conversion itself. The Committee also heard from the professional experts that appropriate patient selection and development of surgical skills through experience would be expected to lower the conversion rate and that for an experienced surgeon, a conversion rate of less than 10% is achievable.

4.3.5 The Committee considered the appropriate training of surgeons and surgical teams to be essential to ensure the clinical effectiveness and safety of the technique as an alternative to open surgery. The Committee therefore concluded that laparoscopic colorectal surgery should be performed only by surgeons who: (a) have completed appropriate training in the technique and; (b) perform the procedure often enough to maintain competence. The Committee considered that these criteria should be determined by the relevant national professional bodies. Cancer networks and constituent Trusts should ensure that any local laparoscopic colorectal surgical practice meets these criteria as part of their clinical governance arrangements. The professional experts informed the Committee that there are many existing training courses in laparoscopic colorectal surgery in the UK, including the preceptorship programme set up by the Association of Laparoscopic Surgeons of Great Britain and Ireland and the Association of Coloproctology of Great Britain and Ireland in 2004.

4.3.6 The Committee was aware of the existence of the National Bowel Cancer Audit Project commissioned by the Healthcare Commission and managed jointly by the National Clinical Audit Support Programme and the Association of Coloproctology of Great Britain and Ireland. The Committee understood that this audit has the potential to be developed to encompass the recommendations in this guidance. The Committee was also persuaded that relevant data collection on a national basis is of paramount importance in closely monitoring the introduction of the laparoscopic procedure.