4 Evidence and interpretation

The Appraisal Committee considered evidence from a number of sources (see Appendix B).

4.1 Clinical effectiveness

4.1.1 The evidence presented in the Assessment Report was primarily drawn from a recent Cochrane Review of ECT in schizophrenia and a systematic review commissioned by the Department of Health on the use of ECT in schizophrenia, depressive illness and mania. Both reviews are of high quality and consider a total of 119 randomised controlled trials (RCTs) and a number of observational studies. Evidence submitted by patient and professional groups was also considered.

4.1.2 There were problems with the design of many of the RCTs. A large proportion were conducted before the introduction of modern techniques of administering ECT, and therefore do not reflect current practice. There were large variations in the parameters of the ECT administered that included the machine used, the number of sessions, the dosage and wave form, electrode placement, and the type and dosage of concomitant therapy. A number of studies used fixed dosage rather than individual thresholds. There was little evidence to support the routine prescription of a set number of treatment sessions per course of ECT or of the value of continuation (maintenance) ECT. The validity of many of the measurement scales used in the studies to measure outcome has not been clearly established and no study adequately captured either users' views or quality of life.

4.1.3 The Assessment Report reviews data from 90 RCTs in individuals with depressive illness, of different grades of clinical severity, who were referred for ECT. Overall, these RCTs provide evidence that real ECT (that is, where an electric current was applied) is more effective than sham ECT (where no electric current was applied) in the short term. The data provide evidence that the stimulus parameters have an important influence on efficacy; at the end of a course of treatment, bilateral ECT was reported to be more effective than unilateral ECT. Raising the electrical stimulus above the individual's seizure threshold was found to increase the efficacy of unilateral ECT at the expense of increased cognitive impairment. In trials comparing ECT with pharmacotherapy, ECT had greater benefit than the use of certain antidepressants but the trials were of variable quality and inadequate doses and durations of drug therapy were frequently used. The combination of ECT with pharmacotherapy was not shown to be superior to ECT alone, although the duration of the RCTs was insufficient to show whether pharmacotherapy was beneficial. Compared with placebo, continuation pharmacotherapy with tricyclic antidepressants and/or lithium reduced the rate of relapses in people who had responded to ECT. Preliminary studies indicate that ECT is more effective than repetitive transcranial magnetic stimulation.

4.1.4 Evidence from 25 RCTs suggests that ECT may be effective in acute episodes of certain types of schizophrenia and reduce the occurrence of relapses, although the results are not conclusive and the design of many of the studies did not reflect current practice. The data on differing efficacy related to electrode placement and stimulus parameters are equivocal and firm conclusions could not be drawn. No RCT investigated the use of ECT in comparison with atypical antipsychotics, and the studies that included individuals with treatment-resistant schizophrenia did not consider the use of clozapine. The combined weight of evidence suggests that ECT is not more effective, and may be less effective, than antipsychotic medication. The combination of ECT and pharmacotherapy may be more effective than pharmacotherapy alone, but the evidence is not conclusive.

4.1.5 The four RCTs reviewed in the Assessment Report suggest that ECT may be of benefit in the rapid control of mania and catatonia and this suggestion is supported by evidence from a number of observational studies and testimony from clinical experts. However, the evidence on which to base any general conclusions about the effectiveness of ECT or to determine the most appropriate therapeutic strategy is weak.

4.1.6 There is clear evidence that cognitive impairment occurs both immediately after administration of ECT and following a course of therapy, and this may cause considerable distress to those affected. The impairment is greater in individuals who have had electrodes applied bilaterally than in those who have had them placed unilaterally, and unilateral placement to the dominant hemisphere causes more impairment than placement to the non-dominant hemisphere. A reduction in the risk of cognitive impairment is, however, mirrored by a reduction in efficacy. There is some limited evidence from RCTs to suggest that the effects on cognitive function may not last beyond 6 months, but this has been inadequately researched. There is also evidence to suggest that the impairment of cognitive function associated with ECT differs between individuals and that it is linked to the dose administered, although the relationship with the seizure threshold has not been adequately defined. There is no evidence to suggest that the effect of ECT on cognitive function differs between diagnoses.

4.1.7 In addition to testimony from user groups, a systematic review of evidence from non-randomised studies relating to patients' accounts and experiences of ECT was also considered. This provided important evidence on the experiences of individuals receiving ECT, particularly cognitive impairment and its impact, and the validity of neuropsychological instruments used in clinical trials. There was evidence that the measurement scales used in RCTs do not adequately capture the nature and extent of cognitive impairment, and qualitative studies have indicated that the impairment may be prolonged or permanent. Additionally, there was testimony that individuals are not provided with sufficient information on which to base a decision regarding consent. Also, some individuals are unaware of their rights to refuse treatment, or may feel unable to do so because of the perceived threat of detainment under the Mental Health Act.

4.1.8 There was no evidence to suggest that the mortality associated with ECT is greater than that associated with minor procedures involving general anaesthetics, and there were limited data on mortality extending beyond the trial periods. The six reviewed studies that used brain-scanning techniques did not provide any evidence that ECT causes brain damage. While there is no evidence to suggest that benefits and safety are age-dependent, there are no studies on the impact of ECT on the developing brain. Furthermore, it is likely that co-morbidities could increase the risk of harm. The use of ECT during pregnancy is known to cause complications, but the risks associated with ECT need to be balanced against the risks of using alternative (drug) treatments.

4.2 Cost effectiveness

4.2.1 There are no published economic studies relating to ECT, and none of the submissions from consultees contained any economic analyses. The Assessment Group therefore constructed economic models of ECT for depressive illness and schizophrenia based on a review of published evidence. They were not able to construct robust models for mania and catatonia because of the low volume of data in these areas.

4.2.2 The depressive illness model had a 1-year time horizon and compared the cost effectiveness of inpatient ECT with other inpatient treatments for adults with severe depressive illness. The key comparators were different classes of antidepressants, with lithium given in addition for third-line therapy. After three drug treatment strategies, non-responders were assumed to receive 8 weeks of psychotherapy and to make a moderate improvement.

4.2.3 The results of the depressive illness model showed that, for eight different scenarios, total costs range from £10,592 to £15,354, and total quality-adjusted life years (QALYs) range from 0.424 to 0.539. Given the small differences in total costs and QALYs between the strategies that included ECT and the one that did not, and the uncertainty in the data available, ECT and pharmacotherapy are likely to be equally cost effective.

4.2.4 The schizophrenia model also had a 1-year time horizon and compared the cost effectiveness of ECT in combination with a typical antipsychotic with that of (a) clozapine, and (b) chlorpromazine or haloperidol for adults hospitalised with treatment-resistant schizophrenia of moderate symptomatology.

4.2.5 The results of the schizophrenia model suggest that ECT is dominated by clozapine – that is, ECT is associated with fewer QALYs (0.842 vs 0.863) at a higher cost (£55,267 vs £34,787). For individuals who do not respond to clozapine, ECT dominates chlorpromazine/haloperidol, resulting in more QALYs (0.842 vs 0.820) at a lower cost (£55,267 vs £58,265). However, these results do not take into account the degree of uncertainty in the estimates of both cost and effectiveness.

4.2.6 To summarise, there is no published evidence regarding the cost effectiveness of ECT. The modelling exercises undertaken by the Assessment Group, while fairly crude and based on a number of uncertain assumptions, suggest that – for those with severe depressive illness and treatment-resistant schizophrenia – ECT and pharmacological treatment may be equally cost effective, with no consistent differences in costs or outcomes.

4.3 Consideration of the evidence

4.3.1 The Committee reviewed the evidence on both the clinical effectiveness and the cost effectiveness of ECT. It considered written and verbal evidence on the nature of the conditions and the experience of people who have received or may be eligible for ECT, those who represent them, and clinical experts. It was also mindful of the need to ensure that its advice took account of the efficient use of NHS resources.

4.3.2 The evidence submitted to the Committee, both written and verbal, demonstrated that, on balance, current opinion is that ECT is an effective treatment for certain subgroups of individuals with mental disorders. However, opinion varies from those who consider that its adverse effects are tolerable to those who consider that it is associated with unacceptable side effects including brain damage, severe confusion and considerable cognitive impairment in both the short and longer terms. While some individuals considered ECT to be a beneficial and lifesaving treatment, others reported feelings of terror, shame and distress, and found it positively harmful and an abusive invasion of personal autonomy, especially when it was administered without their consent.

4.3.3 In consideration of these extremes of opinion, the Committee concluded that the wishes of the patient must be of paramount importance and that it is essential that all attempts should be made to obtain valid and informed consent, following recognised guidelines. The Committee felt strongly that consent should never be obtained by coercion – either explicit or implicit – through threat of compulsory treatment under the Mental Health Act, and mechanisms to monitor and prevent this from occurring should be developed and implemented, in consultation with appropriate professional and user organisations.

4.3.4 Testimony was heard that the information currently given to individuals does not always adequately inform consent, and the Committee discussed the need for nationally agreed evidence-based patient information leaflets. These should be accessible to a wide range of service users (see Section 7) and should emphasise the right of the individual to withhold consent or to withdraw it at any point.

4.3.5 While the limitations of advance directives were appreciated (see Section 3.7), the Committee believed that, whenever possible, they should be developed and documented in individuals' care programmes and be taken into account when considering ECT.

4.3.6 The Committee considered that, on the evidence put before it, the short-term effectiveness of ECT in individuals with severe depressive illness has been demonstrated. There is less robust RCT evidence to suggest that it is effective in the acute treatment of catatonia and mania. However, the Committee considered that the data appraised, taken in conjunction with the strength of clinical opinion and the experiences of users, provided a sufficient basis on which to recommend the use of ECT in restricted circumstances when the alternative treatment options have proven ineffective. The evidence for the effectiveness of ECT in schizophrenia in general was not conclusive and therefore ECT is not recommended in this population. Further research is required to establish clearly the benefits in subgroups of individuals with schizophrenia, for example those with severe symptoms of depressive illness or catatonia.

4.3.7 The Committee considered that there was no conclusive evidence to support the effectiveness of ECT beyond the short term or that it is more beneficial as a maintenance therapy in depressive illness than currently available pharmacological alternatives. It was particularly concerned that the value of ECT maintenance therapy remained unproven in the context of the lack of information on whether the adverse effects of ECT (for example, on cognitive function) may be cumulative with repeated administration.

4.3.8 In appreciation of the special circumstances in which ECT is administered and the recognition that RCTs cannot adequately capture the long-term effects of ECT, the Committee took special note of the evidence from observations of users' experiences relating to the adverse effects of ECT. In particular, the incidence, extent and timescale of cognitive impairment following ECT was discussed in detail. It was apparent that the nature of cognitive impairment experienced by users was variable and often long lasting to such a degree that it outweighed their perception of any benefit from ECT treatment. The Committee considered that further research, both qualitative and quantitative, was needed to define the effect of ECT on cognitive impairment, especially whether the effects are cumulative with repeated administrations. It was also concerned that the potential for cognitive impairment following ECT may not be highlighted during the consent process. These factors featured significantly in the Committee's deliberations, and specifically in its decision to restrict the use of ECT to situations in which all other alternatives had been exhausted or where the nature of the mental illness was considered to be 'life-threatening'.

4.3.9 The Committee noted that the efficacy and adverse effects of ECT are clearly linked to the method of delivery, although the optimum technique and stimulus parameters have not been adequately researched; for example, gains in efficacy as a result of modifications to electrode placement and stimulus parameters are achieved at the expense of an increased risk of cognitive side effects. The Committee therefore considered that the evidence was not sufficient to allow conclusions to be drawn.

4.3.10 The RCT evidence considered by the Committee also leaves unanswered a number of important questions, and these require further research (see Section 5). Consideration was given to the fact that the majority of the RCTs are not applicable to modern practice because of advances in pharmacological management and ECT administration techniques. The outcomes considered in the RCTs also did not adequately capture the experience of service users, and the validity of many of the scales used to measure outcome had not been clearly established. There was insufficient information to allow appropriate risk–benefit assessment for certain groups of individuals, for example during pregnancy, in older individuals, and in children and young people. Of particular concern were the lack of research into the number of treatment sessions that should be given, and the lack of long-term evidence regarding adverse effects on cognitive function and mortality. The Committee could not establish, in the context of the use of appropriate pharmacological treatment, the value of 'maintenance' ECT therapy following its use to achieve rapid and short-term improvement of severe symptoms. The Committee was persuaded on the balance of the evidence received from patients and carers that the practice of 'continuation' of ECT therapy for short periods following the initial control of severe symptoms was only acceptable in the context of Sections 1.6 and 1.7 of the guidance.

4.3.11 The ongoing deficiencies in current practice were highlighted to the Committee, and the Committee strongly believed that action is required to ensure that appropriate standards of care are enforced whenever ECT is undertaken and that outcomes are continuously monitored. The Committee considered that ECT should be administered only in a suitably equipped unit by professionals who have been trained in its delivery and in the anaesthetic techniques required for the administration of ECT. These professionals should maintain an appropriate level of skill, both through the regular clinical practice of ECT and through undertaking appropriate continuing professional development. Urgent consideration should be given to the establishment of units dedicated to ECT, and of audit networks, which have been shown to be successful in Scotland.

4.3.12 Despite the uncertainty in the estimates of clinical effectiveness and the small differences in costs and outcomes generated in the economic models, the Committee considered that ECT is likely to be cost effective in appropriate patient groups.

4.3.13 In summary, the Committee considered that the evidence appraised supported the effectiveness of ECT in certain groups of individuals. However, the Committee recognised there remained a number of uncertainties, including a lack of information on longer-term outcomes. The Committee was aware of the negative experiences of some individuals who have undergone ECT. Therefore the Committee considered that that ECT should be used with caution and only in the restricted circumstances recommended in the guidance in Section 1. It is anticipated that NICE guidelines currently under development (see Section 8) will further define the place of ECT in the care pathways for individuals with depressive illness.