The assessment

The technology

The technology evaluated in this report is Space from Depression (SilverCloud), an online programme used to treat depression, delivered with support and guidance from a therapist, as an alternative to face-to-face cognitive behavioural therapy (CBT).

Space from Depression was assessed by NICE as part of NHS England's assessment of digitally enabled psychological therapies for Improving Access to Psychological Therapies IAPT. In 2018, an IAPT assessment briefing was published on Space from Depression for treating depression in adults. This reported the results of assessments of therapeutic content, digital technological factors, clinical evidence and resource impact. The NICE IAPT expert panel considered these assessments and recommended that Space from Depression should progress to the evaluation in practice phase of this programme.

Evaluation data

The evaluation in practice was designed to gather real-world activity data to help understand how well Space from Depression worked in routine use in IAPT services, in terms of clinical outcomes and resource use.

The evaluation took place at 3 IAPT services in Lincolnshire, Warrington and Haringey (see table 1). In total, 526 people finished a course of digital treatment with Space from Depression and 6,440 people completed a course of non-digital treatment. People were considered to have completed a course of treatment if the therapist recorded 2 treatment sessions with them.

Table 1 Number of people with depression in each IAPT service who finished a course of non-digital treatment or Space from Depression

IAPT service

Number of people who finished a course of non-digital treatment

Number of people who finished a course of digital treatment with Space for Depression

Lincolnshire Steps 2 Change

3,162

187

Warrington Talking Matters

817

224

Haringey Let's Talk IAPT

2,461

115

Total

6,440

526

SilverCloud gave training to all therapists using Space from Depression within the evaluation. Therapists were asked to follow the recommended Space from Depression treatment protocol.

Therapists were asked to offer Space from Depression to all people with depression with a PHQ-9 score less than 20, unless they had learning difficulties or their level of English language was not high enough to read the course material and feedback from therapists, or complete written homework.

Parameters

The evaluation data considered by the panel was:

  • starting and finishing PHQ-9 scores (with effect sizes)

  • recovery rates

  • reliable improvement

  • reliable deterioration

  • service user feedback

  • user demographics

  • therapist feedback

  • summarised responses from IAPT service exit interviews

  • figures showing the average number of appointments and total therapist time taken.

Comparators for the outcome data were within-service data for people having standard care options for depression, and national outcome data for people having guided self-help using books, and all treatments, from the IAPT annual report.

Also, a summary of the economic evaluation from an academic-in-confidence manuscript of a new randomised controlled trial on Space from Depression was considered.

Key outcomes from the evaluation in practice are summarised below. A full description of the evaluation and a report of the data and feedback generated through this evaluation is available on request from the IAPT project team.

Change in PHQ-9 values

Depression severity was measured using the PHQ-9 outcome measure (see table 2). GAD-7 values were also collected, to measure severity of anxiety symptoms. To compare, the national average starting and finishing PHQ-9 values are shown for people having guided self-help (using books) in IAPT services, and for people having any treatment in IAPT services.

Across the 3 services, the change in PHQ-9 values was -3.7 for people who had Space for Depression, compared with -6.6 for people who had standard care. For comparison, the national change in PHQ-9 values for people having guided self-help (using books) was -5.0, and people having any treatment was -7.1.

Table 2 PHQ-9 values before and after treatment across 3 IAPT services

Type of treatment

Lincolnshire PHQ-9 (SD)

Warrington PHQ-9 (SD)

Haringey PHQ-9 (SD)

Average of 3 services PHQ-9 (SD)

Standard care: before treatment

15.5 (3.3)

15.7 (3.3)

16 (5.3)

15.7 (4.2)

Standard care: after treatment

9.2 (6.2)

10 (6.5)

8.6 (6.3)

9.1 (6.3)

Standard care: change before and after

-6.3

-5.7

-8.2

-6.6

Space from Depression: before treatment

14.7 (3.4)

12.5 (4.3)

13.1 (3.3)

13.4 (3.9)

Space from Depression: after treatment

11.2 (5.9)

9.4 (5.2)

7.7 (4.7)

9.7 (5.5)

Space from Depression: change before and after

-3.6

-3.1

-5.4

-3.7

Table abbreviations: SD, standard deviation.

Recovery rates

Recovery was calculated as the percentage of people whose problem descriptor (a description of a person's mental health condition used by IAPT services) was depression and who had both:

  • a PHQ-9 score above 9 and/or a GAD-7 score above 7 before treatment

  • a PHQ-9 score below 9 and a GAD-7 score below 7 by the end of treatment.

Across the 3 services, the average rate of recovery was 45.9% for people who had Space for Depression and 55.1% for people who had standard care. For comparison, the national recovery rates for people with depression having guided self-help (using books) was 40.8% and people with depression having any treatment was 52.1% (see table 3).

Table 3 Recovery rates for people treated with Space from Depression or standard care in 3 IAPT services

Type of treatment

Lincolnshire

Warrington

Haringey

Average of 3 services

Guided self-help

Any treatment

Standard care

55.0%

51.4%

58.9%

55.1%

40.8%

52.1%

Space from Depression

38.5%

34.1%

65.1%

45.9%

Reliable improvement rates

Reliable improvement (see table 4) was calculated using PHQ-9 and GAD-7 scores. It was measured as the percentage of people who had completed a course of treatment and had either a 'reliable improvement' in both their PHQ-9 and GAD-7 scores, or had a 'reliable improvement' in one of these measures and no 'reliable deterioration' in the other measure. These were calculated like so:

  • on the PHQ-9 scale, a reliable improvement is a decrease of 6 or more points, and a reliable deterioration is an increase of 6 or more points

  • on the GAD-7 scale, a reliable improvement is a decrease of 4 or more points and a reliable deterioration is a decrease of 4 or more points.

Across the 3 services, the average rate of reliable improvement was 46.9% for people who had Space for Depression and 68.2% for people who had standard care. For comparison, the national reliable improvement rates for people having guided self-help (using books) was 55.6% and people having any treatment was 67.4%.

Table 4 Reliable improvement rate for people treated with Space from Depression or standard care in 3 IAPT services

Type of treatment

Lincolnshire

Warrington

Haringey

Average of 3 services

Guided self-help

Any treatment

Standard care

67.6%

65.0%

72.0%

68.2%

55.6%

67.4%

Space from Depression

47.6%

34.8%

58.3%

46.9%

Reliable deterioration rates

Reliable deterioration was calculated as the percentage of people who had completed a course of treatment and had either a 'reliable deterioration' in both their PHQ-9 and GAD-7 scores, or had a 'reliable deterioration' in one of these measures and no 'reliable improvement' in the other measure.

Across the 3 services, the average rate of reliable deterioration was 8.5% for people who had Space for Depression and 7.0% for people who had standard care. For comparison, the national reliable deterioration rates for people having guided self-help (using books) was 7.9% and people having any treatment was 5.8%.

Table 5 Reliable deterioration rate for people treated with Space from Depression or standard care in 3 IAPT services

Type of treatment

Lincolnshire

Warrington

Haringey

Average of 3 services

Guided self-help

Any treatment

Standard care

7.7%

8.0%

5.2%

7.0%

7.9%

5.8%

Space from Depression

8.6%

10.7%

6.3%

8.5%

Feedback from service users and therapists

Service users

Service user feedback was gathered from the 5 standard questions asked of all service users in the IAPT patient evaluation questionnaire (PEQ5) with 5 additional questions added for people using digital therapies. Service user responses to the multiple-choice questionnaire and free-text responses were discussed. Service user feedback on Space from Depression from 134 people (27%) across the 3 sites included very positive responses from some people and very negative responses from others. It showed that some users were dissatisfied with the low level of therapist contact and felt isolated and unsupported. Others felt that this care model was ideal for them and that it offered flexible therapy and adequate support. Two-thirds of responders (66.3%) reported being completely or mostly satisfied with their online treatment. Fifty-one responders gave free-text comments, 24 of these gave positive feedback and 27 gave negative feedback.

Therapists

Twenty-one out of 26 therapists taking part in the evaluation gave feedback. Seventeen of these gave largely positive feedback. Twenty responders said that they totally or mostly agreed that Space from Depression was easy to use. Eighteen totally or mostly agreed that they were happy to offer Space from Depression to their patients. Negative responses showed some concern about the training given by the company, the ability for supervisors to review therapists' work, and being unable to see how long the patients had spent using the programme.

IAPT services

IAPT service exit interviews helped to show what effect Space from Depression had on services, and services' perspectives on the evaluation. Services reported that Space from Depression was well liked by therapists who enjoyed using it. Space from Depression gave some variety to the therapists' workload and enabled homeworking, which helped with retention of staff. Concerns were raised around the level of communication between therapist and user in the Space from Depression protocol. Services expressed that they would like to use occasional telephone contact alongside the online feedback, as this helped users to stay engaged with the programme. There was some frustration that therapists could only view messages from users on the day of fixed review appointments. These appointments were usually set to be weekly, but their frequency could be changed by the therapist. Therapists felt that the capacity for ad-hoc messaging would be beneficial, although this is not needed for all service users. SilverCloud has stated that increasing the level of communication between therapist and user could increase the therapists' workloads, and would not align with face-to-face therapy, where there is not normally communication between appointments.

Resource impact

Number of appointments

The evaluation recorded the mean number of appointments in a course of treatment using all standard care options and Space from Depression.

Table 6 Mean number of appointments per course of treatment with standard care or Space from Depression

Type of treatment

Lincolnshire

Warrington

Haringey

Standard care (SD)

7.3 (4.8)

6.4 (4.5)

8.8 (5.4)

Space from Depression (SD)

7.1 (3.1)

4.7 (1.6)

5.2 (1.8)

Table abbreviations: SD, standard deviation.

Total therapist time taken

The total amount of time taken for a course of treatment was recorded. The standard care figures included a 45-minute assessment appointment, which was not included in the figures for Space from Depression. Table 7 shows the therapist time figures adjusted to remove the 45-minute assessment appointment from the standard care figures.

Table 7 Adjusted mean therapist time taken per course of treatment in minutes

Type of treatment

Lincolnshire

Warrington

Haringey

Average of 3 services

Standard care

222.1

154.4

312.2

229.6

Space from Depression (SD)

132.6 (68.8)

50.5 (30.5)

112.0 (57.9)

92.8 (64.7)

Table abbreviations: SD, standard deviation.

The adjusted therapist time values were used to recalculate the cost per treatment of Space from Depression. An average IAPT service would pay around £24 per licence (at 0% VAT), which gives 1 user access to the programme for 1 year. In addition to the licence cost, the cost of 92.8 minutes of therapist time (band 5 psychological wellbeing practitioner) was calculated to be £54. This cost was compared with the expected costs for other standard care options (see table 8).

Table 8 Cost of Space from Depression compared with standard care

Type of treatment

Cost of standard care

Cost of using Space from Depression

Difference in cost

Guided self-help (3 to 6 sessions)

£59

£54

£5 saved

Group CBT

£97

£54

£43 saved

Workshop CBT

£8

£54

£46 cost

Face-to-face, individual CBT

£560

£54

£506 saved

Antidepressant medication for 6 months (weighted average cost based on minimum daily dose)

£110

£54

£56 saved

Table abbreviations: CBT, cognitive behavioural therapy.