Evidence on effectiveness

A literature search was carried out for this briefing. The most relevant or best available published evidence relating to the clinical effectiveness of the technology has been summarised below. Further information about how the evidence for this briefing was selected, and full summaries of the included studies, are available on request by contacting medtech@nice.org.uk.

This briefing summarises 6 out of a total of 23 identified studies on Sleepio (table 2). This includes 1 unpublished and 5 published studies, involving 8,097 users of Sleepio in total. The evidence is based on 5 well-designed and well-reported randomised controlled trials and 1 large prospective unpublished audit.

Table 2 Summary of evidence

Espie et al. (2012)

Study size, design and location

Randomised, placebo-controlled trial comprising n=164 adults meeting proposed DSM V criteria for insomnia disorder, UK.

Intervention and comparator(s)

3 arms: digital cognitive behavioural training (n=55, dCBT‑I: Sleepio), imagery relief therapy (n=55: placebo), treatment as usual (n=54).

Primary endpoint: sleep efficiency (total time asleep expressed as a percentage of the total time spent in bed).

Key outcomes

dCBT‑I was associated with sustained improvement at post-treatment and at 8 weeks follow‑up, compared with both treatment as usual and imagery relief therapy.

Strengths and limitations

Placebo-controlled study design.

Random, blind assignment to each group.

The inclusion of healthcare providers in the study design limits the generalisability of the results to the self-referral setting.

This study evaluated the web-based version, not the app.

Patients were recruited by online survey and may represent a cohort with a high interest in addressing sleep problems. They all had access to, and competencies in, using the internet, thus limiting the generalisability of this cohort to the wider population with insomnia.

Bostock et al. (2016)

Study size, design and location

Parallel group, randomised controlled trial comprising n=270 people self-identified as having poor sleep, US.

Intervention and comparator(s)

2 arms: Sleepio (n=135, by website and app) and 'waiting list' control (n=135).

Patients in the waiting list group did not have any intervention or advice.

Key outcomes

Patients having Sleepio showed a statistically significant improvement in Sleep Condition Indicator score compared with the waiting list group (1.66 vs 0.52 respectively). This benefit was maintained at 3‑month follow‑up. Workplace performance was also statistically significantly enhanced.

Strengths and limitations

A formal screening of other disorders of sleep was not done, so it is unknown if patients had sleep breathing or sleep motor problems.

The sleep diary is a staple tool of insomnia assessment. Only the Sleepio group used diaries as part of their therapy programme, so it is not possible to compare these data between the 2 groups.

McGrath et al. (2017)

Study size, design and location

Phase II blinded, randomised controlled trial comprising n=134 community-based patients with self-reported insomnia and hypertension, Ireland.

Intervention and comparator(s)

Intervention: 8 weekly dCBT‑I sessions (n=67) delivered with the Sleepio website in addition to standard care (a vascular risk factor education session, delivered by specialist trained nurses in small groups over 30 minutes during week 1 of the trial).

Comparator: standard care only (n=67).

Key outcomes

After 8 weeks, there were statistically significant differences between groups seen across all 3 sleep measures, and also for the assessments of depression and anxiety.

However, there was no difference between the 2 groups in terms of blood pressure control.

Strengths and limitations

Only the short-term effect of Sleepio on blood pressure was assessed, which limits the generalisability of the results in cases with long-term follow-up. Only 50% of patients completed all 6 online sessions, which may limit the ability to detect a treatment effect. The sleep measures and the assessments of depression and anxiety are secondary outcomes and therefore the study was not adequately powered to detect significant differences.

Barnes et al. (2017)

Study size, design and location

Parallel group, randomised controlled trial comprising n=223 adults with self-reported insomnia. Patients were recruited through the international World Sleep Survey.

Intervention and comparator(s)

2 arms: dCBT‑I (n=117) and waiting list (n=106) control.

Intervention: 6 weekly dCBT‑I were delivered with the Sleepio website.

Comparator: no intervention or advice. Patients in this group completed all major assessments for the trial and were offered Sleepio 10 weeks after the study period.

Key outcomes

After 10 weeks, the Sleepio group had statistically significant improvements in sleep quality, mood, job satisfaction and self-control. Sleep efficiency was improved by 27% in both groups.

Strengths and limitations

The use of an experimental design with a waiting list control group aids the degree to which causal inferences can be made.

No placebo intervention.

The outcome variables were all self-reported. Self-reports may include distortions that lower the accuracy of this method of data collection. However, the analyses were within individuals. Thus, distortions such as response biases that occur at the person-level of analysis are already statistically controlled for.

Patients were recruited by online survey and may represent a cohort with a high interest in addressing sleep problems. They all had access to, and competencies in, using the internet, thus limiting the generalisability of this cohort to the wider population with insomnia.

Luik et al. (2017)

Submitted for peer review but unpublished at the time of writing.

Study size, design and location

Prospective audit comprising n=3,551 Sleepio users who had graduated from the programme and had completed a post-intervention assessment. No information on the location of the study was provided.

Intervention and comparator(s)

Intervention: 6 weekly dCBT‑I sessions delivered with the Sleepio website.

No comparator.

Key outcomes

Sleep quality significantly improved following Sleepio (p<0.001), as did mental health symptoms, stress, life satisfaction and work productivity.

Those who use a wearable device have similar therapy effects to those who do not, although those who do connect a wearable device tend to interact with the programme more.

Strengths and limitations

There was no comparator for this study. Based on the Sleep Condition Indicator, similar effectiveness was observed for Sleepio in this real-world practice as in randomised controlled trials. The sample of users had completed a post-therapy survey, which might create a selection bias.

Freeman et al. (2017)

Study size, design and location

Single-blind, randomised controlled trial comprising n=3,755 university students with self-reported insomnia, UK.

Intervention and comparator(s)

2 arms: dCBT‑I (n=1,891) and control (n=1,864).

Intervention: 6 weekly dCBT‑I were delivered with the Sleepio website.

Control: usual practice; the authors describe this as either no intervention or medication prescription for a small proportion of the patients. Patients were offered Sleepio after their final assessment.

Key outcomes

Sleepio was associated with significant reductions, at all time points (3, 10 and 22 weeks), in insomnia, paranoia and hallucinations compared with the control group (all p<0.0001). Treatment effects were greater for patients who completed the whole programme.

Mediation analysis revealed that early changes in sleep explain approximately half of the total sleep-mediated changes in psychotic experiences by the end of treatment, supporting the study's hypothesis that insomnia is a contributory cause of psychotic experiences.

The dropout from the study assessments was high (50%), and was greater in the treatment group than in the control group.

Strengths and limitations

The dropout rate was high, leading to a high number of missing data which will impact data analyses. However, the treatment differences were robust to 3 types of sensitivity analyses for missing data.

The control group was essentially no treatment because only 3% of the population had medication and 7% psychological treatment.

The outcome variables were all self-reported. Self-reports may include distortions that lower the accuracy of this method of data collection.

Patients were predominately in the non-clinical range of psychotic experiences, restricting the conclusions to the less severe cases. The generalisability of the results might be further restricted by the fact that the patients were self-reported for inclusion into the trial.

Abbreviation: dCBT‑I, digital cognitive behavioural therapy for insomnia.

Overall assessment of the evidence

  • The size of the treatment effect varies between studies. Espie et al. (2012) reported a 20% improvement in sleep efficiency (SE) from baseline after Sleepio in patients who met the DSM‑V criteria, whereas Bostock et al. (2016) and McGrath et al. (2017) reported an increase in SE of only 10%. In Freeman et al. (2017), Sleepio was associated with significant reductions at all time points (3, 10 and 22 weeks) in insomnia, paranoia and hallucinations compared with the control group (all p<0.0001).

  • The baseline SE varied between studies. Patients in Espie et al. (2012) had close to 60% SE at baseline, whereas Bostock et al. (2016) and McGrath et al. (2016) reported baseline SE rates of 76% and 81% respectively. The Barnes et al. (2017) randomised controlled trial reported a higher increase in SE (27%) in both the Sleepio and control groups, but this study also reported a lower baseline SE of approximately 69%. Normal SE is around 85% to 90%. Therefore the different size effects in the different population settings could be attributed to SE reaching the normal limit with a smaller improvement in the less severely affected populations.

  • All 6 included studies list employees of the developer among the study authors, and so the studies may be subject to bias.

  • None of the available studies compared Sleepio with face-to-face cognitive behavioural therapy for insomnia (CBT‑I).

  • None of the studies indicates which Sleepio version was used. The developer states that there have been no substantive changes to the CBT‑I content since launch, but it is not known how any other changes may affect the generalisability of the evidence to the current version.

  • Although the 4 randomised controlled trials (Espie et al. 2012, Barnes et al. 2016, Bostock et al. 2016, McGrath et al. 2016) are relatively small compared with the potential reach of Sleepio, all except for McGrath et al. are adequately powered, well-designed and reported. The randomised controlled trial by Freeman et al. had to revise the sample size calculation because the dropout rate was more than expected.

  • Sample sizes are similar to those in other randomised controlled trials done on digital CBT‑I technologies (Seyffert et al. 2016). Freeman et al. (2017) represents the largest randomised controlled trial of a psychological intervention for a mental health problem.

  • Three of the studies were done in the UK; it is not clear if the results of the others are generalisable to an NHS setting.

  • Although evidence on reduced prescribing of hypnotics with Sleepio has not yet been published, the developer has presented unpublished data from a case study on Sleepio's use in a large US company involving approximately 17,500 employees.

  • The reported outcomes on sleep improvement, psychological wellbeing, improved labour market participation and productivity, and reduced prescribing of hypnotics are all relevant to the NHS care pathway.

Recently completed and ongoing studies

Three recent, ongoing or in‑development trials on the use of Sleepio for people with insomnia were identified in the preparation of this briefing.

  • Digital insomnia therapy to assist your life as well as your sleep (ISRCTN60530898). Recruitment completed.

  • Efficacy of a sleep quality intervention in people with low back pain: protocol for a feasibility randomized co‑twin controlled trial (ACTRN12615000672550). According to the developer, the anticipated completion date is 2017.

  • Sleep treatment outcome predictors (STOP; NCT03062891). Currently recruiting patients by invitation.

  • A sleep program to improve sleep quality in people with HIV (NCT02571595). Currently recruiting patients.

  • Sleep to prevent evolving affective disorders (SPREAD; NCT02988375). Currently recruiting patients.

  • Research on expecting moms and sleep therapy (REST; NCT02805998). Currently recruiting patients.

  • Defining the impact of improved sleep on cognitive function (ISRCTN89237370). Recruitment completed.