Appendix C: The evidence

This appendix lists evidence statements provided by three reviews and links them to the relevant recommendations (see appendix B for the key to study types and quality assessments). The evidence statements are presented here without references – these can be found in the full review (see appendix E for details).It also sets out a brief summary of findings from the economic appraisal and fieldwork report.

The three reviews of effectiveness are:

  • Review 1: 'Systematic review of the effectiveness of interventions to promote mental wellbeing in children in primary education. Report 1: universal approaches (non-violence related outcomes)'.

  • Review 2: 'Mental wellbeing of children in primary education (targeted/indicated activities)'.

  • Review 3: 'Systematic review of the effectiveness of interventions to promote mental wellbeing in primary schools: universal approaches with a focus on prevention of violence and bullying'.

Evidence statement number UES1 indicates that the linked statement is numbered 1 in review 1; evidence statement TES1 indicates that the linked statement is numbered 1 in review 2; and evidence statement VPES1 indicates that the linked statement is numbered 1 in review 3.

The reviews and economic appraisal are available on the NICE website. Where a recommendation is not directly taken from the evidence statements, but is inferred from the evidence, this is indicated by IDE (inference derived from the evidence) below.

Recommendation 1: Evidence statement VPES2, IDE

Recommendation 2: Evidence statements UES1, VPES1

Recommendation 3: Evidence statements TES1, TES2, TES5

Evidence statements

Evidence statement VPES1

There is evidence from three out of four 'moderate' quality RCTs and two out of two good quality controlled trials (CTs) that multi-component programmes comprising teacher training in management of behaviour, parenting education and a social skills development curriculum are effective in improving outcomes relevant to bullying, violence and mental health.

Two of these studies have reported positive long-term outcomes (RCT [+]) reporting on arrests at 3 years post intervention and reporting violent delinquent acts and school misbehaviour at 18 years.

Examples of this type of multi-component programme include: the Linking Interests of Families and Teachers (LIFT) programme, the Seattle Social Development Project and the Resolving Conflict Creatively programme.

Evidence statement VPES2

There is evidence from a 'good quality' RCT and a 'moderate' quality RCT indicating that the Peace Builders programme is effective in improving outcomes related to violence and mental health (as measured by teacher report on social competence and aggressive behaviour and visits to the school nurse for injury). The main focus of the Peace Builders programme is on change to the school ethos and environment. This aims to incorporate positive social values and ways of behaving among children and staff into every aspect of school life. The programme also includes peer mentoring, parent engagement behaviour management and a small classroom component. While no long-term studies are available, effects have been demonstrated at 2 years post-implementation of the intervention (as measured by teacher report on social competence and aggression).

Evidence statement UES1

There is good evidence (five randomised controlled trials of high quality [++]) to support the implementation of multi-component programmes, which include significant teacher training and development and support for parenting. Most of these programmes have been researched and developed in the US and may need adapting for the UK. Interventions with similar characteristics are available in the UK but have not been the subject of robust trials. While the majority of these programmes were implemented over a year or more, further research is needed to establish the optimum content and length as well as the appropriate level of teacher training and support and support for parenting.

Evidence statement TES1

Cognitive behavioural therapy (CBT) based programmes targeted at reducing anxiety disorders have been transferred successfully between countries, indicating a high degree of generalisability or applicability.

Two studies (both quality rated 1 [++]) show that brief (10 weeks and 9 weeks) targeted interventions aimed at reducing anxiety or preventing the development of symptoms into full blown disorders appear to be successful in groups of children showing the precursor symptoms associated with anxiety disorders. One study (quality rated 1 [++]) was able to demonstrate that when parent training is combined with child group CBT there are additional benefits for children.

Two studies (quality rated 1 [++]) of indicated interventions aimed at children of divorce and children who are anxious school refusers show sustained benefit for children from CBT-based skills training.

Evidence statement TES2

All studies examined use CBT-based approaches. One study (quality score 1 [+]), the Penn Prevention Programme, showed that it may be possible to relieve and prevent depressive symptoms using a targeted school-based approach where a traditional cognitive behaviour component was allied with a social problem-solving component.

Evidence from other treatment programmes with children with mild to moderate depressive symptoms is mixed. Co-morbid conditions with depression (often conduct or hyperkinetic disorders) make intervention delivery difficult and can confound treatment effects.

One study (quality rating 1 [+]) assessed the effectiveness of an 8 week programme comprising small group-based cognitive-behavioural sessions (entailing role play, games, video and homework activities) in producing improvements in depression scores in children scoring high on the 'Children's depression inventory'. Children receiving the intervention were significantly more likely to have reduced levels of depressive symptoms immediately post-intervention and at 9 months follow-up, compared with children receiving the no-treatment control.

One study (1 [+]) found that social competence training (1 hour sessions for 8 weeks) for children (aged 7–11 years) who were within the 'clinical depression range' of the 'Children's depression inventory', did not significantly improve depression scores at 2 months follow-up, compared with either an attention placebo or no treatment control. Interventions directed at indicated subgroups show some degree of success (two 1 [+]). One study (quality rating 1 [+]) of young people exposed to violence and showing clinical symptoms of post-traumatic stress disorder (PTSD) showed reasonable effect sizes. The programme involved a high proportion of black and minority ethnic children and also used trained school personnel to deliver part of the programme.

Evidence Statement TES5

Multi-component interventions designed for targeted groups of children suffering from conduct disorders show that improved social problem-solving and the development of positive peer relations are among the outcomes with the strongest programme effects. Two studies (both rated 1 [++]) showed improved academic achievement as significant outcomes of intervention.

Timing may be critical. Complex longitudinal multi-component studies like that undertaken by the Metropolitan Area Child Study Research Group (quality rating 1 [++]) support the case for early intervention with aggressive disruptive children, but also attest to the improved benefits of giving a booster intervention towards the end of primary education. Significant 'school effects' were found in the study. Better understanding of school effects, including impediments and resources, is called for.

Recruitment and retention into parent programmes is clearly a major challenge, even when incentives (for example, childcare and transport costs) are offered. Given a choice, evidence from one study (quality rating 1 [++]) indicates that parents may prefer targeted children to receive the intervention at school rather than at home.

Some adverse effects are reported by Metropolitan Area Child Study Research Group (quality rating 1 [++]) as a consequence of bringing aggressive hostile children together in small groups only in later elementary stages, with such groups setting up negative norms of aggressive behaviour.

Cost-effectiveness evidence

Overall, the economic modelling demonstrated that universal interventions to promote mental health in primary schools do lead to short-term health benefits and are cost effective. In the longer term, these interventions could lead to further benefits for society as a whole, making them even more cost effective.

The modeling found that targeted interventions are not cost effective in the short term, as they incur similar costs to universal interventions but only a small proportion of the school population benefits. However, they may be cost effective in the longer term (after 4 years) when both the health and broader societal benefits are taken into account.

The systematic review of published literature did not find any published analyses of the cost effectiveness of universal interventions and only one for focused (targeted) initiatives.

Fieldwork findings

Fieldwork aimed to test the relevance, usefulness and the feasibility of implementing the recommendations and the findings were considered by PHIAC in developing the final recommendations. For details, go to the fieldwork section in appendix B and full fieldwork reports: 'Mental wellbeing of children public health guidance', and 'Children and young people's voices on emotional well-being report for NICE 2007'.

Overall, first stage fieldwork participants felt the recommendations were appropriately targeted and easy to understand and could help promote social and emotional wellbeing and prevent mental illness among primary school-age children.

In addition, the following implementation issues were raised:

  • Whether the recommendations could be implemented as part of the Healthy Schools and SEAL (Social and Emotional Aspects of Learning) Programmes.

  • Whether the resources needed for training would be made available.

  • Whether the recommendations might be used as part of the Ofsted inspection process to assess a school's progress in meeting the outcomes set out in 'Every child matters'.

  • Whether it is possible to increase access to a range of specialists, particularly child and adolescent mental health services, given the constraints on resources.

Second stage fieldwork participants considered the recommendations useful and relevant, although again, there were concerns about implementation, particularly in terms of the resources needed.

Third stage fieldwork (the research on childrens' views) highlighted the importance of children being involved in helping to develop and implement methods of promoting emotional and social wellbeing in schools. Children stated that bullying and racism were key issues, although school itself made some participants very sad.

Pupil participation in the recruitment of emotional support staff would ensure they could turn to staff they valued. However, participants also felt it might mean they could no longer turn to other staff they have talked to in the past.

The report also highlighted a need to remove the stigma surrounding mental health issues and a need to share information about mental heath services with children – and to increase their access to such information.