9 Summary of the methods used to develop this guidance

Introduction

The reviews and economic reports include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the Programme Development Group (PDG) meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

Guidance development

The stages involved in developing public health programme guidance are outlined in the box below.

1. Draft scope released for consultation

2. Stakeholder meeting about the draft scope

3. Stakeholder comments used to revise the scope

4. Final scope and responses to comments published on website

5. Evidence reviews and economic reports undertaken and submitted to PDG

6. PDG produces draft recommendations

7. Draft guidance (and evidence) released for consultation

8. PDG amends recommendations

9. Final guidance published on website

10. Responses to comments published on website

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PDG to help develop the recommendations. The overarching questions were:

  1. Which interventions are an effective and cost effective way of changing someone's behaviour and then helping them to sustain that change?

  2. Which specific behaviour change techniques (and combinations of behaviour change techniques) are effective and cost effective at helping individuals change and then sustain the new behaviour in the long term (for at least 6 months following the intervention)?

  3. Which behaviour change techniques are only effective for specific behaviours, such as helping people to quit alcohol or smoking? Which techniques can be used to tackle a range of behaviours?

  4. What characteristics and competencies are needed to deliver behaviour change interventions and techniques effectively?

  5. How do the effects of individual interventions/behaviour change techniques vary across different population groups?

  6. Which theories explain when, why and how behaviour change is maintained?

These questions were made more specific for each review (see reviews for further details).

Reviewing the evidence

Effectiveness reviews

Two reviews of effectiveness were conducted.

For more details on the reviews see Supporting evidence.

Identifying the evidence

The NICE website was searched in July 2012 for public health guidance relating to individual-level behaviour change interventions published since 2006 that address: alcohol, diet, physical activity, sexual behaviour, or smoking. See Review 1: Individual-level behaviour change: review of current NICE guidance and recommendations for details.

A number of databases were searched between July and September 2012 for papers relating to individual-level behaviour change interventions published since 2003 that address: alcohol, diet, physical activity, sexual behaviour, or smoking. See Review 2: Individual-level behaviour change: review of evidence of effectiveness of interventions and behaviour change techniques in individual level interventions for details of the databases searched.

Selection criteria

Studies were included in the effectiveness reviews if they:

  • covered individual-level interventions aimed at behaviour change in relation to at least 1 of the following: alcohol, diet, physical activity, sexual behaviour and smoking

  • were published by NICE (review 1)

  • were randomised controlled trials or systematic reviews published from 2003 onwards in English (review 2).

Studies were excluded from both reviews if they focused on:

See each review for details of the inclusion and exclusion criteria.

Other reviews

One review of qualitative data was undertaken.

Identifying the evidence

A number of databases were searched in September 2012 for papers published since 2003 on the characteristics and competencies needed to deliver individual-level behaviour change interventions and techniques. Specifically, the search focused on papers addressing alcohol, diet, physical activity, sexual behaviour or smoking. See Review 3: Individual-level behaviour change: A qualitative review of studies describing the skills base needed to deliver behaviour change interventions or techniques for details of the databases searched.

Selection criteria

Studies were included in review 3 if they described the skills or training needed to deliver behaviour change interventions in relation to at least 1 of the following: alcohol, diet, physical activity, sexual behaviour or smoking.

Studies were excluded if they focused on:

  • community- or population-level interventions

  • clinical or pharmacological methods with no public health or health promotion element

  • psychiatric interventions delivered as part of the therapeutic process for people with a mental health problem.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in Methods for the development of NICE public health guidance. Each study was graded (++, +, −) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.

+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.

− Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see the reviews in Supporting evidence).

The findings from the reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the external contractors (see Supporting evidence). The statements reflect their judgement of the strength (quality, quantity and consistency) of evidence and its applicability to the populations and settings in the scope.

Cost effectiveness

There were 2 reviews of economic evaluations.

Review of economic evaluations

Review 1: Behaviour change update stage 1: behaviour change technique (BCT) analysis of existing, cost-effective interventions involved an analysis of interventions already assessed by NICE as cost effective. The aim was to identify and classify the behaviour change techniques used in individual-level interventions and those based on choice architecture.

Review 2: Behaviour change update: Stage 3: BCT analysis of behaviour change interventions reported in studies of cost effectiveness involved a search of economic databases using the search strategies developed for the effectiveness reviews for the behaviour change update. The overall aim was to identify any additional economic evidence not already covered by NICE's existing analyses. The specific aims were to:

  • identify and classify the behaviour change techniques used

  • compare the behaviour change techniques used in interventions judged either cost-effective and cost-ineffective by NICE

  • compare the behaviour change techniques of cost-effective interventions identified in review 1 with cost-effective interventions in review 2.

How the PDG formulated the recommendations

At its meetings in September, October and December 2012 and March and April 2013, the Programme Development Group (PDG) considered the evidence, expert testimony and cost effectiveness to determine:

  • whether there was sufficient evidence (in terms of strength and applicability) to form a judgement

  • where relevant, whether (on balance) the evidence demonstrates that the intervention or programme/activity can be effective or is inconclusive

  • where relevant, the typical size of effect (where there is one)

  • whether the evidence is applicable to the target groups and context covered by the guidance.

The PDG developed recommendations through informal consensus, based on the following criteria:

  • Strength (type, quality, quantity and consistency) of the evidence.

  • The applicability of the evidence to the populations/settings referred to in the scope.

  • Effect size and potential impact on the target population's health.

  • Impact on inequalities in health between different groups of the population.

  • Equality and diversity legislation.

  • Ethical issues and social value judgements.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of harms and benefits.

  • Ease of implementation and any anticipated changes in practice.

The PDG noted that although effectiveness can vary according to the context, there was little evidence addressing the impact of interventions on different groups of people (for example according to ethnicity, socioeconomic status, or disability).

Where evidence was lacking, the PDG also considered whether a recommendation should be implemented only as part of a research programme. One 'in-research only' recommendation was made (see recommendation 4).

Where possible, recommendations were linked to an evidence statement (see The evidence for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

  • National Institute for Health and Care Excellence (NICE)