Appendix D: summary of the methods used to develop this guidance

Introduction

The reports of the review and economic analysis include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

In addition, a study was commissioned on current service provision. The first part of the study examined NHS Stop Smoking Services in England through case studies and an on-line survey of NHS Stop Smoking Services coordinators. The second part of the study examined learning from Scotland and Ireland, following the introduction of legislation to make workplaces smokefree.

The minutes of the PHIAC meetings provide further detail about the Committee's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E and are available from the NICE website.

The guidance development process

The stages of the guidance development process are outlined in the box below:

1. Draft scope

2. Stakeholder meeting

3. Stakeholder comments

4. Final scope and responses published on website

5. Reviews and cost-effectiveness modelling

6. Synopsis report of the evidence (executive summaries and evidence tables) circulated to stakeholders for comment

7. Comments and additional material submitted by stakeholders

8. Review of additional material submitted by stakeholders (screened against inclusion criteria used in reviews)

9. Synopsis, full reviews, supplementary reviews and economic modelling submitted to PHIAC

10. PHIAC produces draft recommendations

11. Draft recommendations published for comment by stakeholders on website

12. PHIAC amends recommendations

13. Responses to comments published on website

14. Final guidance published on website

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and facilitated the development of recommendations by PHIAC. The overarching question was:

Which interventions are effective and cost effective in the workplace?

The subsidiary questions were:

  • Which interventions work best in workplaces where comprehensive smokefree legislation has been introduced?

  • What are the most effective and appropriate interventions for different sectors of the workforce such as men and women, younger and older workers, minority ethnic groups and temporary/casual workers?

  • What are the most effective ways of encouraging employee compliance with a smokefree policy?

  • How can employers support and encourage people who smoke to quit?

  • What support can employers offer people who smoke and who are not currently ready to quit?

  • How can employers be encouraged to provide smoking cessation support?

  • What are the resource needs of large, medium and small enterprises in implementing smokefree legislation and helping people who smoke to quit?

  • Which interventions are cost effective?

Reviewing the evidence of effectiveness

A review of effectiveness was conducted.

Identifying the evidence

The following databases were searched in four stages, as follows.

Stage 1

The search for systematic reviews and reviews was undertaken in the following databases for the years 1990–2006: Cochrane database of systematic reviews; DARE; National Research Register; Health Technology Assessment Database; SIGN Guidelines; National Guideline Clearinghouse; HSTAT; TRIP; Medline (1966–May 2006); Embase (1974–2006); CINAHL (1982–2006); British Nursing Index (1994–2006); PsycINFO (1806–2006); DH-Data (1983–2006); King's Fund (1979–2006).

Stage 2

The search for other publications was undertaken in the following databases: Medline (1966–May 2006); Embase (1974–2006); CINAHL (1982–2006); British Nursing Index (1994–2006); PsycINFO (1806–2006); DH-Data (1983–2006); King's Fund (1979–2006); CENTRAL (2006/2).

Stage 3

A further search of Medline was undertaken for abstracts (as well as titles) of all publications.

Stage 4

A search was undertaken of the following websites to identify any additional reports and documents of relevance:

Further details of the databases, search terms and strategies are included in the review report.

Selection criteria

Studies were included if they covered:

  • people who smoke aged 16 and over

  • workplace smoking cessation interventions delivered either at work or externally.

Studies were excluded if they described workplace health improvement programmes that did not include a smoking-related component.

Outcomes of interest included non-validated and validated measures of smoking behaviour. In the case of Cochrane reviews, effectiveness studies were only included if they had a follow-up after 6 months or longer.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for development of NICE public health guidance' (see appendix E). Each study was described by study type (classified 1–4) and graded (++, +, -) to reflect the risk of potential bias arising from its design and execution.

Study type

1. Meta-analyses, systematic reviews of RCTs, or RCTs (including cluster RCTs).

2. Systematic reviews of, or individual, non-randomised controlled trials, case-control studies, cohort studies, controlled before-and-after (CBA) studies, interrupted time series (ITS) studies, correlation studies.

3. Non-analytic studies (for example, case reports, case series studies).

4. Expert opinion, formal consensus.

Study quality

++ All or most criteria have been fulfilled. Where they have not been fulfilled the conclusions are thought very unlikely to alter.

+ Some criteria fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.

– Few or no criteria fulfilled. The conclusions of the study are thought likely or very likely to alter.

Study design and quality were combined. For example, a type 1 study fulfilling most criteria and a type 2 study fulfilling very few criteria would appear in the format 1 (++) and 2 (-) respectively. Each review includes a number of evidence statements that reflect the strength (quantity, type and quality) of evidence.

The studies were also assessed for their applicability to the UK.

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see full reviews and synopsis). The findings from the review were synthesised and used as the basis for a number of evidence statements relating to each question. The evidence statements reflect the strength (quantity, type and quality) of the evidence and its applicability to the populations and settings in the scope.

Economic appraisal

The economic appraisal consisted of a review of economic evaluations and a cost-effectiveness analysis.

Review of economic evaluations

A systematic search was carried out on the NHS EED database and the Centre for Reviews and Dissemination (CRD) internal database. This was supplemented by material found in the effectiveness and cost-effectiveness reviews undertaken for the NICE smoking cessation programme (under development).

The criteria for inclusion in the review were as follows:

  • studies included a specific intervention to assist smoking cessation

  • the study population was smoking at the start of the study (unless drawn from a general population)

  • studies reported on both the cost and effectiveness of the smoking cessation intervention (although cost and effectiveness was not necessarily combined into a single cost-effectiveness ratio).

Ten studies met the inclusion criteria. These were assessed for their methodological rigour and quality using the critical appraisers' checklists provided in appendix B of the 'Methods for development of NICE public health guidance' (see table 3.1). Each study was categorised by study type and graded using a code (++), (+) or (-), based on the potential sources of bias.

Cost-effectiveness analysis

An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The results are reported in: 'Cost effectiveness of interventions for smoking cessation' (Flack et al. 2007a) and 'Cost impact analysis of workplace-based interventions for smoking cessation' (Flack et al. 2007b). These reports are available on the NICE website.

How PHIAC formulated the recommendations

At its meetings in November and December 2006, PHIAC considered the evidence of effectiveness and cost effectiveness and comments from stakeholders to determine:

  • whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement

  • whether, on balance, the evidence demonstrates that the interventions are effective or ineffective, or whether it is equivocal

  • where there is an effect, the typical size of effect.

PHIAC developed draft recommendations through informal consensus, based on the following criteria:

  • Strength (quality and quantity) of evidence of effectiveness and its applicability to the populations/settings referred to in the scope.

  • Effect size and potential impact on population health and/or reducing inequalities in health.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of risks and benefits.

  • Ease of implementation and the anticipated extent of change in practice that would be required.

PHIAC also considered whether a recommendation should only be implemented as part of a research programme where evidence was lacking.

Where possible, recommendations were linked to an evidence statement(s) (see appendix A for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance including the recommendations was released for consultation in December 2006. The guidance was signed off by the NICE Guidance Executive in March 2007.