8 Summary of the methods used to develop this guidance

Introduction

The reviews, primary research, commissioned reports and economic modelling report include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the Programme Development Group (PDG) meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in About this guidance.

Guidance development

The stages involved in developing public health programme guidance are outlined in the box below.

1. Two draft scopes released for consultation

2. Stakeholder meeting about the draft scopes

3. Stakeholder comments used to revise the scopes

4. Final scopes and responses to comments published on website

5. Evidence reviews and economic modelling undertaken and submitted to PDG

6. PDG produces draft recommendations

7. Draft guidance (and evidence) released for consultation

8. PDG amends recommendations

9. Final guidance published on website

10. Responses to comments published on website

Key questions

The key questions were established as part of the 2 original scopes developed for guidance on smoking cessation for acute and maternity services and for separate guidance on mental health services. (These 2 pieces of guidance have now been amalgamated.) They formed the starting point for the reviews of evidence and were used by the PDG to help develop the recommendations. The overarching questions were:

  1. How effective and cost effective are smoking cessation interventions in secondary care settings in helping people to quit?

  2. How effective and cost effective are interventions in secondary care settings to help people temporarily abstain from smoking?

  3. How effective and cost effective are the current approaches used by secondary care services to identify and refer people to stop smoking services or to provide them with smoking cessation information, advice and support?

  4. What type of approaches are effective and cost effective at encouraging secondary care professionals to record smoking status, offer smoking cessation information, advice and support, or to refer people to stop smoking services?

  5. How can community, primary and secondary care providers collaborate more effectively to provide seamless smoking cessation services?

  6. What barriers and facilitators affect the delivery of effective interventions in secondary care?

  7. What are the effects of nicotine intake, or changes in levels of nicotine intake, on the mental and physical health of people using secondary care services who are on medication? What are the effects of tobacco consumption, or changes in tobacco consumption on this group?

  8. What are the effects of nicotine intake, or changes in levels of nicotine intake, on the mental and physical health of people using secondary care services?

  9. How effective and cost effective are strategies and interventions for ensuring compliance with smokefree legislation and local smokefree policies in secondary care settings?

  10. Are there any unintended consequences from adopting a smokefree approach in secondary care settings?

  11. What factors encourage or discourage compliance with smokefree policies in secondary care settings? What are the views, perceptions and beliefs of secondary care staff and people who use or visit these services?

These questions were made more specific for each review (see reviews for further details).

Reviewing the evidence

Below is a summary of the review methods. For full details see the reviews and economic analysis: available online.

Effectiveness reviews

Three reviews of effectiveness were conducted. For more details on the reviews see What evidence is this guidance based on? These covered:

  • Smoking cessation interventions in acute and maternity services (review 2).

  • Smoking cessation interventions in mental health services (review 4).

  • Smokefree strategies and interventions in secondary care settings (review 6).

Identifying the evidence

A number of databases and national and international websites were searched as follows:

Review 2 A search was conducted in December 2011 for systematic reviews and randomised controlled trials (RCTs) from January 1990 onwards.

Review 4 A search was conducted in February 2012 for evidence from January 1985 onwards. This included: reviews of reviews, systematic reviews, RCTs, non-randomised controlled trials, controlled before-and-after studies, interrupted time series and uncontrolled before-and-after studies.

Review 6 A search was conducted in February 2012 for evidence from January 1990 onwards. This included: reviews of reviews, systematic reviews, RCTs, non-randomised controlled trials, controlled before-and-after studies, interrupted time series, uncontrolled before-and-after studies and retrospective comparison studies.

See each review for details of the databases and websites searched.

A call for evidence from registered stakeholders was made in June 2012.

Selection criteria

Studies were included in the effectiveness reviews if they covered the following:

  • people who:

    • use secondary care services or people who live in the same household as someone who is using these services

    • visit secondary care settings

    • work in secondary care settings

  • interventions to:

    • identify and refer people to stop smoking services or to increase general uptake of stop smoking services

    • help people stop smoking

    • help people temporarily abstain from smoking

    • smokefree strategies and interventions in hospitals and other secondary care settings.

Studies were excluded if they:

  • were aimed at people who use primary care services

  • covered interventions aimed at preventing people from taking up smoking.

See each review for details of the inclusion and exclusion criteria.

Other reviews

Three reviews of the barriers to and facilitators for quitting smoking were conducted. These covered:

  • acute and maternity services (review 3)

  • mental health services (review 5)

  • smokefree strategies and interventions in secondary care settings (review 7).

Identifying the evidence

A number of databases and national and international websites were searched:

Review 3 A search was conducted in December 2011 for evidence from January 1990 onwards. This included: systematic reviews, trials (controlled and non-controlled), descriptive studies (including questionnaire surveys and views or process evaluations), qualitative studies and discussion papers or reports.

Review 5 A search was conducted in February 2012 for evidence from January 1985 onwards. This included: systematic reviews, trials (controlled and non-controlled), descriptive studies (including questionnaire surveys and views or process evaluations), qualitative studies and reports.

Review 7 A search was conducted in February 2012 for evidence from January 1990 onwards. This included: systematic reviews, trials (controlled and non-controlled), descriptive studies (including questionnaire surveys and views or process evaluations), qualitative studies and reports.

Selection criteria

The selection criteria for the barriers and facilitators reviews were the same as for the effectiveness reviews (see above).

See each review for details of the inclusion and exclusion criteria.

Review 1: Review of effects of nicotine in secondary care

Identifying the evidence

A number of databases and national and international websites were searched in December 2011 for studies published from January 1990 onwards.

This included quantitative (both experimental and observational) studies, qualitative studies, systematic reviews, reviews of reviews and reports (see review for details).

See review 1 for details of the databases and websites searched.

A call for evidence from registered stakeholders was made in June 2012.

Selection criteria

Studies were included in review 1 if they covered people who use secondary care (acute, maternity or mental health) services and:

Studies were excluded if they reported on the long-term health effects of tobacco use or of stopping smoking.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in Methods for the development of NICE public health guidance. Each study was graded (++, +, –) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.

+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.

– Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.

Summarising the evidence and making evidence statements

The review data were summarised in evidence tables (see full reviews).

The findings from the review were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the external contractors (see About this guidance). The statements reflect their judgement of the strength (quality, quantity and consistency) of evidence and its applicability to the populations and settings in the scope.

Cost effectiveness

There was a review of economic evaluations and an economic modelling exercise.

Review of economic evaluations

Three economic evaluation databases were searched in February 2012 for studies from January 1990 onwards.

To supplement database and website searches, potentially relevant economic studies were identified using the screening results from the searches carried out for the effectiveness reviews.

Studies were included if they reported on a full economic evaluation with the same populations and interventions as in the effectiveness reviews (see above). Included studies were then quality-assessed.

Economic modelling

A number of assumptions were made that could underestimate or overestimate the cost effectiveness of the interventions (see economic modelling report for further details).

Economic models were constructed to incorporate data from the reviews of effectiveness and cost effectiveness as follows:

A general model that considered the long-term impacts of smoking (which are similar for all population groups including patients, staff and visitors).

Six models based on case studies which focus on the specific impact of smoking in a secondary care context (recovery times and the likelihood of complications associated with secondary care, generally within 12 months).

The results are reported in 'Smoking cessation in secondary care: cost-effectiveness review' and 'Economic analysis of smoking cessation in secondary care' (see What evidence is the guidance based on?).

How the PDG formulated the recommendations

At its meetings between March 2012 and July 2013 the Programme Development Group (PDG) considered the evidence, expert papers and cost effectiveness to determine:

  • whether there was sufficient evidence (in terms of strength and applicability) to form a judgement

  • where relevant, whether (on balance) the evidence demonstrates that the intervention or programme/activity can be effective or is inconclusive

  • where relevant, the typical size of effect (where there is one)

  • whether the evidence is applicable to the target groups and context covered by the guidance.

The PDG developed recommendations through informal consensus, based on the following criteria:

  • Strength (type, quality, quantity and consistency) of the evidence.

  • The applicability of the evidence to the populations/settings referred to in the scope.

  • Effect size and potential impact on the target population's health.

  • Impact on inequalities in health between different groups of the population.

  • Equality and diversity legislation.

  • Ethical issues and social value judgements.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of harms and benefits.

  • Ease of implementation and any anticipated changes in practice.

Where possible, recommendations were linked to an evidence statement(s) (see The evidence for details). If a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

  • National Institute for Health and Care Excellence (NICE)