8 Summary of the methods used to develop this guidance

Introduction

The reviews, primary research, commissioned reports and economic modelling report include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the Programme Development Group (PDG) meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in About this guidance.

Guidance development

The stages involved in developing public health programme guidance are outlined in the box below.

1. Draft scope released for consultation

2. Stakeholder meeting about the draft scope

3. Stakeholder comments used to revise the scope

4. Final scope and responses to comments published on website

5. Evidence reviews and economic modelling undertaken and submitted to PDG

6. PDG produces draft recommendations

7. Draft guidance (and evidence) released for consultation and for field testing

8. PDG amends recommendations

9. Final guidance published on website

10. Responses to comments published on website

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PDG to help develop the recommendations. The overarching questions were:

Question 1: How effective and cost effective are pharmacotherapies in helping people to:

  • cut down smoking before quitting

  • cut down or abstain from smoking, temporarily or indefinitely?

How effective and cost effective are different combinations of NRT products?

Question 2: How effective and cost effective are nicotine-containing products in helping people to:

  • cut down smoking before quitting

  • cut down or abstain from smoking, temporarily or indefinitely?

Question 3: Which kinds of behavioural support, counselling, advice or self-help (with or without pharmacotherapy) are effective and cost effective in helping people to:

  • cut down smoking before quitting

  • cut down or abstain from smoking, temporarily or indefinitely.

Question 4: Do some tobacco harm-reduction approaches have a differential impact on different groups (for example, people of different ages, gender, socioeconomic status or ethnicity)?

Question 5: Are there any unintended consequences from adopting a tobacco harm-reduction approach, for example, does it deter people from trying to stop smoking?

Question 6: How can practitioners deliver messages about tobacco harm reduction without weakening the impact of advice about the benefits of stopping smoking?

Question 7: What factors might act as barriers or facilitators to tobacco harm-reduction approaches?

Question 8: Does long-term use of pharmacotherapies or 'nicotine-containing products' have any ill-effects on health?

These questions were made more specific for each review (see reviews for further details).

These questions were made more specific for each review (see reviews for further details).

Reviewing the evidence

Effectiveness reviews

Three reviews of effectiveness were conducted (reviews 2, 3 and 5). These covered:

  • tobacco harm-reduction approaches with the intention of quitting with or without help

  • long-term tobacco harm-reduction approaches without intending to quit and with or without help

  • long-term use of non-tobacco nicotine-containing products among people who have quit smoking abruptly.

Identifying the evidence

A number of databases were searched in August 2011 for: systematic reviews, guidelines, randomised controlled trials (RCTs), controlled trials, controlled and uncontrolled before-and-after studies and interrupted time series studies from January 1990.

A number of national and international websites were also searched.

In addition, a range of databases were searched for information on studies in progress, unpublished research or research reported in the grey literature.

A call for evidence from registered stakeholders was made in August 2011.

See each review for details of the databases and websites searched (and dates of any update searches).

Selection criteria

Studies were included in the effectiveness reviews if: they covered the following people and at least 1 of the following interventions:

  • people who want to quit smoking gradually, reduce their cigarette consumption or temporarily abstain from smoking

  • people who have quit smoking abruptly and replaced cigarettes with nicotine replacement therapy or products containing nicotine

  • pharmacotherapies licensed for cutting down, temporary abstinence or harm reduction

  • other non-tobacco 'nicotine-containing products', such as 'electronic cigarettes' and topical gels

  • behavioural support, counselling or advice for individuals or groups

  • self-help.

Studies were excluded if they focused on pregnant women or were about:

  • pharmacotherapies not licensed for cutting down, temporary abstinence or harm reduction

  • products containing tobacco, including products that claim to deliver reduced levels of toxicity or that reduce exposure to tobacco smoke

  • products that are smoked that do not contain tobacco.

  • smokeless tobacco products

  • alternative or complementary therapies.

See each review for details of the inclusion and exclusion criteria.

Other reviews

Review 1

Identifying the evidence

A range of databases and websites were searched in August 2011 for studies published from 1980 onwards. Studies were included if they were: RCTs, systematic reviews of RCTs, non-randomised trials and randomised and non-randomised pharmacokinetic studies.

This was supplemented by grey literature searches and a call for evidence.

See the review for details.

Selection criteria

Studies were included in review 1 if they:

  • reported on the safety, risks and pharmacokinetic profiles of tobacco harm-reduction strategies

  • addressed special pharmacokinetic or safety considerations that arise when using nicotine-containing products to help reduce the harm from smoking.

Studies were excluded if they focused on pregnant women.

Review 4

Identifying the evidence

A range of databases and websites were searched in August 2011 for: RCTs, systematic reviews of RCTs, non-randomised trials, qualitative and quantitative evidence of views and opinions, and process evaluations of intervention studies, from 1990 onwards.

This was supplemented by grey literature searches and 2 calls for evidence. Follow-up database searches were conducted in November 2011 and January 2012.

A call for evidence was also made.

See the review for details.

Selection criteria

Studies were included if they focused on the following people or services and at least 1 of the following interventions:

  • people (or the families of people) who want to quit smoking gradually, reduce their cigarette consumption or temporarily abstain from smoking

  • service providers, healthcare personnel and policy makers who may deliver/commission/refer smokers to tobacco harm reduction interventions

  • pharmacotherapies licensed for cutting down, temporary abstinence or harm reduction

  • other non-tobacco products containing nicotine (electronic cigarettes and topical gels)

  • behavioural support, counselling, advice or self-help.

Studies were excluded if they focused on pregnant women.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in Methods for the development of NICE public health guidance. Each study was graded (++, +, –) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.

+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.

– Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.

The evidence was also assessed for its applicability to the areas (populations, settings, interventions) covered by the scope of the guidance. Each evidence statement concludes with a statement of applicability (directly applicable, partially applicable, not applicable).

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see full reviews).

The findings from the reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the external contractors (see About this guidance). The statements reflect their judgement of the strength (quality, quantity and consistency) of evidence and its applicability to the populations and settings in the scope.

Cost effectiveness

There was a review of economic evaluations and an economic modelling exercise.

Review of economic evaluations

The databases searched included most of NICE's core databases. The search strategies from the effectiveness reviews were used with a filter designed to identify economic and costs studies.

Studies were included if they reported on a full economic evaluation with the same populations and interventions as in the effectiveness reviews (see above).

Two studies were identified that met the inclusion criteria but only 1 was applicable. The results are reported in A rapid review of economic evidence on tobacco harm reduction strategies on the NICE website.

Economic modelling

An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The model covered quitting smoking using nicotine-containing products on a long-term basis. It also covered interventions to reduce smoking. Following a request from the Programme Development Group, a supplementary analysis was undertaken to explore additional scenarios.

The results are reported in: 'An economic evaluation of different interventions to promote tobacco harm reduction'; and the supplementary analysis entitled, 'An economic evaluation of different interventions to promote tobacco harm reduction: supplementary report'.

Fieldwork

Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are for practitioners and how feasible it would be to put them into practice.

It was conducted with practitioners and commissioners who are involved in stop smoking services or who have a wider public health remit. This included those working in local authorities, the NHS and the voluntary and community sectors. It also included manufacturers and retailers of licensed nicotine-containing products.

The fieldwork comprised: focus groups, telephone interviews and face-to-face interviews, carried out in the East Midlands, East of England, London, North West, North East, South West, West Midlands and Yorkshire by ICF GHK.

The main issues arising from the fieldwork are set out in Fieldwork findings. The full fieldwork report is Tobacco: harm-reduction approaches to smoking – final fieldwork report.

How the PDG formulated the recommendations

At its meetings between October 2011 and June 2012, the Programme Development Group (PDG) considered the evidence, expert papers and cost effectiveness to determine:

  • whether there was sufficient evidence (in terms of strength and applicability) to form a judgement

  • where relevant, whether (on balance) the evidence demonstrates that the intervention or programme/activity can be effective or is inconclusive

  • where relevant, the typical size of effect (where there is one)

  • whether the evidence is applicable to the target groups and context covered by the guidance.

The PDG developed draft recommendations through informal consensus, based on the following criteria:

  • Strength (type, quality, quantity and consistency) of the evidence.

  • The applicability of the evidence to the populations/settings referred to in the scope.

  • Effect size and potential impact on the target population's health.

  • Impact on inequalities in health between different groups of the population.

  • Equality and diversity legislation.

  • Ethical issues and social value judgements.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of harms and benefits.

  • Ease of implementation and any anticipated changes in practice.

Where possible, recommendations were linked to an evidence statement(s) (see The evidence for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

  • National Institute for Health and Care Excellence (NICE)