8 Summary of the methods used to develop this guideline | Oral health: local authorities and partners | Guidance

Introduction

The reviews, commissioned reports and economic modelling report include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the Public Health Advisory Committee (PHAC) meetings provide further detail about the Committee's interpretation of the evidence and development of the recommendations.

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PHAC to help develop the recommendations. The overarching questions were:

Question 1: What are the most effective and cost-effective programmes and interventions to promote, improve and maintain the oral health of a local community? In particular, what are the most effective and cost-effective approaches for groups of people who are disadvantaged and at high risk of poor oral health?

Question 2: What methods and sources of information will help local authorities identify the oral health needs and severity of oral health problems in their local community?

These questions were made more specific for each review.

Reviewing the evidence

Effectiveness reviews

One review of effectiveness was conducted. See evidence review 1: review of evidence of the effectiveness of community-based oral health improvement programmes and interventions.

Identifying the evidence

Several databases were searched in May 2013 for papers published since May 1993 that related to the effectiveness of programmes and interventions aiming to promote, improve and maintain the oral health of a local community. The review included studies from May 2003, with older studies (May 1993–May 2003) used to inform any gaps in the evidence. In addition, the grey literature was searched and supplemental searching was undertaken. See evidence review 1.

Selection criteria

Studies were included in the effectiveness reviews if they covered:

community based oral health promotion programmes and interventions that aimed to reduce and prevent dental and periodontal disease, oral cancer or other oral disease and promote oral health
programmes and interventions aimed at children, adults or older people living in the community, including people from disadvantaged populations such as homeless people.

Studies were excluded if they were conducted:

in a non-Organisation for Economic Cooperation and Development (OECD) country
with children or adults not living independently in the community, such as those living in residential care, prisons, or hospitals.

See evidence review 1 for details of the inclusion and exclusion criteria.

Other reviews

One review of the barriers and facilitators to implementing community-based oral health programmes was conducted. See evidence review 2: qualitative evidence review of barriers and facilitators to implementing community-based oral health improvement programmes and interventions.

Identifying the evidence

Several databases were searched in May 2013 for qualitative and quantitative studies from May 1993. Studies were included from May 2003, with older studies (May 1993–May 2003) used to inform any gaps in the evidence. In addition, the grey literature was searched and supplemental searching was undertaken. See evidence review 2 for details of the databases searched.

Selection criteria

Studies were included if:

they described user or provider views of the barriers or facilitators to the implementation, or uptake, of community-based oral health programmes.

Studies were excluded if they:

were conducted in a non-Organisation for Economic Cooperation and Development (OECD) country
focused on children or adults living in residential care, prisons, hospitals or other institutions.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in methods for the development of NICE public health guidance. Each study was graded (++, +, −) to reflect the risk of potential bias arising from its design and execution.

Study quality

++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.

+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.

− Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.

The evidence was also assessed for its applicability to the areas (populations, settings, interventions) covered by the scope of the guideline. Each evidence statement concludes with a statement of applicability (directly applicable, partially applicable, and not applicable).

Summarising the evidence and making evidence statements

The review data were summarised in evidence tables (see the reviews in supporting evidence).

The findings from the reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the external contractors (see supporting evidence). The statements reflect their judgement of the strength (quality, quantity and consistency) of evidence and its applicability to the populations and settings in the scope.

Commissioned reports

Oral health needs assessments

A structured review and survey of oral health needs assessments was conducted:

Report 1: An overview of oral health needs assessments.

Identifying the evidence

A mixed method approach was undertaken to identify the evidence relating to oral health needs assessment. This included a survey of public health consultants and a structured literature review. For the literature review, a range of databases was searched in June 2013 for studies from June 1946. See report 1 for details of the databases searched.

Selection criteria

Studies were included if they described how oral health needs assessment was carried out among vulnerable groups from a population perspective.

Studies were excluded if they focused on care provision or attitudes to specific treatments.

See report 1 for details of the inclusion and exclusion criteria and quality appraisal methods.

Overview of systematic reviews

An overview of relevant systematic reviews was undertaken to supplement and contextualise the effectiveness review:

Report 2: Commentary on selected systematic reviews.

Relevant systematic reviews were identified by the searches undertaken for the effectiveness reviews and by topic experts on the PHAC. These papers were appraised and summarised by a topic expert and described in a short report. See report 2 for details.

Expert papers

Two expert papers were presented to the PHAC:

Expert paper 1 'Working with vulnerable adults and older people at greater risk of poor oral health'.
Expert paper 2 'Overview of the Childsmile programme'.

Cost effectiveness

There was a review of economic evaluations and an economic modelling exercise. See 'Literature review of economic evaluations on oral health improvement programmes and interventions' and 'RX058: Economic analysis of oral health improvement programmes and interventions'.

Review of economic evaluations

The search strategy developed for the effectiveness review (report 1) was adapted to identify research for the cost effectiveness review.

Eight databases were searched from 1993 onwards. In addition, reference lists of reviews and studies included in the review were scanned to identify any further relevant studies. Citation searches and named author searches were also carried out to identify other publications by the authors of studies selected for inclusion.

Studies were included if they met the inclusion criteria for evidence review 1 and reported on a full economic evaluation with the same populations and interventions. Included studies were then quality-assessed.

Economic modelling

Assumptions were made that could underestimate or overestimate the cost effectiveness of the interventions (see review modelling report for further details).

Due to a paucity of data from the review of economic evaluations, an economic model was constructed to provide an idea of which interventions would be cost effectiveness.

Due to the large number of assumptions that had to be made, rather than report a single incremental cost effectiveness ratio (ICER), the model was used to undertake 2 analyses. Both provided a range of cost-effectiveness estimates. In addition, the probability that an intervention is cost effective in each given scenario was tested.

The results are reported in RX058: Economic analysis of oral health improvement programmes and interventions.

Fieldwork

Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are for practitioners and how feasible it would be to put them into practice. It was conducted with practitioners and commissioners who are involved in oral health services. They included those working in: early years services, local authorities, the NHS, social care and the voluntary and community sector.

The fieldwork comprised:

10 focus groups were carried out in Birmingham, London, Manchester and York by Word of Mouth: 2 groups in Birmingham (total of 26 people); 3 groups in London (total of 29 people); 3 groups in Manchester (total of 37 people); and 2 groups in York (total of 25 people). They involved: local and national commissioners; consultants in dental public health, local authority public health and oral health representatives, people from local professional dental networks and community dentistry; representatives from children and adult social care services; frontline staff working in early years and primary care services; people working in local education authorities and schools; representatives from the local Healthwatch; and from voluntary and community groups.
One-to-one in-depth telephone interviews with 15 people, carried out by Word of Mouth.

The study was commissioned to ensure there was ample geographical coverage. The main issues arising from the study are set out in section 9 under fieldwork findings. Or see the full fieldwork report.

How the PHAC formulated the recommendations

At its meetings between July 2013 and January 2014, the Public Health Advisory Committee (PHAC) considered the evidence, expert testimony and cost effectiveness to determine:

whether there was sufficient evidence (in terms of strength and applicability) to form a judgement
where relevant, whether (on balance) the evidence demonstrates that the intervention, programme or activity can be effective or is inconclusive
where relevant, the typical size of effect
whether the evidence is applicable to the target groups and context covered by the guideline.

The PHAC developed recommendations through informal consensus, based on the following criteria:

Strength (type, quality, quantity and consistency) of the evidence.
The applicability of the evidence to the populations/settings referred to in the scope.
Effect size and potential effect on the target population's health.
Effect on inequalities in health between different groups of the population.
Equality and diversity legislation.
Ethical issues and social value judgements.
Cost effectiveness (for the NHS and other public sector organisations).
Balance of harms and benefits.
Ease of implementation and any anticipated changes in practice.

Where possible, recommendations were linked to evidence statements (see the section on the evidence for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).