Guidance
9 Summary of the methods used to develop this guideline
9 Summary of the methods used to develop this guideline
Introduction
The review and economic modelling report include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.
The minutes of the Public Health Advisory Committee (PHAC) meetings provide further detail about the Committee's interpretation of the evidence and development of the recommendations.
Guideline development
The stages involved in developing public health guidelines are outlined in the box below.
1. Draft scope released for consultation
2. Stakeholder meeting about the draft scope
3. Stakeholder comments used to revise the scope
4. Final scope and responses to comments published on website
5. Effectiveness reviews and economic modelling undertaken and submitted to PHAC
6. PHAC produces draft recommendations
7. Draft guideline (and evidence) released for consultation
8. PHAC amends recommendations
9. Final guideline published on website
10. Responses to comments published on website
Key questions
The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PHAC to help develop the recommendations. The overarching questions were:
1. How effective and cost effective are interventions to increase awareness and implementation of existing guidance on vitamin D among health professionals or others working with specific population groups? What are the implications for professional training and practice?
2. How effective and cost effective are interventions to increase awareness and uptake of existing guidance on vitamin D among specific population groups (with special consideration given to those eligible for the Healthy Start scheme)?
3. What helps or hinders the implementation of existing guidance on vitamin D by commissioners, providers, practitioners, those working with specific population groups and people in specific population groups?
4. What local provision is made to ensure vitamin D supplements are available for different specific population groups (including Healthy Start, prescriptions and over-the-counter sales)?
These questions were made more specific for the effectiveness review 'Vitamin D: a systematic review of effectiveness and cost-effectiveness of activities to increase awareness, uptake and provision of vitamin D supplements in specific population groups'.
Reviewing the evidence
Effectiveness review
One effectiveness review was conducted.
Identifying the evidence
Several databases were searched in June 2013 for evidence of any type published from 2000 onwards. See review 1: Vitamin D: a systematic review of effectiveness and cost-effectiveness of activities to increase awareness, uptake and provision of vitamin D supplements in at-risk groups for details.
Evidence was also identified through:
-
citation searches of papers identified for inclusion
-
a search for additional studies by authors of papers identified for inclusion
-
a search of identified webpages
-
a call for evidence issued by NICE in March 2013.
Selection criteria
Studies were included in the effectiveness review if they:
-
were undertaken in the UK
-
addressed at least 1 of the key questions.
Studies were excluded if they focused on:
-
the management of vitamin D deficiency or conditions that may increase the risk
-
fortification of food and drinks with vitamin D
-
vitamin D for different population groups.
See review 1 for details.
Review of systematic review
One review of systematic reviews was conducted.
Identifying the evidence
Several systematic review databases were searched in November 2013 for systematic reviews published from 2004 onwards.
Selection criteria
Studies were included if they were systematic reviews and addressed at least 1 of the key questions. Studies were excluded if they were not systematic reviews. See review 2.
Quality appraisal
Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in Methods for the development of NICE public health guidance. A checklist developed by Cardiff University was used for the cross-sectional studies and survey reports. Systematic reviews were assessed using the AMSTAR checklist. Each study was graded (++, +, −) to reflect the risk of potential bias arising from its design and execution.
Study quality
++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.
+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.
− Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.
Summarising the evidence and making evidence statements
The review data were summarised in evidence tables (see review 1 and review 2).
The findings from the review were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the external contractor (see Supporting evidence). The statements reflect their judgement of the strength (quality, quantity and consistency) of evidence and its applicability to the populations and settings in the scope.
Cost effectiveness
There was a review of economic evaluations and an economic modelling exercise.
Economic modelling
An economic model was constructed, incorporating data from the reviews of effectiveness and cost effectiveness. The results are reported in: An economic evaluation of interventions to improve the uptake of vitamin D supplements in England.
How the PHAC formulated the recommendations
At its meetings in September and October 2013, the Public Health Advisory Committee (PHAC) considered the evidence, expert testimony and cost effectiveness to determine:
-
whether there was sufficient evidence (in terms of strength and applicability) to form a judgement
-
where relevant, whether (on balance) the evidence demonstrates that the intervention or programme or activity can be effective or is inconclusive
-
where relevant, the typical size of effect
-
whether the evidence is applicable to the target groups and context covered by the guideline.
The PHAC developed recommendations through informal consensus, based on the following criteria:
-
Strength (type, quality, quantity and consistency) of the evidence.
-
The applicability of the evidence to the populations/settings referred to in the scope.
-
Effect size and potential impact on the target population's health.
-
Impact on inequalities in health between different groups of the population.
-
Equality and diversity legislation.
-
Ethical issues and social value judgements.
-
Cost effectiveness (for the NHS and other public sector organisations).
-
Balance of harms and benefits.
-
Ease of implementation and any anticipated changes in practice.
Where possible, recommendations were linked to an evidence statement (see The evidence for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).