Appendix B Summary of the methods used to develop this guidance
The reviews and economic analysis include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.
The minutes of the PDG meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.
The stages involved in developing public health programme guidance are outlined in the box below.
1. Draft scope released for consultation
2. Stakeholder meeting about the draft scope
3. Stakeholder comments used to revise the scope
4. Final scope and responses to comments published on website
5. Evidence reviews and economic analysis undertaken
6. Evidence and economic analysis released for consultation
7. Comments and additional material submitted by stakeholders
8. Review of additional material submitted by stakeholders (screened against inclusion criteria used in reviews)
9. Evidence and economic analysis submitted to PDG
10. PDG produces draft recommendations
11. Draft guidance released for consultation and for field testing
12. PDG amends recommendations
13. Final guidance published on website
14. Responses to comments published on website
The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PDG to help develop the recommendations. The primary questions were:
Question 1: What type of price controls are effective and cost effective in reducing alcohol consumption, alcohol misuse, alcohol-related harm or alcohol-related social problems among adults and young people?
Question 2: Which interventions are effective and cost effective at managing alcohol availability to reduce levels of consumption, alcohol misuse, alcohol-related harm or alcohol-related social problems among adults and young people?
Question 3: Is the control of alcohol promotion (for example, advertising) effective and cost effective in reducing levels of consumption, alcohol misuse, alcohol-related harm or alcohol-related social problems among adults and young people?
Question 4: What are the key factors that increase the risk of an individual misusing alcohol? When are individuals most vulnerable to alcohol misuse?
Question 5: Are alcohol screening questionnaires, biochemical markers or clinical indicators (for example, hypertension, dilated facial capillaries) an effective and cost effective way of identifying adults and young people who currently misuse – or are at risk of misusing – alcohol?
Question 6: Are brief interventions effective and cost effective in managing hazardous and harmful drinking among adults and young people?
Question 7: What are the key barriers to helping adults and young people manage their drinking behaviour (for example, is access to services a problem)? What are the key facilitators?
These questions were made more specific for each review (see reviews for further details).
Two reviews of effectiveness were conducted.
Relevant literature was identified using an iterative search process. Study types and years were not predefined. The following databases were searched.
ASSIA (Applied Social Science Index and Abstracts)
Cochrane Library (Cochrane database of systematic reviews, Database of abstracts of reviews of effects, Health technology assessment and Cochrane-controlled trials register)
MEDLINE (including MEDLINE in process)
NHS Economic Evaluation Database (NHS EED)
Social Science Citation Index
Additional searches (non-systematic) were carried out on the following websites:
Studies were included in the effectiveness reviews if:
people of a range of ages were involved
interventions were relevant to the key questions set out in the reviews
outcomes such as alcohol consumption, alcohol misuse, alcohol-related harm, social problems, costs and economic impact were reported.
Studies were excluded if:
they were not published in English
the study population was below the age of 10 years
the evidence did not originate in economically developed countries (that is, if it did not come from countries that are members of the Organisation for Economic Cooperation and Development [OECD]).
Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for the development of NICE public health guidance' (see appendix E). Each study was graded (++, +, –) to reflect the risk of potential bias arising from its design and execution.
++ All or most of the methodology checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are thought very unlikely to alter.
+ Some of the methodology checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.
– Few or no methodology checklist criteria have been fulfilled. The conclusions of the study are thought likely or very likely to alter.
The review data was summarised in evidence tables (see full reviews).
The findings from the reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the public health collaborating centre (see appendix A). The statements reflect the collaborating centre's judgement of the strength (quantity, type and quality) of evidence and its applicability to the populations and settings in the scope.
The economic analysis consisted of two cost effectiveness reviews and an economic modelling report .
The following databases were searched for economic literature, in addition to the searches carried out for the effectiveness reviews:
NHS Economic Evaluation Database (NHS EED).
Studies were included if:
they addressed key questions 1, 2, 3, 5 and 6
they were from peer-reviewed journals published in English
the study population involved a range of ages (10+ years)
they were carried out in OECD countries.
A number of assumptions were made which could underestimate or overestimate the cost effectiveness of the interventions (see review modelling report for further details).
An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The results are reported in: 'Modelling to assess the effectiveness and cost-effectiveness of public health-related strategies and interventions to reduce alcohol attributable harm in England using the Sheffield alcohol policy model version 2.0'.
Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are and how feasible it would be to put them into practice. It was conducted with commissioners, practitioners and other interested parties who are involved in alcohol services in the NHS, local authorities and the private, voluntary and community sectors. They included: policy makers, applied researchers, economists, trading standards, representatives of licensing boards, retailers and the alcohol industry, and representatives from criminal justice and social welfare.
The fieldwork comprised:
five meetings in Birmingham, Bristol, Leicester, Liverpool and London conducted by Liverpool John Moores University with policy makers, commissioners, industry representatives and practitioners
an online survey of professionals (14) who could not attend the fieldwork meetings.
The fieldwork meetings and online survey were commissioned to ensure there was ample geographical coverage. The main issues arising are set out in appendix C under fieldwork findings. The fieldwork report, 'Alcohol-use disorders: preventing the development of hazardous or harmful drinking', is available online.
At its meeting in July 2009, the PDG considered the evidence of effectiveness and cost effectiveness to determine:
whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement
whether, on balance, the evidence demonstrates that the intervention is effective, ineffective or equivocal
where there is an effect, the typical size of effect.
The PDG developed draft recommendations through informal consensus, based on the following criteria:
Strength (quality and quantity) of evidence of effectiveness and its applicability to the populations/settings referred to in the scope.
Effect size and potential impact on the target population's health.
Impact on inequalities in health between different groups of the population.
Cost effectiveness (for the NHS and other public sector organisations).
Balance of risks and benefits.
Ease of implementation and any anticipated changes in practice.
The PDG noted that effectiveness can vary according to the context. For example, it depends on the enforcement of different regulatory regimes.
Where possible, recommendations were linked to an evidence statement(s) (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).
The draft guidance, including the recommendations, was released for consultation in September 2009. At its meeting in December 2009, the PDG amended the guidance in light of comments from stakeholders and experts and the fieldwork. The guidance was signed off by the NICE Guidance Executive in March 2009.