Guidance
7 Summary of the methods used to develop this guideline
7 Summary of the methods used to develop this guideline
Key questions
The key questions were established as part of the guideline scope. They formed the starting point for the reviews of evidence and were used by the PHAC to help develop the recommendations. The overarching questions were:
Question 1: Which subpopulations are more vulnerable to cold temperatures and poorly heated or expensive‑to‑heat homes? What factors contribute to vulnerability and how do these factors interact with each other?
Question 2: How effective and cost effective are interventions and approaches to reduce excess winter deaths and morbidity and the negative health consequences of cold weather and cold homes?
The subsidiary questions were:
1. How effective are these interventions?
2. How does effectiveness vary according to demographic, geographic, health, housing and socioeconomic characteristics?
3. What effect do these interventions have on health inequalities?
4. What effect do these interventions have on the wider determinants of health (for example, carbon dioxide emissions)?
5. What adverse effects are associated with changes to energy efficiency or the cost of heating? (For example, reduced ventilation may be associated with increased levels of indoor air pollution, including radon, and overheating may be associated with an increased risk of cot death.)
Question 3: What systems and strategies have been used to identify vulnerable and at‑risk populations and what effect do they have?
The subsidiary questions were:
1. What activities and interventions support effective delivery and implementation of approaches to reduce excess winter deaths and the negative health consequences of cold weather?
2. What influences the effectiveness of an integrated approach to addressing risk and vulnerability?
3. What are the most effective methods for reaching at‑risk and vulnerable subpopulations?
4. What approaches increase uptake and enhance the acceptability of effective interventions?
5. What facilitators and barriers influence delivery and implementation?
These questions were made more specific for each review.
Reviewing the evidence
Effectiveness reviews
Three evidence reviews were conducted:
-
Review 1: 'Factors determining vulnerability to winter‑ and cold weather‑related mortality/morbidity'.
-
Review 2: 'Interventions and economic studies'.
-
Review 3: 'Delivery and implementation of approaches for the prevention of excess winter deaths and morbidity'.
Identifying the evidence
The literature search involved searching a range of databases and grey literature resources. Databases searched included: Avery Index, HMIC, ICONDA International MEDLINE, PsycINFO, Social Science Citation Index and Social Policy and Practice. The searches were limited to the last 20 years (1993 to October 2013) and to English language publications. See reviews 1 to 3.
Details of the search strategies are given in the reviews.
Quality appraisal
Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in Methods for the development of NICE public health guidance. Each study was graded (++, +, −) to reflect the risk of potential bias arising from its design and execution.
Study quality
++ All or most of the checklist criteria have been fulfilled. Where they have not been fulfilled, the conclusions are very unlikely to alter.
+ Some of the checklist criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are unlikely to alter the conclusions.
− Few or no checklist criteria have been fulfilled. The conclusions of the study are likely or very likely to alter.
Summarising the evidence and making evidence statements
The review data were summarised in evidence tables (see the reviews in Supporting evidence). The findings from the studies were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the external contractors (see 'Supporting evidence').
The statements reflect their judgement of the strength (quality, quantity and consistency) of evidence and its applicability to the populations and settings in the scope.
Commissioned reports
Expert papers were also commissioned. These were:
-
Alzheimer's and dementia in relation to cold homes and excess winter mortality and morbidity
-
Benefit changes, fuel poverty and disability
-
Children's health and wellbeing and cold homes
-
OFGEM's vulnerable consumer strategy and related initiatives
-
Policy update and the ECO
-
The role of CCGs in addressing the impact of cold homes
-
The role of energy companies in addressing the impact of cold homes
-
Working in local partnerships to address the impact of cold homes.
Cost effectiveness
There was a review of economic evaluations and an economic modelling exercise. See evidence review 2 'Interventions and economic studies', and 'Excess winter deaths: economic modelling report'.
Economic modelling
A model was developed to quantify the changes in indoor environmental conditions associated with energy efficiency interventions (improvements to the building fabric and ventilation control). The model also aimed to explore the potential impact of being able to afford more effective heating due to a fuel subsidy.
An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The results are reported in 'Excess winter deaths: economic modelling report'.
Economic analysis was undertaken from different perspectives including the NHS, NHS and local government, householder and societal. The risks and benefits associated with home energy efficiency measures and a fuel subsidy were quantified using a complex chain of assumed causal linkages. For some links, the evidence base was limited and the results are therefore uncertain.
The model did not address potential non‑health benefits, such as the carbon savings resulting from the modelled changes in energy demand. This means the benefits may have been underestimated.
The results indicated that using home energy efficiency measures, combined with a fuel subsidy, was cost effective. Home energy efficiencies alone were more cost effective than a fuel subsidy. Greater health benefits were achieved when the former were targeted at households with a low standard assessment procedure.
How the PHAC formulated the recommendations
At its meetings between October 2013 and April 2014, the Public Health Advisory Committee (PHAC) considered the evidence, expert report and cost effectiveness to determine:
-
whether there was sufficient evidence (in terms of strength and applicability) to form a judgement
-
if relevant, whether (on balance) the evidence demonstrates that the intervention, programme or activity can be effective or is inconclusive
-
if relevant, the typical size of effect
-
whether the evidence is applicable to the target groups and context covered by the guideline.
The PHAC developed recommendations through informal consensus, based on the following criteria:
-
Strength (type, quality, quantity and consistency) of the evidence.
-
The applicability of the evidence to the populations and settings referred to in the scope.
-
Potential effect on the target population's health, and the size of the effect.
-
Effect on inequalities in health between different groups of the population.
-
Equality and diversity legislation.
-
Ethical issues and social value judgements.
-
Cost effectiveness (for the NHS and other public sector organisations).
-
Balance of harms and benefits.
-
Ease of implementation and any anticipated changes in practice.
If possible, recommendations were linked to evidence statements (see the section on evidence for details). If a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).