Appendix C: The evidence

This appendix lists evidence statements from two evidence reviews provided by public health collaborating centres (see appendix A) and links them to the relevant recommendations. (See appendix B for the key to quality assessments.) The evidence statements are presented here without references – these can be found in the full review (see appendix E for details). It also sets out a brief summary of findings from the economic analysis.

Evidence statement number E4d indicates that the linked statement is numbered 4d in review 1 'Preventing unintentional injuries among under-15s in the home. Systematic reviews of effectiveness and cost-effectiveness of home safety equipment and risk assessment schemes'.

Evidence statement number B1 indicates that the linked statement is numbered 1 in review 2 'Barriers to, and facilitators of the prevention of unintentional injury in children in the home: a systematic review of qualitative research.'

The reviews are available online.

Where a recommendation is not directly taken from the evidence statements, but is inferred from the evidence, this is indicated by IDE (inference derived from the evidence) below.

Recommendation 1: economic modelling; IDE

Recommendation 2: evidence statements B4, B5, B6, B8, B9, B11, B12, B13, B14, B15; IDE

Recommendation 3: evidence statements E2a, E2b, E3b, E3c, E3d, E4b, E4c, E4d, E6b, E7b, E9b, B3, B4, B5, B6, B7, B8, B9, B10, B11, B12, B13, B14, B15; IDE

Recommendation 4: evidence statement B3; economic modelling, IDE

Recommendation 5: evidence statements E3e, E3f, E3h, E4b, B2, B9; IDE

Evidence statements

Please note that the wording of some evidence statements has been altered slightly from those in the review team's report to make them more consistent with each other and NICE's standard house style.

Evidence statement E2a

There is inconsistent evidence about impact on injury from one cluster RCT (++) and one controlled before-and-after study (CBA) (+). There is evidence from the better quality cluster RCT that the free supply and installation of smoke alarms had no significant effect on the incidence of fire-related hospitalisations and deaths (rate ratio 1.0 [95% confidence interval {CI} 0.5, 2.0]). However, the CBA study suggests that the free supply and installation of smoke alarms decreased the incidence of fire-related injuries (within-group pre-post intervention comparison: 0.2 [95% CI 0.1, 0.4] for the intervention group and 1.1 [95% CI 0.7, 1.7] for the remainder of the city).

Evidence statement E2b

There is inconsistent evidence about impact on rates of installation of home safety equipment from two cluster RCTs (one [++], one [+]) and one CBA (+). There is evidence from the better quality cluster RCT that the free supply and installation of smoke alarms had no significant effect on the installation or functioning of smoke alarms within households (Rate ratio 1.0 [95% CI 0.4, 2.4]). However, there is evidence from the other cluster RCT that the free supply and installation of smoke alarms had a significant effect on the installation and functioning of smoke alarms: odds ratio (OR) 4.82 (95% CI 3.97, 5.85). The CBA study reported that 51% of intervention households (identified as being without a smoke alarm prior to the intervention) had a correctly installed and functioning smoke alarm at 12 months follow-up.

Evidence statement E3b

There is moderate evidence from three RCTs (one [++] one [+] and one [-]) that the free or discounted supply of smoke alarms in conjunction with safety education increases the rate of installation of these devices.

Evidence statement E3c

There is weak evidence from two RCTs (one [++] and one [+]) about interventions with free or discounted supply of home safety equipment in conjunction with safety education. Outcomes about three types of home safety equipment (buffers, electrical outlet covers and cupboard locks/latches) are reported, showing mixed evidence of effect. Outcomes about other types of home safety equipment (non-slip bathroom items, window locks, fire guards and stair gates) are presented in one report, with only fire guards reported as being more likely to be present post-intervention (based on self-report).

Evidence statement E3d

There is weak evidence from one RCT (++) that the free or discounted supply of a range of safety equipment, in conjunction with safety education, increases the rate of installation of safety equipment as a whole (mean difference [MD] 21.1 [95% CI 13.90, 28.30]) (based on self-report).

Evidence statement E3e

There is strong evidence from four RCTs (two [++], one [+] and one [-]) that the free or discounted supply of a range of safety equipment, in conjunction with safety education, increases knowledgeabout the prevention of poisoning and scalds.

Evidence statement E3f

There is inconsistent evidence from three RCTs (two [++] and one [+]) about how a free or discounted supply of a range of safety equipment, in conjunction with safety education, affects knowledge about: the prevention of fires, falls and wounds.

Evidence statement E3h

There is weak evidence from one RCT (+) that the free or discounted supply of a range of safety equipment, in conjunction with safety education, increases knowledge about the prevention of suffocation.

Evidence statement E4b

There is weak evidence from one RCT (++) that free home safety equipment (or its delivery) and installation with safety education increases the use of smoke alarms at 12 months (OR 1.83 [95% CI 1.33, 2.53]) and 24 months (OR 1.67 [95% CI 1.21, 2.32]). The intervention did not have a statistically significant impact on reducing socioeconomic inequalities in the uptake and continued use (12 months post-intervention) of smoke alarms.

Evidence statement E4c

There is weak evidence from one RCT (++) that showed mixed evidence of effect of the supply of free home safety equipment (or its delivery) and installation with safety education. Outcomes showed no impact on fire guards being fitted and always used after 12 or 24 months, and increased use of stair gates and window locks at 12 months, but not 24 months. The intervention had a statistically significant impact on reducing socioeconomic inequalities in the uptake and continued use (12 months post-intervention) of stair gates.

Evidence statement E4d

There is weak evidence from one RCT (++) that free home safety equipment (or its delivery) and installation with safety education may increase the safe storage at 12 months of cleaning products and sharp objects, but these effects are no longer seen after 24 months for safe storage of sharp objects.

Evidence statement E6b

There is inconsistent evidence from two RCTs (one [+] and one [++]) and one CBA (+) about interventions with a home-risk assessment and free or discounted supply of home safety equipment that included a smoke alarm. Outcomes about the rates of installation of smoke alarms (all self-reported) show mixed evidence of effect(no effect, increased, increased).

Evidence statement E7b

Three studies (one CBA [+] and two before-and-after [BA] [{-}, {+}]) report on the continued presence and use of installed equipment after home-risk assessment and free or discounted supply and installation of home safety equipment. There is mixed evidence about the impact on continued working equipment. One study found that 60% of installed hot water tempering valves remained in situ after 6 to 9 months. One study found significant improvements in the numbers of households with working window guards and fire extinguishers post-intervention. Finally, two studies (one CBA [+] and one BA [+]) showed significantly more smoke alarms installed and working post-intervention (p<0.0001; OR 0.30 [95% CI 0.24, 0.38: showing less alarm absence in the intervention group]).

Evidence statement E9b

There is inconsistent evidence from six robust studies (which use both observed outcome measures and a controlled study design) about the presence of functional smoke alarms. Four suggest that the intervention increased functioning presence (one RCT [+], one CBA [+], one RCT [-] and one CBA [+]) and two suggest that no significant impact was seen on smoke alarms (both RCT [++]).

Evidence statement B2

Three studies (three [-]) found that parents felt there was a lack of information or knowledge about existing policies or support. Examples included lack of knowledge of poison centre telephone number, and lack of 'direct information' on poisoning prevention.

A lack of communication about programmes to install smoke alarms limited uptake, especially for the most high-risk families (those in rented accommodation with a rapid turnover of tenants).

Timing of information was shown to be important. One study found that parents given information in hospital, at the time of a child's birth, did not retain this, while information provided subsequently in a community or physician setting was better retained.

Evidence statement B3

Three studies (all [-]) found that partnerships and collaborations between different service providers facilitated the effectiveness of interventions to reduce unintentional injuries to children in low income communities.

Collaborations perceived as useful included multi-agency partnerships between different agencies, and between agencies and hard-to-reach groups. These collaborations aided the effectiveness of a UK smoke alarm installation programme and a partnership between health officials and low income mothers in home safety visits offering advice and provision of safety equipment

The importance of devising information and advice in ways that suit the target community (in terms of language, style, examples used) was noted in both of these papers dealing with low income populations with many ethnic minorities.

Evidence statement B4

Nine studies (four [-], four [+] and one [++]) found that a major barrier to implementing safety equipment and childproofing a home was living in a home one was not free to modify.

The studies found that mothers particularly found a lack of control over their home environment due to living in rented accommodation, and/or with extended family. In rented accommodation, landlords were reported as unresponsive to requests for installation or maintenance of safety equipment. In extended family homes, often in overcrowded situations, young parents often did not have a say in how the home was arranged. Two studies noted that high turnover of tenants in cheap rented accommodation limited the effectiveness of projects to organise effective installation and maintenance. In two studies, having landlords with the ability and eagerness to make repairs led to more effective interventions.

Evidence statement B5

Four studies (two [-] and two [+]) found that faulty or poor quality equipment was a barrier to interventions to reduce unintentional injuries to children in the home. For example, mothers resorted to taping over electric sockets when safety plugs were not provided or did not work.

The four studies made recommendations for different or better equipment. Studies recommended the provision of tamper-proof smoke alarms with 10- year batteries, alternatives of sprinkler systems for some populations, smoke alarms with longer lasting batteries, help for fitting alarms, or simpler systems for older residents, and more systematic provision of child-resistant containers.

Suspicion by those in vulnerable communities of strangers coming into their homes to assess or install property, and suspicion of 'free' offers, needs to be mitigated in successful interventions.

Evidence statement B6

The two studies on smoke alarm installation (one [+], one [-]) both found that people balance immediate and longer term risks to health and wellbeing when they disable alarms. They were aware that it was less than ideal to disable smoke alarms, but weighed this against other factors, especially the inconvenience and stress of malfunctioning alarms.

Evidence statement B7

Three studies (one [+] and two [-]) based on evaluation of specific interventions all found that training in installation and equipment use/replacement was a facilitator to reducing the incidence of unintentional injuries to children in the home.

Evidence statement B8

Cost emerged as a theme in five of the studies, always as a barrier to reducing accidents to children in the home, or to obtaining help if a child had been injured (two [-], two [+] and one [++]). Three studies found that the perceived cost of installing safety devices or making repairs was a major barrier in the correct use of smoke alarms and in general for safety equipment. However, in one study the provision of free safety equipment, in this case a smoke alarm, led to the equipment being rejected due to suspicions precisely because it was free, which suggests that making equipment or installations totally free may not always be appropriate.

Evidence statement B9

Four studies (one [-], two [+] and one [++]) found that young or poorly educated mothers found it hard to anticipate the child's rate of development in terms of ability to climb, open containers or locks, or light fires. One study, in contrast, found that mothers were good at anticipating developmental milestones and adjusting the home environment in advance of changes, thereby reducing the rate of unintentional injuries in the home (+).

Evidence statement B10

One study (++) found that exposure to a child poisoning incident, either in real life or in the media, increased awareness of that particular danger and was a motivator for implementing safety measures. This suggests that providing information on unintentional poisoning via media outlets might be an effective facilitator in raising awareness of risk.

Evidence statement B11

One study (-) found that adolescent mothers found it hard to deal with issues of blame, oscillating between ideas of the accident-prone child who would have accidents whatever you did, and the negligent adult who was responsible for their child's accidents. The study recommends that care providers approach the topic of injury in a forthright manner when working with adolescent mothers, challenging the idea that injuries are unavoidable while not assigning blame to the mother for injury to the child. It also suggests that 'helping mothers identify risks to their specific child in their specific environment may be the most effective intervention'.

Evidence statement B12

Five studies (two [-], two [+] and one [++]) noted the large and constant amount of effort which mothers put into preventing unintentional injuries in the home as a major facilitator of reducing unintentional injuries in the home. Authors picked up on several main components of this maternal safeguarding work – commonsense safeguarding, constant vigilance and teaching children about safety.

While these maternal safeguarding activities do act as a short-term facilitator to accident reduction, it is important to note that they are time and energy- intensive and that, for this reason, need supplementing with other forms of unintentional injury prevention.

Evidence statement B13

Three studies (two [+] and one [++]) noted cultural practices which, while they may have been adequate safety measures in the parents' culture of origin, were risky in a new cultural context. There were two aspects to this theme; lack of experience of the particular risks of a host context, and lack of understanding by health officials about different child safety norms and expectations in immigrants' cultures.

Mexican mothers in one US study mostly came from rural and semi-rural backgrounds, so had less experience with urban hazards such as multi-storey buildings and hot water taps which could cause falls or scalds. Mexican mothers were also more likely to use Mexican products, which were more likely to come without safety warnings/packaging. Two US studies found significant cultural differences in experience and expectations which led to health visitors classing behaviour as risky because of a lack of understanding of immigrants' perception of safety and risk.

Evidence Statement B14

Five studies (two [-], two [+] and one [++]) found that a major barrier to child safety in the home was mothers' worry that asking about child injury in any context, including unintentional injury prevention, or taking an unintentionally hurt child to hospital, would result in the child being removed/seen as at risk, and they would be accused of abuse or neglect. All of these studies were in the US or Canada and focused on low-income mothers, and additionally, most were adolescent mothers or immigrant mothers.

Evidence Statement B15

Two studies (one [+], one [++]) found that a major barrier to child safety in the home was mothers' lack of autonomy to make household or financial decisions. Policies/interventions might need to reconsider the often automatic targeting of mothers about safety equipment or behaviour, especially in populations where the fathers (or parents-in-law) traditionally make decisions about household purchases.

Cost-effectiveness evidence

To supplement the cost-effectiveness review, two cost–utility analyses were carried out using the same model of the lifetime costs and effectiveness of relevant home safety interventions.

The first analysis compared the supply and installation of free smoke alarms versus no intervention. It found that a free smoke alarm scheme would probably be cost effective (incremental cost-effectiveness ratio [ICER] £23,046). However, there were many uncertainties in this model and it should be noted that the empirical evidence is inconsistent.

The second analysis compared general home safety consultation and equipment provision versus no intervention. (This includes home safety consultation visits, provision of educational materials and advice, as well as the free supply and installation of a range of equipment.)

The sensitivity analyses demonstrate that, from a public sector perspective, cost–utility is likely to be highly dependent on:

  • the proportion of households that participate, the prevalence of existing safety devices in use and the proportion of households that correctly install or use any devices provided

  • how long the device is effective ('functional decay') and whether or not other changes take place in the household which affect its use

  • fixed or overhead costs of programmes relative to the number of households targeted

  • number of people in a household and their age

  • relative reduction in risk due to the device being properly fitted and used (or due to people adopting safer behaviour in the home).

Fieldwork findings

Fieldwork aimed to test the relevance, usefulness and the feasibility of putting the recommendations into practice. PHIAC considered the findings when developing the final recommendations. For details, go to the fieldwork section in appendix B and 'Preventing unintentional injuries in the home among under-15s: providing safety equipment and home-risk assessments: fieldwork report'.

Fieldwork participants who work with children and young people aged under 15 and their parents and carers were very positive about the recommendations and their potential to help prevent unintentional injuries among this group in the home.

However, they thought they represented an ideal scenario and that, currently, it was not feasible to implement some of the advice. Lack of resources was a key issue. In addition, they pointed out that children under 5 have different needs compared with older children – and that these differences should be acknowledged.

Participants wanted to see a greater emphasis on educational interventions that are delivered alongside the installation of home safety equipment. It was also important to overcome any possible stigma that particular households or communities might experience as a result of being prioritised for free safety kit.

Lack of clear lines of responsibility was deemed a key barrier to implementing the recommendations locally. Most participants felt this was due to the lack of national targets and indicators for reducing unintentional injuries among children in the home. Responsibility usually lay with local safeguarding children's boards in the focus group areas, but this was not always the case.

  • National Institute for Health and Care Excellence (NICE)