Shared learning database

Leicester Hospitals
Published date:
January 2014

An audiovisual description of the NICE Feverish Illness in Children (CG160) Traffic Light Table was created for Junior Doctors working in Emergency Departments.

A before and after study (202 doctors registered) was performed. Levels of satisfaction were high, attitudes to managing febrile children improved (p<0.0001 across all domains), knowledge scores increased significantly (Z-3.942, p<0.001) and there was a trend to change in behaviours in the application of the guidance.

Audiovisual (i.e video and sound recordings) demonstrating the application of guidelines in practice is a valid methodology to improve guideline implementation.

Guidance the shared learning relates to:
Does the example relate to a general implementation of all NICE guidance?
Does the example relate to a specific implementation of a specific piece of NICE guidance?


Aims and objectives

The focus of the educational intervention was improving knowledge and use of the 'National Institute for Health and Carel Excellence Feverish Illness in Children: Assessment and initial management in children younger than 5 years' Guideline [CG47 and now CG160]. Specifically the guideline contains a Traffic Light Table guiding the approach health care professionals should take when managing a child with a fever of uncertain source. The traffic light table contains 5 domains (Colour, Activity, Respiratory, Hydration and Other) which are derived from clinical examination of the patient. These domains are separated into three risk categories. The guidelines suggest sending patients home (with safety net advice) if in Green, obtaining the opinion of a paediatrician for those in Red and doing either for those in Amber.

The range of signs and symptoms within the table enables specific education on these points to be delivered with the binary decision tree for those in Green and Red allowing objective knowledge testing. Furthermore the diverse and subjective nature of the some of the clinical signs (e.g. 'responds normally to social cues') lends itself very well to teaching via video cases. Videos demonstrate the potentially subjective interpretation of these signs very well and along with features such as chest retractions and reduced GCS are a powerful educational medium. The aim of this intervention was to demonstrate the effectiveness of a video based e-learning package on the practice of junior doctors in the management of feverish children in Emergency Departments.

The objectives of the study were:
i) To evaluate junior doctors' perceptions of confidence, competence and safety in managing feverish children before and after the intervention
ii) To determine knowledge of the facets of Traffic Light before and after the intervention
iii) To determine whether behaviour changed, measured by a validated audit tool, before and after the study
iv) To assess any difference in clinical behaviours in those who performed the tool versus those who didn't
v) To assess satisfaction with the learning resource

Reasons for implementing your project

Fever remains a common, but potentially high risk condition, in which there remains morbidity and mortality as a result of poor practice. The Feverish Illness in Children guideline, available since 2007 is a commonly used tool to aid health care professionals make evidence based decisions. A previous audit performed in the department (winner of a Shared Learning Award 2010-11) had demonstrated an improvement in safety net advice delivery following a focused induction programme. That audit had looked at only one facet of the NICE Feverish Illness in Children Guideline and it was felt that a more comprehensive review of practice was needed. Audiovisual e-learning packages are now relatively easy to produce, can be made freely available via the internet and reduce the need for bespoke education courses in individual departments. The applicability and acceptability of e-learning was to be tested not only in the host institution but in 11 other Emergency Departments in England to demonstrate the potential for widespread implementation.

A pilot study of e-learning package was undertaken in 2011 to debug technical errors and determine correct scoring criteria for the knowledge assessments. The host institution sees over 34000 children a year with a sizeable proportion being undifferentiated febrile illness.

How did you implement the project

An e-learning package was designed with the main learning intervention an interactive traffic light table page (preceded by a short introduction to the background of the guidelines). For each of the 5 domains and the three risk categories of the table there was a video describing the features within that box. For example clicking on 'Activity Green' would reveal a short video sequence demonstrating children responding normally to social cues, being content and smiling etc.

The e-learning package, delivered via a website with password protected access, also contained a pre-learning section consisting of a demographic questionnaire, an attitudinal survey, knowledge testing in the form of MCQs and video based assessments and a small selection of questions on guideline use. This section was mandatory prior to commencement of the traffic light table page. On viewing all the videos on the traffic light table page a post learning section become available containing a repeat of the attitudinal survey and knowledge testing. The website was funded by a NIHR doctoral research fellowship grant as part of research into educational evaluation.

The weblink to the e-learning package was distributed to 12 emergency departments throughout England who had expressed an interest in being part of the evaluation. In order to increase engagement the clinical leads at each of the sites were given an administrative access to the site enabling them to see the progress of the participants at their centre (but only their centre). The project was commenced in August 2012 during the induction period of Junior Doctors as this would enable integration with local learning initiatives. Participants were requested to complete the pre-learning phase of the package within 2 weeks of commencing their Emergency Department attachment.

The major barrier encountered during piloting was the NHS IT infrastructure. Often the videos did not display or took a considerable time period to load. A novel method of streaming via an external server solved most of these issues but serves as an example of the importance of rigorously testing any web-based technological initiatives in NHS as extremely old browsers are still used in some trusts.

Key findings

A total of 202 people expressed an interest in undertaking the intervention, 115 (56.9%) commenced the package, 91 (45.3%) completed the pre-learning and 58 (28.7%) completed the post-learning. Although the fall off was substantial, this was a voluntary initiative as part of a research study, total numbers were healthy and more than adequate for statistical analysis.

In respect of attitudinal change there were separate domains on perceptions of safety, confidence and competence in relation to assessment, investigation and management of febrile children (i.e. 9 questions in total). Questions were graded on a 10-point Likert scale. There was an average 1 point improvement across all the domains in the rating after the package had been completed ((p<0.0001) via Wilcoxon signed ranks test).

For those who completed the post learning the Knowledge Test scores improved significantly in the multiple choice questions from pre-learning result (Z - 3.942, p<0.001).

ROLMA matrices, a previously published methodology of describing adherence to evidence based practice, were used to examine case-notes of children with undifferentiated febrile illness. Data was available for 48 patient prior to the intervention and 20 patients following participants completion of the post-learning following intervention. There was a trend to improved evidence based management in the post learning group but this was not significant (Chi-Square 2.776, p=0.096).

In respect of satisfaction and success with the e-learning system 83.6% (56/67) participants agreed or strongly agreed. Those participants who didn't complete the post learning but did complete the pre-learning were no different in terms of demographics, attitude or knowledge test scores.

Overall data in the lead department demonstrated a rise in admissions of children with undifferentiated fever compared to the previous year but less than the 5 previous years despite a rise in presentations over that time.

Educational initiatives which aid implementation must undergo thorough evaluation as simply obtaining satisfaction scores is insufficient to determine benefit. Although in this case the 'cost-savings' are difficult to determine the improved application of guidance across hundreds of doctors, and potentially thousands with widescale dissemination, will result in the delivery of the patient benefit the guideline was originally intended to realise.

Key learning points

An audio-visual description of NICE guidance translated into attitudinal change, knowledge benefit and a trend towards altered behaviour. Specialist skills are not needed to create such resources as current mobile and tablet technologies are more than adequate to capture quality images. General Medical Council guidance provides an easy to follow template for consenting images and web-based education platforms are freely available at extremely low or no cost. For guidelines which require the application of clinical knowledge in respect of signs and symptoms this methodology proves especially useful.

Clearly defining intended outcomes is vital at the start of nationwide initiatives. Although admission rates rose in the lead institution, potentially attributable to the intervention but obviously affected by confounding factors, it is not clear whether this represents a poorer patient outcome. Only having a small number of children in the red category (via the College of Emergency Medicine Feverish Illness in Children Audit 2012) meant it was difficult to determine whether the increase related to improved safety or unnecessary caution. Only a resource intensive widescale note review would be able to answer this question. The shortly to be released Quality Standard for the NICE Feverish Illness Guideline may aid with additional outcome based metrics.

Engaging clinical leads with a responsibility for education is a useful way to implement guidance. There is an overlap between the skill levels of health care professionals and the application of evidence based practice. Ensuring junior staff are adequately trained in recognising the necessary clinical features provides an opportunity to encourage the implementation of NICE clinical practice guidance.

This system will eventually be made available for general use and will provide a useful tool for any emergency or urgent care centre dealing with febrile infants.

Contact details

Dr Damian Roland
Senior Paediatric Emergency Medicine Registrar
Leicester Hospitals

Primary care
Is the example industry-sponsored in any way?