An audiovisual description of the NICE Feverish Illness in Children (CG160) Traffic Light Table was created for Junior Doctors working in Emergency Departments.
A before and after study (202 doctors registered) was performed. Levels of satisfaction were high, attitudes to managing febrile children improved (p<0.0001 across all domains), knowledge scores increased significantly (Z-3.942, p<0.001) and there was a trend to change in behaviours in the application of the guidance.
Audiovisual (i.e video and sound recordings) demonstrating the application of guidelines in practice is a valid methodology to improve guideline implementation.
Aims and objectives
The range of signs and symptoms within the table enables specific education on these points to be delivered with the binary decision tree for those in Green and Red allowing objective knowledge testing. Furthermore the diverse and subjective nature of the some of the clinical signs (e.g. 'responds normally to social cues') lends itself very well to teaching via video cases. Videos demonstrate the potentially subjective interpretation of these signs very well and along with features such as chest retractions and reduced GCS are a powerful educational medium. The aim of this intervention was to demonstrate the effectiveness of a video based e-learning package on the practice of junior doctors in the management of feverish children in Emergency Departments.
The objectives of the study were:
i) To evaluate junior doctors' perceptions of confidence, competence and safety in managing feverish children before and after the intervention
ii) To determine knowledge of the facets of Traffic Light before and after the intervention
iii) To determine whether behaviour changed, measured by a validated audit tool, before and after the study
iv) To assess any difference in clinical behaviours in those who performed the tool versus those who didn't
v) To assess satisfaction with the learning resource
Reasons for implementing your project
A pilot study of e-learning package was undertaken in 2011 to debug technical errors and determine correct scoring criteria for the knowledge assessments. The host institution sees over 34000 children a year with a sizeable proportion being undifferentiated febrile illness.
How did you implement the project
The e-learning package, delivered via a website with password protected access, also contained a pre-learning section consisting of a demographic questionnaire, an attitudinal survey, knowledge testing in the form of MCQs and video based assessments and a small selection of questions on guideline use. This section was mandatory prior to commencement of the traffic light table page. On viewing all the videos on the traffic light table page a post learning section become available containing a repeat of the attitudinal survey and knowledge testing. The website was funded by a NIHR doctoral research fellowship grant as part of research into educational evaluation.
The weblink to the e-learning package was distributed to 12 emergency departments throughout England who had expressed an interest in being part of the evaluation. In order to increase engagement the clinical leads at each of the sites were given an administrative access to the site enabling them to see the progress of the participants at their centre (but only their centre). The project was commenced in August 2012 during the induction period of Junior Doctors as this would enable integration with local learning initiatives. Participants were requested to complete the pre-learning phase of the package within 2 weeks of commencing their Emergency Department attachment.
The major barrier encountered during piloting was the NHS IT infrastructure. Often the videos did not display or took a considerable time period to load. A novel method of streaming via an external server solved most of these issues but serves as an example of the importance of rigorously testing any web-based technological initiatives in NHS as extremely old browsers are still used in some trusts.
In respect of attitudinal change there were separate domains on perceptions of safety, confidence and competence in relation to assessment, investigation and management of febrile children (i.e. 9 questions in total). Questions were graded on a 10-point Likert scale. There was an average 1 point improvement across all the domains in the rating after the package had been completed ((p<0.0001) via Wilcoxon signed ranks test).
For those who completed the post learning the Knowledge Test scores improved significantly in the multiple choice questions from pre-learning result (Z - 3.942, p<0.001).
ROLMA matrices, a previously published methodology of describing adherence to evidence based practice, were used to examine case-notes of children with undifferentiated febrile illness. Data was available for 48 patient prior to the intervention and 20 patients following participants completion of the post-learning following intervention. There was a trend to improved evidence based management in the post learning group but this was not significant (Chi-Square 2.776, p=0.096).
In respect of satisfaction and success with the e-learning system 83.6% (56/67) participants agreed or strongly agreed. Those participants who didn't complete the post learning but did complete the pre-learning were no different in terms of demographics, attitude or knowledge test scores.
Overall data in the lead department demonstrated a rise in admissions of children with undifferentiated fever compared to the previous year but less than the 5 previous years despite a rise in presentations over that time.
Educational initiatives which aid implementation must undergo thorough evaluation as simply obtaining satisfaction scores is insufficient to determine benefit. Although in this case the 'cost-savings' are difficult to determine the improved application of guidance across hundreds of doctors, and potentially thousands with widescale dissemination, will result in the delivery of the patient benefit the guideline was originally intended to realise.
Key learning points
Clearly defining intended outcomes is vital at the start of nationwide initiatives. Although admission rates rose in the lead institution, potentially attributable to the intervention but obviously affected by confounding factors, it is not clear whether this represents a poorer patient outcome. Only having a small number of children in the red category (via the College of Emergency Medicine Feverish Illness in Children Audit 2012) meant it was difficult to determine whether the increase related to improved safety or unnecessary caution. Only a resource intensive widescale note review would be able to answer this question. The shortly to be released Quality Standard for the NICE Feverish Illness Guideline may aid with additional outcome based metrics.
Engaging clinical leads with a responsibility for education is a useful way to implement guidance. There is an overlap between the skill levels of health care professionals and the application of evidence based practice. Ensuring junior staff are adequately trained in recognising the necessary clinical features provides an opportunity to encourage the implementation of NICE clinical practice guidance.
This system will eventually be made available for general use and will provide a useful tool for any emergency or urgent care centre dealing with febrile infants.