Clinical and technical evidence

A literature search was carried out for this briefing in accordance with the published process and methods statement. This briefing includes the most relevant/best available published evidence relating to the clinical effectiveness of the technology. The literature search strategy, evidence selection methods and detailed data extraction tables are available on request by contacting mibs@nice.org.uk.

Published evidence

Four studies (Mitchell et al. 2016; Prytherch 2006; Pullinger 2016; Schmidt 2015) relevant to the use of VitalPAC for recording and calculating early warning scores (EWS) are summarised in this briefing (112,218 adults and 21 nurses in total).

Mitchell et al. 2016 evaluated the implementation of a quality improvement initiative, which included using VitalPAC, on reducing outbreaks of nosocomial viral gastroenteritis. The study was a single-centre before-and-after retrospective study and also compared the centre outbreaks with those at regional and national level.

Prytherch et al. 2006 carried out a classroom exercise comparing processing of clinical datasets using pen and paper with VitalPAC. Using VitalPAC reduced errors and the number of instances in which errors affected clinical decisions.

Pullinger et al. 2016 carried out a before-and-after feasibility study on using VitalPAC instead of pen and paper to record EWS in the emergency department. Although results showed that more EWS were recorded when VitalPAC was used, fewer data from the patients included in the pen-and-paper stage were available for analysis, possibly introducing bias.

Schmidt et al. 2015 used statistical process-control charts to study trends in in-hospital seasonally-adjusted mortality associated with the 56 diagnoses used to monitor UK-hospital performance over the period in which VitalPAC was implemented in 2 UK general hospitals. There was some evidence that seasonally-adjusted mortality for these conditions was lower in both hospitals and within medicine, surgery and trauma, and orthopaedics grouping after VitalPAC was introduced. Other changes over the study period could have influenced the findings and total mortality and deaths in the immediate period after discharge were not considered.

Table 1 summarises the clinical evidence as well as its strengths and limitations.

Table 1: Summary of selected studies

Study

Details of intervention and comparator

Outcomes

Strengths and limitations

Mitchell et al. 2016

Reducing the number and impact of outbreaks of nosocomial viral gastroenteritis: time-series analysis of a multidimensional quality improvement initiative.

Before-and-after retrospective observational study

Single centre

UK

Sample size not reported

Pre-intervention practice regarding norovirus infection was compared with a multi-dimensional quality-improvement initiative which included VitalPAC.

The annual number of outbreaks dropped after the quality-improvement initiative when compared with the period before the initiative and with regional and national data.

The quality-improvement initiative included the introduction of 3 other interventions: a public health education campaign; targeted norovirus care and control bundle; and the opening of a new hospital block. The interventions were not introduced in a controlled manner and so it is not possible to quantify the influence of VitalPAC on outbreak reduction.

Prytherch et al. 2006

Calculating early warning scores – a classroom comparison of pen and paper and hand-held computer methods.

Single-centre classroom comparison of processing 5 clinical data sets

UK

21 nurses

VitalPAC was compared with pen and paper. Participants were asked to rate their preference for pen and paper or VitalPAC using a 5-point Likert scale.

Fewer errors in input and charting occurred with VitalPAC resulting in fewer incorrect EWS and fewer consequent incorrect clinical actions.

Data entry with VitalPAC took less time.

Participants preferred VitalPAC.

Although the study quantified errors and times associated with processing clinical data, results from a classroom situation may not be directly applicable to the hospital environment. The design could have been strengthened by randomising participants and the order in which the datasets were processed.

Pullinger et al. 2016

Implementing an electronic observation and early warning score chart in the emergency department: a feasibility study.

Patients over 16 years from the 'majors' area of the emergency department: stage 1 (paper charts) 3,219 patients; stage 2 (VitalPAC) 3,352 patients.

Before-and-after study

Single emergency department

UK

Stage 1: paper charts.

Stage 2: VitalPAC.

More patients in the VitalPAC stage had accurate EWS recorded in available documentation. There were no statistically significant differences in mortality, length of hospital admission, transfers to the resuscitation room or intensive therapy unit, and cardiopulmonary events. People whose records were processed using VitalPAC were slightly more likely to be admitted.

A weakness was data availability which was poorer in stage 1 than stage 2. This finding was actually part of the benefit of introducing VitalPAC – more patients had more observations done, and those data could be easily retrieved. A before-and-after single-centre design cannot adequately adjust for changes over the period studied other than the intervention measured.

Schmidt et al. 2015

Impact of introducing an electronic physiological surveillance system on hospital mortality.

Number of patients admitted pre- and post-implementation:

Hospital 1: 27,959; 29,676.

Hospital 2: 21,771; 26,241.

Number of deaths before discharge pre- and post-implementation:

Hospital 1: 2,168; 1,904.

Hospital 2: 1,648; 1,614.

Before-and-after study with analysis of seasonally-adjusted monthly mortality rate using statistical process-control charts.

2 acute hospitals

UK

VitalPAC electronic physiological surveillance system was compared with previous paper-based EWS systems.

The monthly number of observation sets was used to track implementation of VitalPAC.

Results were considered for medicine, surgery and trauma, and orthopaedics specialty groups.

In both hospitals crude mortality for 56 diagnosis groups used by the UK NHS to monitor and compare hospital performance at the point of hospital discharge fell after implementation and seasonally-adjusted mortality fell after VitalPAC was fully implemented. In both hospitals, seasonally-adjusted mortality showed some evidence of special cause variation with lower death rates after VitalPAC implementation. Within both hospitals, results for medicine, surgery and trauma, and orthopaedics were consistent with overall results.

Strengths included that 2 unselected hospital populations were studied; mortality data were from an external source; mortality was seasonally adjusted.

Limitations included that there was no comparison with control hospitals retaining paper-based EWS systems (other changes may have influenced hospital death rates over the period observed); total mortality rates were not used; mortality rates were not adjusted for age, sex or other factors, apart from seasonality; relatively short period observed after implementation; mortality at the point of hospital discharge may have missed the effect of later deaths.

Strengths and limitations of the evidence

There is limited evidence showing that using the VitalPAC system may be associated with improved recording and calculating of EWS and patient outcomes.

Given the uncontrolled before-and-after design of the studies identified, it is possible other factors may have influenced the findings. A feasibility study in an ED showed an improvement in the recording of EWS with VitalPAC rather than pen and paper, as did a classroom study. The study design could have been strengthened by randomising the order of pen-and-paper and VitalPAC processing as well as the order of dataset processing. Real-life experience in the hospital environment might differ from a study environment. A reduction in the outbreak numbers of nosocomial norovirus infections was reported, although the decrease has to be attributed to a combination of the quality improvement initiatives and not exclusively to the implementation of VitalPAC.

Recent and ongoing studies

No ongoing or in-development trials were identified.