Evidence review

Clinical and technical evidence

Regulatory bodies

A search of the Medicines and Healthcare Products Regulatory Agency website revealed no manufacturer Field Safety Notices or Medical Device Alerts for this device. No reports of adverse events were identified from a search of the US Food and Drug Administration (FDA) database: Manufacturer and User Device Facility Experience (MAUDE).

Clinical evidence

Of 9 studies identified, 3 were excluded due to small sample sizes (10 or fewer individuals) and 2 did not report relevant outcomes. The remaining 4 papers are summarised in this briefing.

Lau et al. (2013) carried out a diagnostic accuracy evaluation of the AliveCor Heart Monitor, compared with a standard 12‑lead ECG. Two groups of people aged over 65 were recruited into this study from an Australian treatment centre; firstly a group of 109 people, and secondly a group of 204 people. All of the people in both groups of the study were assessed firstly with a 12‑lead ECG, followed by an AliveCor Heart Monitor reading within 6 hours.

For the first group of people, ECG readings recorded using the AliveCor Heart Monitor were stored in the mobile device, before being interpreted by 2 cardiologists blinded to the 12‑lead ECG results. The AliveCor Heart Monitor ECG readings were also interpreted by automated software (a prototype version of the AliveECG app). The 12‑lead ECG readings were independently interpreted by a third cardiologist, to confirm the results. Using this data, the automated software was improved to achieve better diagnostic accuracy.

The second group of people were analysed in the same way, using 12‑lead ECG followed by AliveCor Heart Monitor ECG. However, for this group the AliveCor Heart Monitor readings were interpreted with the improved automated software, and this updated software was also applied retrospectively to the readings from the first group.

The improved automated software is the basis for the diagnostic element of the current AliveECG app. Before the commercial launch, the software was recoded so that it could operate on all compatible mobile phones and tablets.

One cardiologist, interpreting the AliveCor Heart Monitor readings, made an accurate diagnosis in 93.6% (102/109; 95% confidence interval [CI] 87.2–97.4%) of cases for the first group. The second cardiologist made an accurate diagnosis in 95.4% (104/109; 95% CI 89.6–98.5%) of cases. The results of similar analyses were not provided for the second group. The original automated software had an overall accuracy of 93.6% (102/109; 95% CI 87.2–97.4%). This increased with the updated software to 97.2% (106/109; 95% CI 92.2–99.4%) in the first group, and 97.1% (198/204; 95% CI 93.7–98.9%) in the second. Full study outline and results are reported in table 1.

Lowres et al. (2014) conducted a community‑based, opportunistic screening programme of 1000 people aged 65 years and over, using the AliveCor Heart Monitor to detect AF. The study was set in 10 pharmacies in Sydney, Australia. Diagnoses were based on a cardiologist's interpretation of the AliveCor Heart Monitor ECG recordings. In a subsequent analysis, the AliveECG app was used to retrospectively interpret the original AliveCor Heart Monitor readings to identify AF. The 2 different sets of results were then compared. The cardiologist found the AliveCor Heart Monitor readings sufficient to diagnose 97.5% of people (975/1000), although probable diagnoses were provided for the remaining 2.5% of people (25/1000) based on noise‑reduced AliveCor Heart Monitor readings and 12‑lead ECGs where available. Following the analysis of 996 readings, excluding 4 people with pacemakers, the AliveECG app had 98.5% sensitivity (67/68; 95% CI 92.1%–100%) and 91.4% specificity (849/929; 95% CI 89.4–93.1%), using the cardiologists' diagnosis from the AliveCor Heart Monitor reading as the reference case. Full study outline and results are reported in table 2.

Haberman et al. (2015) measured the accuracy of the AliveCor Heart Monitor in detecting major cardiac abnormalities, including AF, for 3 groups of people; athletes (n=123), young healthy individuals (n =128) and cardiology clinic patients (n= 130). The mean ages across these groups were 19 for the athletes, 25 for the young healthy adults and 59 for the cardiac patients; with an overall mean age of 35 years. All people were assessed with a 12‑lead ECG and the AliveCor Heart Monitor, which were used in immediate succession. In this study, the AliveECG app was not used to diagnose AF, but simply to store readings for clinical interpretation. For this process, 2 electrophysiologists collaboratively interpreted both the 12‑lead ECG and the AliveCor Heart Monitor readings for all groups, with the 12‑lead ECG diagnosis serving as the reference standard. For all groups combined, the electrophysiologists' interpretation of the AliveCor Heart Monitor reading had a sensitivity of 94.4% (17/18; 95% CI 72.7%–99.9%) and a specificity of 99.4% (361/363; 95% CI 98.0%–99.9%). Full study outline and results are reported in table 3.

Tarakji et al. (2015) used the AliveCor Heart Monitor as a monitoring device in people following a cardiac ablation. This study recruited 55 people with a mean age of 60 years, who had the procedure at a US clinic, and were then discharged. They were given both an AliveCor Heart Monitor and a traditional transtelephonic monitor (TTM), and were asked to use them simultaneously when they had symptoms at home. A TTM is an event recorder used for monitoring heart rhythm outside of hospital. These readings are transmitted to a clinician by connecting the device to a telephone line. As in the Haberman et al. (2015) study, the AliveECG app was not used to diagnose AF, but to store ECG readings, which were then emailed to the electrophysiologists for interpretation. The group was divided between 2 electrophysiologists, each of whom independently interpreted both the AliveCor Heart Monitor and the TTM readings for their subgroup, with the TTM readings used as the reference standard. The electrophysiologists' interpretations of the AliveCor Heart Monitor readings had 100% sensitivity (46/46; 95% CI 92.3%–100%) and 97.0% specificity (328/338; 95% CI 94.6%–98.5%), with a Kappa statistic of 0.82 (when AF and atrial flutter were considered to be a single disease state). When asked about their preferences, 92% of the monitored users in the study preferred the AliveCor Heart Monitor to the TTM. The electrophysiologists responsible for interpreting the device readings reported that of the 55 readings, 12.7% (7) AliveCor Heart Monitor recordings were more easily interpreted than TTM recordings; 72.7% (40) of the AliveCor Heart Monitor recordings were as easily interpreted as the TTM recordings; and 14.6% (8) AliveCor Heart Monitor recordings were less easily interpreted than the TTM recordings. Full study outline and results are reported in table 4.

Recent and ongoing studies

Twenty‑three recent, ongoing or in‑development trials on the use of the AliveCor Heart Monitor in the assessment of AF and arrhythmias were identified in the preparation of this briefing. Four of these were listed on the clinicaltrials.gov website:

Eight of the remaining studies are set in the UK and 11 in the USA. Research questions are on the diagnostic accuracy of the AliveCor Heart Monitor (n=8), its use in screening (n=6), use in paediatrics (n=3) and the monitoring of medication (n=2). One additional study was identified that is assessing the use of the AliveCor Heart Monitor in heart failure detection.

Costs and resource consequences

The AliveCor Heart Monitor could be used in primary care as an alternative to other portable ECGs recorders to detect paroxysmal AF. There are several different portable ECGs available to the NHS. These devices, and the number of people in England who were monitored using the device in 2013–14 following an outpatient appointment, are listed below (Health & Social Care Information Centre 2015):

  • ECG loop recorder – 63 people

  • 24‑hour ambulatory ECG – 80,264 people

  • 48‑hour ambulatory ECG – 2863 people

  • Holter extended ECG – 7186 people

  • Cardiomemo ECG monitor – 4973 people.

It is expected that the purpose of portable ECG monitoring in a large proportion of these people was to detect paroxysmal AF. Therefore, the AliveCor Heart Monitor and AliveECG app may be suitable for a large number of people.

Given that the mean cost of ambulatory ECG monitoring is £170, the adoption of the AliveCor may produce savings to the NHS. Savings will occur if the AliveCor Heart Monitor is cheaper than current portable devices, and if it requires less staff time. There is anecdotal evidence to suggest that the adoption of the AliveCor system may lead to fewer GP appointments, because people will feel reassured by the results generated by the AliveECG app. Alternatively, because any AF diagnosis needs to be confirmed with a 12‑lead ECG, the total cost for undertaking 12‑lead ECGs may increase if the AliveCor system increases the rate of detection for paroxysmal AF.

If the AliveCor system leads to the earlier detection of AF, it will impact on resource use, in particular an increase in the number of treatments (such as warfarin) being given to prevent stroke. This increase is expected to be partly offset by lowering the incidence of stroke, and associated treatment costs. The NICE costing report for atrial fibrillation, estimated that the net cost of implementing the NICE guidance on AF would be £88,000 per 100,000 of population.

Health economic evaluation

Lowres et al. (2014) evaluated the cost effectiveness of using the AliveECG app to diagnose AF in an Australian population screening programme. This study assumed AF prevalence to be 4.4% and AF incidence to be 1.4% in a population aged 65 years or over. The study assumed costs of £9.35 per AliveECG screen and £118 for a standard diagnostic assessment for AF, including a GP consultation, specialist consultation and then a 12‑lead ECG reading. From an Australian healthcare perspective, the incremental cost‑effectiveness ratio (ICER) per quality‑adjusted life year (QALY) gained was £2799 (95% confidence interval £754 to £6280). Sensitivity analysis produced a range of £1448 to £7742. The full study outline and results are reported in table 2. (All costs were converted into pounds sterling from Australian dollars, using 2014 purchasing power parities [Organisation for Economic Co‑operation and Development 2015]).

Strengths and limitations of the evidence

Findings from the evaluation of the 4 studies using the QUADAS‑2 tool are reported in table 5.

Table 5 Summary of QUADAS‑2 results for diagnostic evaluation papers

Lau et al. (2013)

Lowres et al. (2014)

Haberman et al. (2015)

Tarakji et al. (2015)

Domain 1: Patient selection

Risk of bias

Unknown

No

Unknown

No

Concern over match to review question

Yes

Yes

Yes

Yes

Domain 2: Index test

Risk of bias

Unclear

Unclear

Unclear

No

Concern over match to review question

No

No

No

No

Domain 3: Reference standard

Risk of bias

Unclear

Yes

Yes

Yes

Concern over match to review question

No

Yes

No

Yes

Domain 4: Flow and timing

Risk of bias

Yes

Yes

No

No

In the study by Lau et al. (2013), reporting of the study population characteristics and recruitment method was limited, and so it was difficult to assess bias and generalisability to the NHS setting. The study used the AliveCor Heart Monitor and an early version of the AliveECG app, which was subsequently recoded for use on a wider range of mobile devices. Recoding of the app may have affected the sensitivity and specificity of the app, but this effect has not been assessed. The study did use an appropriate reference test and the cardiologists were blinded to the 12‑lead results before interpreting the AliveCor readings. However, it also had an interval between the 12‑lead ECG and the AliveCor Heart Monitor reading of up to 6 hours, giving rise to the possibility of change in heart rhythm during the intervening period.

Lowres et al. (2014) carried out a diagnostic accuracy study of the AliveCor Heart Monitor, with readings retrospectively interpreted using the AliveECG app to indicate the presence of AF, and a cost‑effectiveness analysis of their use in an opportunistic screening programme. Recruitment procedures and patient characteristics in the diagnostic study were well reported and a suitable reference test of a 12‑lead ECG was used, but only for patients who tested positive with the AliveCor Heart Monitor. The main weakness was the mean time of 16.6 days between the use of AliveCor and the reference test, giving rise to the possibility of a change in a person's condition in the meantime. Patients did not have access to the full AliveECG app during the study; instead it was used to interpret the readings retrospectively. Therefore, patients could not use the heart monitor and app together, as they would in clinical practice.

For the cost‑effectiveness analysis (Lowres et al. 2014), the Australian costs may not be generalisable to the NHS. The sensitivity analysis is limited, so it is difficult to establish how much uncertainty exists around the results. Furthermore, a full economic evaluation should have compared the use of the AliveCor Heart Monitor with all possible comparators, including manual pulse palpation as a screening option.

In the Haberman et al. (2015) study, the recruitment process and the characteristics of those recruited were not reported in sufficient detail to rule out the possibility of bias. The population recruited to this study may not match those who would be tested in clinical practice in the UK, because it included athletes and healthy young people as well as cardiology clinic patients. Therefore, it is uncertain if aspects of the analysis can be generalised to an NHS setting. It is also unclear whether the cardiologists interpreted the readings from the 2 technologies independently from each other or not, introducing the risk of interpretation bias. Strengths of this study include that an appropriate reference test was used, and that there was no time delay between the 12‑lead ECG and the AliveCor Heart Monitor ECG assessments. However, the study used the AliveECG app to record the AliveCor Heart Monitor ECG, and not to interpret the ECG readings. This means that the results may not be generalisable to how the device would be used in practice. The authors noted that positioning of the body between the 2 readings may influence results.

The major strength of the Tarakji et al. (2015) study is that patients were able to use the AliveCor Heart Monitor multiple times in a community setting. This may better represent how the device could be used in clinical practice in UK, although use of the device in people who have had cardiac ablation may not. Patient characteristics were well reported, the timing of tests was adequate, clinicians were blinded and there was a valid reference test.

All 4 papers used the Kappa statistic to measure the agreement in diagnoses between the AliveCor Heart Monitor and the comparator test. Given the binomial nature of the test, the McNemar's statistic may be a more appropriate measure. Insufficient information was presented in all 4 of the papers for researchers to calculate this independently.