Clinical and technical evidence

A literature search was carried out for this briefing in accordance with the interim process and methods statement for medtech innovation briefings. This briefing includes the most relevant or best available published evidence relating to the clinical effectiveness of the technology. Further information about how the evidence for this briefing was selected is available on request by contacting mibs@nice.org.uk.

Published evidence

This briefing summarises 5 studies including 1,181 people. The evidence includes 2 comparative studies, 2 observational studies, and 1 bench test study.

The clinical evidence and its strengths and limitations is summarised in the overall assessment of the evidence.

Overall assessment of the evidence

Smart Peak Flow is a tier C digital health technology for active monitoring and self-management based on NICE's evidence standards framework for digital health technologies. The evidence is limited and lacks details of study methods and findings. Two studies included 10 people or fewer. The evidence showed that Smart Peak Flow passed bench tests of accuracy and there may be higher reported numbers of people using Smart Peak Flow compared with mechanical peak flow meters. There were no studies evaluating the effect of the device on clinical or patient-reported outcomes, clinical decision making, or user satisfaction. Further research is therefore needed.

Sakkatos and Williams (2021, brief communication)

Study size, design and location

Cross-sectional comparative study comparing the accuracy of Smart Peak Flow with spirometry in 9 adults who did not have a history of chronic lung conditions or airflow obstruction (mean age 41.9 years, standard deviation [SD] 13.1) in the UK.

Everyone in the study did 3 expiratory tasks using low, medium, and maximal effort. Acceptable limit of agreement (LOA) was set at 40 litres/minute before the start of the study based on a previous validation of peak flow meters.

Intervention and comparator

Smart Peak Flow compared with Piston PDD‑301/sh spirometer.

Smart Peak Flow was tested in series connection with the spirometer. This meant that the same exhale passed through both devices at the same time.

Key outcomes

A Bland–Altman plot showed good agreement between Smart Peak Flow and the spirometer across peak expiratory flow (PEF) values ranging from 210 litres/minute to 626.4 litres/minute. The mean difference between the devices was -0.29 litres/minute with LOA ranging from -30.6 litres/minute to 30.0 litres/minute.

Strengths and limitations

This is a peer reviewed brief communication with limited detail in the reporting of results. It was a pilot study on adults who were healthy. The study set the acceptable LOA at 40 litres/minute. This differs from the American Thoracic Society and European Respiratory Society's recommendation that PEF must be measured with an accuracy of 20 litres/minute. The pilot aimed to recruit 25 people but ended early because of COVID‑19 restrictions. Because of the small sample size, the authors cautioned that no firm conclusion can be made about the in vivo accuracy of Smart Peak Flow. The company was involved in the research.

Antalffy et al. (2020, brief communication)

Intervention

Smart Peak Flow.

Key outcomes

A total of 766 people had the Smart Peak Flow device. Of these, 500 were offered the device by their healthcare professional during an outpatient appointment, with 133 (27%) of these patients downloading the app and recording at least 1 PEF measurement. Additionally, 266 people bought the device themselves and recorded at least 1 PEF reading. After 3 months, 32% of the 399 people using the device recorded their PEF daily and 63% measured their PEF at least twice a week. After 6 months, 28% of these people recorded their PEF daily, while 67% measured their PEF at least once a week. The authors stated that adherence is higher than traditional PEF monitoring as reported in Garrett et al. (1994), which found that 16% of people measured PEF daily at baseline which dropped to 6% after 9 months.

Strengths and limitations

This is a peer reviewed brief communication that reported real-world data of app use collected by the company. There was no information on peoples' demographics, health conditions, or treatment. The study is also limited in its reporting of results. The authors report adherence as the proportion of people who downloaded and used the app (n=399), rather than the proportion of people who had the device (n=766). This therefore makes it difficult to compare the reported levels of adherence with traditional PEF monitoring in other studies. Further research is therefore needed comparing the use of Smart Peak Flow with traditional PEF monitoring. The authors stated that no research evidence is available on factors related to usability, acceptability, and adherence of the device.

Coelho et al. (2020, abstract)

Intervention and comparator

Smart Peak Flow and Mini‑Wright peak flow meter.

Everyone recorded PEF with both devices 6 times daily for 5 days, with 3 readings taken per session for each device. The meter used for the first reading alternated between and within each session. The mean within-session coefficient of variant for each person and meter was calculated.

Key outcomes

A total of 336 readings were recorded using Smart Peak Flow and 347 using Mini‑Wright. This difference in the number of recordings was because of technical issues with Smart Peak Flow, including the session timing out and not enough ambient light. The mean coefficient of variation for Smart Peak Flow was 15.7 compared with 3.12 for Mini‑Wright. The authors suggested several reasons for the higher variation of Smart Peak Flow. These included variation in PEF in some people, environmental factors such as lighting or background noise, and the phone and connection used (headphone jack or adapter).

Strengths and limitations

This was a small evaluation in people who are healthy. Mini‑Wright is an appropriate comparator. The study was reported in abstract only and therefore lacked details on study methods and findings. Several factors were proposed to account for the higher variation in readings by Smart Peak Flow, but these were not assessed in this study with further research recommended.

Sakkatos et al. (2020, abstract)

Intervention

Smart Peak Flow.

An algorithm based on artificial neural networks used daily PEF readings to make predictions of the colour coded PEF zone of the next day.

Key outcomes

Daily PEF readings from 21 consecutive days significantly predicted the next day PEF zone (F1 score 0.72, 95% confidence interval [CI] 0.70 to 0.76; p<0.01). Daily PEF recordings could significantly differentiate PEF zones, with receiver operating characteristic area under the curve estimated as 0.87 (95% CI 0.85 to 0.92) for green, 0.79 (95% CI 0.76 to 0.85) for yellow, and 0.90 (95% CI 0.88 to 0.95) for red zones. The authors concluded that Smart Peak Flow could be an early indicator of asthma worsening.

Strengths and limitations

This study was also reported in abstract in Sakkatos et al. (2020). Both abstracts lacked details of study methods. Data was collected retrospectively, but it was not stated if this was real-world data from the app. There was no evidence on the use of the algorithm and its effect on clinical decision making or outcomes. The research was done by the company.

VanZeller al. (2019)

Study size, design and location

Bench test study on 9 peak flow meters or spirometers with CE marking that were available in Europe.

Accuracy of the devices was tested using ISO 23747:2015 and a pulmonary waveform generator. Three tests were done on each device to determine:

  • error, repeatability and resistance to peak expiratory output

  • error at body temperature, ambient pressure and saturated conditions

  • frequency response.

Intervention and comparators

Smart Peak Flow and 8 peak flow meters or spirometers (Air Smart Spirometer, AirZone, eMini‑Wright, Medi, Mini‑Wright, MIR Smart One, Philips PersonalBest, Vitalograph).

Key outcomes

Smart Peak Flow and Mini‑Wright were the only devices to pass all 3 tests of accuracy. The authors concluded that these were the only 2 devices that complied with the minimum performance requirements outlined in ISO 23747:2015.

Strengths and limitations

This is a peer reviewed study that objectively tested several peak flow meters according to established accuracy standards. Tests were done in a laboratory using a pulmonary waveform generator and therefore may not reflect accuracy when used by people in real-world settings. The study reported findings as pass or fail and did not include data on the measurements taken or deviation of the readings. The study was funded and done by the company.

Sustainability

The company claims the technology uses fewer raw materials than mechanical peak flow meters because it is smaller and lighter. There is no published evidence to support these claims.

Recent and ongoing studies

Pilot voice sample collection from people with asthma. This is a prospective observational study assessing whether changes in voice quality recorded using the Smart Asthma app are indicative of change in asthma control. ClinicalTrials.gov identifier: NCT04799678. Status: not yet recruiting. Indication: chronic asthma. Device: Smart Asthma app. Last updated: March 2021. Country: not stated.

One clinical expert who commented on this briefing reported that Smart Peak Flow is being evaluated in people with severe refractory asthma who are having dupilumab. This study is being done by researchers at the University of Dundee. There was no record of this study in the clinical trials register.