Advice
Clinical and technical evidence
Clinical and technical evidence
A literature search was carried out for this briefing in accordance with the interim process and methods statement for medtech innovation briefings. This briefing includes the most relevant or best available published evidence relating to the clinical effectiveness of the technology.
Further information about how the evidence for this briefing was selected is available on request by contacting mibs@nice.org.uk.
Published evidence
There were 2 studies identified by the literature search that are summarised in this briefing. They include 48,434 people aged 50 and over with abdominal or chest CT images.
The clinical evidence and its strengths and limitations is summarised in the overall assessment of the evidence.
Overall assessment of the evidence
The evidence base is limited. Currently there are only 2 published studies on the use of the HealthVCF AI algorithm: 1 full‑text publication and 1 conference abstract. Both are retrospective clinical validity cohort studies that report on the performance of the AI algorithm in either detecting vertebral compression fractures or predicting fracture risk (with 2 other products in development). None of the studies report the effect of using the AI technology on patient outcomes or clinical management outcomes.
Only 1 of the studies used the commercially available software (HealthVCF) alone to incidentally detect vertebral compression fractures (Gunasingham et al. 2020). This study reported the specificity and sensitivity of the algorithm in detecting fractures from previous CT scan images, using expert radiologist review as a reference standard. The study reported a specificity of 94% and a sensitivity of 59%. The sensitivity of routine reporting (that is, whether fracture risk had been noted in the original reports of the CT scans) was reported to be 38%. Results were reported in a conference abstract only, which lacked sufficient methodological detail to fully assess the study.
The other study used HealthVCF with 2 other in‑development products to predict fracture risk (Bone Health Solution). In Dagan et al. (2020), the Bone Health Solution algorithm was developed and used to predict fracture risk based on historic data from previous chest or abdominal CT scans of 48,227 people aged 50 to 90. Predictions were then compared with outcomes of major osteoporotic fractures over a 5‑year follow-up period. Study results report non‑inferior performance to FRAX without bone density (FRAXnb) in predicting both major osteoporotic fracture and hip fracture risk. Results reported that Bone Health Solution algorithm could predict major osteoporotic fractures with 66.5% sensitivity and 64.7% specificity, and hip fractures with 92.6% sensitivity and 36.9% specificity.
There is limited evidence on HealthVCF for detecting spinal bone fractures on CT scans. The evidence base would benefit from further evidence using prospective data collection to fully establish the real‑world clinical performance of the HealthVCF AI algorithm. Ideally this evidence would be UK‑based. Also, studies evaluating the effect of using the technology on patient outcomes and changes in clinical management would be useful, such as referrals to osteoporotic care.
Dagan et al. (2020)
Study size, design and location
Retrospective clinical validity cohort study assessing HealthVCF with 2 other in-development algorithms (for CT-derived simulated DXA T-scores and evaluated lumbar trabecular density), in predicting 5-year fracture risk. The study was done in Israel and included 48,227 people aged 50 to 90 who had previously had abdomen or chest CT scan as of 2012. Predictions based on historic CT data were compared with outcomes of hip fractures and major osteoporotic fractures during the 5‑year follow-up period (2012 to 2017). The prediction performance of the algorithm, alone and with FRAXnb, was compared with that of FRAXnb.
Intervention and comparator
Intervention – HealthVCF with 2 other in-development algorithms (for CT‑derived simulated DXA T‑scores and evaluated lumbar trabecular density), alone and in combination with FRAXnb.
Comparator – FRAXnb.
Key outcomes
The area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) measures of the intervention for predicting major osteoporotic fractures were 70.9%, 66.5%, 64.7%, 18.4% and 94.2%, respectively (at a cut‑off of 10.3 absolute risk). They were 76%, 92.6%, 36.9%, 5.7% and 99.2%, respectively for predicting hip fractures (at a cut‑off of 1.7 absolute risk). The study reported that the intervention had comparable discriminatory performance to FRAXnb, both for major osteoporotic fractures (AUC +1.9%, sensitivity +2.4%, PPV +0.7%) and hip fractures (AUC +0.9%, sensitivity +1.5%, PPV +0.1%). The study reported that when the intervention was used with FRAXnb, the combined prediction tool further improved predictive performance for major osteoporotic fractures (AUC +3.2%, sensitivity +3.3%, PPV +0.9%) and hip fractures (AUC +2.1%, sensitivity +1.5%, PPV +0.1%) compared with FRAXnb alone.
Strengths and limitations
The study included a large population and used data from a large healthcare organisation across multiple centres. Prediction performance was compared with that of FRAXnb, a tool with accepted clinical utility used routinely in NHS practice. Only 5.2% of the potential study population aged 50 to 90 had a relevant CT scan available for analysis. The study population had relatively older ages; a higher proportion of men; and higher rates of previous major osteoporotic fracture, secondary osteoporosis, and glucocorticoids use compared with the baseline population without a CT scan. Scans that were technically inadequate for interpretation by the intervention were excluded from the analysis, but details of how inadequacy was determined was not clearly stated. The technology used in this study comprises HealthVCF with 2 other products that are in development and not yet commercially available. The study was funded by the company.
Gunasingham et al. (2020)
Study size, design and location
A retrospective clinical validity cohort study assessing sensitivity and specificity of HealthVCF for detecting incidental vertebral compression fractures in 207 people over 50 with abdominal or chest CT scans compared with routine reporting in Australia (104 people with abdominal CT scans and 103 people with chest CT scans).
Intervention and comparator
Intervention – HealthVCF.
Comparator – routine reporting (that is, whether fracture risk had been noted in the original reports of the CT scans).
Reference standard – expert radiologist review.
Key outcomes
The expert radiologist reported the prevalence of vertebral fractures to be 16.5%. The sensitivity of HealthVCF in detecting vertebral fractures was 59% with a specificity of 94%. The PPV was 67% with a NPV of 92%. The sensitivity of routine reporting was 38%.
Strengths and limitations
The study reports appropriate diagnostic outcome measures, a relevant comparator, and the care setting and application of the technology is comparable to the UK. The expert radiologist was blind to the routine report and algorithm findings. The study is reported as an abstract and methodological detail is limited.