Clinical and technical evidence
A literature search was carried out for this briefing in accordance with NICE's interim process and methods statement. This briefing includes the most relevant or best available published evidence relating to the clinical effectiveness of the technology. Further information about how the evidence for this briefing was selected is available on request by contacting firstname.lastname@example.org.
This briefing summarises 1 study published as a preprint (icolung, icometrix) and 1 UK study into Veye Chest (Aidence) published as 2 conference posters.
The icolung study included automatic segmentation of lung lesions in the chest CT scans of 17 confirmed or suspected people with COVID-19 from Europe, South America and a public dataset. The Veye Chest study included 349 chest CT examinations in 324 people and assessed accuracy of lung nodule segmentation and growth tracking. Three ongoing studies into Veolity (MeVis) are described later in the briefing.
The clinical evidence and its strengths and limitations are summarised in the overall assessment of the evidence.
Two retrospective studies into artificial intelligence (AI) for chest CT are summarised. Neither was published in full. One relatively small study of 17 people with suspected COVID-19 compared the performance of AI algorithms in icolung with chest radiologists to identify abnormal scans and potentially significant lesions in COVID-19. The study included a number of non-UK datasets, therefore the generalisability of the findings to the NHS is unclear. Another study into Veye Chest compared the performance of AI software with chest radiologists for segmentation and growth assessment of lung nodules in adults (aged 50 to 74, who currently smoke or have a smoking history, or are reported to have radiological evidence of pulmonary emphysema). This study was published as 2 conference poster presentations, which limits the amount and quality of information available. The study was UK-based, which may help generalisability to the NHS.
There is limited evidence into AI for chest CT. More studies of the impact on clinical management and outcomes would help provide evidence to support clinical adoption. Ideally, studies would be UK-based and prospective in design.
A retrospective study into 17 people with suspected COVID-19. Images were from European and South American centres and a public dataset.
The segmentation accuracy of 12 algorithms (incorporated into icolung) was compared with radiologists. The segmentation accuracy of the AI algorithm was calculated as the dice coefficient (statistical method used to judge the similarity of 2 samples). COVID-19 status was confirmed by laboratory testing.
Dice scores for lung segmentation, binary lesion segmentation and multiclass lesion segmentation were 0.982, 0.724 and 0.469, respectively. The AI algorithm performed binary lesion segmentation (identifying abnormalities) with an average volume error that was better than visual assessment by human readers. The algorithm performed least accurately in multiclass lesion segmentation (including identifying consolidation and ground glass opacity, which the authors note are important lesions in COVID-19).
The study is very small (n=17) and published as a preprint (rather than as a formally peer reviewed paper). There is no information on the size or composition of the training datasets. A multicentre dataset was used and it is unclear how this generalises to an NHS setting. The authors note that a version of the software in this study is available as icolung in the US and Europe. It is unclear how the software in this study differs from icolung.
A retrospective study of 337 chest CT scans from 314 people (173 women, 164 men) with a total of 470 pulmonary nodules included (Murchison et al. 2019a and Murchison et al. 2019b). Images were from 1 UK regional healthcare database. Inclusion criteria were people aged between 50 and 74, who currently smoke or have a history of smoking, or are reported to have radiological evidence of pulmonary emphysema.
The segmentation accuracy of Veye Chest was compared with 3 experienced chest radiologists. The segmentation accuracy of readers was calculated as the dice coefficient between each radiologist's segmentation and the segmentations of the others and subsequently averaged (inter-reader dice coefficient).
When looking at nodules visible on sequential scans, nodule registration from the AI was scored as either a true positive-pair if the detected registration was included in the nodule registration reference standard, or as a false positive-pair. The mean discrepancy between growth percentages determined by radiologists and AI alone was calculated.
The software was able to successfully segment 95% of the total 428 nodules between 3 mm and 30 mm. The performance of the AI software for segmenting pulmonary nodules on chest CT was comparable with that of experienced thoracic radiologists.
The mean growth percentage of lung nodule pairs was similar between readers and by standalone AI.
This study was done in a UK setting. The Fleischner Society's definition for pulmonary nodules was broadly used during this study. Training data were from people aged 50 to 74 in a registry of people who smoke. It is unclear how these data will apply to other patient populations.
The study is presented as a conference poster presentation. The study population is relatively small.
Five recent and ongoing studies involving Veolity (MeVis) were identified in the development of this briefing. These included the following 3 studies that are registered with a clinical trials database:
International Lung Screen Trial. ClinicalTrials.gov identifier: NCT02871856. Status: active, not recruiting. Indication: people who may be at increased risk of lung cancer because of age and smoking history. Study completion date: December 2023.
Yorkshire Lung Screening Trial. ISRCTN42704678. Status: Enrolling by invitation. Indication: people who may be at increased risk of lung cancer because of age and smoking history. Study completion date: July 2024.
The SUMMIT Study: a cancer screening study. ClinicalTrials.gov identifier: NCT03934866. Status: enrolling by invitation. Indication: people who may be at increased risk of lung cancer because of smoking history. Study completion date: August 2030.