2 Evidence gaps
This section describes the evidence gaps, why they need to be addressed and their relative importance for future committee decision making.
The committee will not be able to make a positive recommendation without the essential evidence gaps (see section 2.1 on evidence that is essential to allow the committee to make a recommendation in future) being addressed. The company can strengthen their evidence base by also addressing as many other evidence gaps (see section 2.2 on evidence that further supports committee decision making) as possible. Addressing these will help the committee to make a recommendation by better understanding the patient or healthcare system benefits of the technology.
2.1 Evidence that is essential to allow the committee to make a recommendation in future
Referrals to CT scan
The review of chest X-rays determines whether people will proceed to have a chest CT scan. This may be influenced by the assessed technologies. To understand their impact on resource use, it is necessary to understand how the software affects the proportion of people with chest X-rays who are referred on to chest CT scan and the overall number of referrals to CT scan.
Time to chest X-ray review, CT referral, and diagnosis
An advantage of the software may be in supporting quicker review and reporting of chest X-rays, leading to quicker referral to chest CT scan and diagnosis. An additional advantage of using artificial intelligence (AI)-derived software to interpret images is that the algorithm can prioritise the images for the reviewer if abnormal findings are detected.
This can be assessed through measuring:
-
time from chest X-ray to report
-
average number of chest X-rays assessed per reviewer per day.
For those who are referred on to CT scan, it is important to assess how the software affects the time from receipt of chest X-ray to CT scan.
The benefits of the technology when used by less experienced trainee radiologists and reporting radiographers should also be considered when collecting data for this outcome.
Diagnostic accuracy and technical failure rates
AI-derived software may improve a chest X-ray reviewer's ability to identify images with features suggesting lung cancer. Improving sensitivity to abnormalities could result in earlier diagnosis, but unnecessary referrals to CT incur costs to the NHS and may cause anxiety.
Information on diagnostic accuracy (positive predictive value) could be assessed by measuring the proportion of abnormal chest X-rays that are confirmed as abnormal by the chest CT scan. Information on the number of cancers detected and missed, and stage of cancer at diagnosis, is also important to inform the economic model.
Technical failure and rejection rates should also be collected.
2.2 Evidence that further supports committee decision making
Software impact on healthcare costs and resource use
Further information on resources needed to implement these technologies in clinical practice is important for use in economic modelling to inform decision making. For example, training and software implementation costs.
Evidence in populations with underlying conditions that could yield images that are challenging to interpret
There is currently a lack of evidence for these technologies in people with conditions that may result in images that are challenging to interpret. Lung nodules and other abnormalities may be difficult to recognise in people with conditions such as asthma, scoliosis, obesity, or chronic obstructive pulmonary disease.
Clinician experience of using AI-derived software
Information on ease of use and acceptability of the software by clinicians is needed. This may include experiences around implementing the technology and any improvements in the delivery of diagnostic services, particularly around accuracy of the technology in identifying abnormalities, appropriateness of image triage, and the impact on speed of review and reporting.