2 Evidence gaps
This section describes the evidence gaps, why they need to be addressed and their relative importance for future committee decision making.
The committee will not be able to make a positive recommendation without the essential evidence gaps (see section 2.1 on essential evidence for future committee decision making) being addressed. The companies can strengthen the evidence base by also addressing as many other evidence gaps (see section 2.2 on evidence that further supports committee decision making) as possible. Addressing these other evidence gaps will help the committee to make a recommendation by ensuring it has a better understanding of the patient or healthcare system benefits of the technology.
2.1 Essential evidence for future committee decision making
Time saving and resource use
To estimate the time-saving benefits of the technologies, it is important to measure the total time needed for contouring. This is because a reduction in the time taken to review radiotherapy contours could reduce patient waiting times by, allowing healthcare professionals to review more patients in a day. In addition, the evidence generated should capture the perceived impact on time saving and other factors that may influence this.
To understand how the software affects resource use, evidence should be generated on the time spent by healthcare professionals on reviewing and editing the software's contours compared with manual or atlas-based contouring. Given that there is substantial variation in time saving between different anatomical structures, information should be collected and presented for each anatomical structure.
Further information is needed on the NHS pay bands of the reviewing healthcare professionals to inform the cost calculations. To estimate total cost, it is also important to understand the cost of training, implementing the software and related administration.
Organ delineation and acceptability of the contour
Acceptability of the software's outputs can be measured using a Likert-type scale considering the number of edits by the reviewing healthcare professional. Fewer or only minor edits would suggest greater acceptability. To complement this, information about the settings included in the software and the guidelines they align with should be included.
To further understand how the technology provides contours that are clinically acceptable, it is essential to capture the experiences and opinions of healthcare professionals during data collection.
2.2 Evidence that further supports committee decision making
Adverse effects of treatment
Artificial intelligence (AI) contouring could improve patient outcomes by more accurately delineating organs, leading to more accurate treatment and fewer adverse effects of treatment. Information on adverse effects of treatment and dosimetric analyses comparing the AI technologies with manual or atlas-based contouring should be collected.
Performance in different anatomical sites and patient subgroups
To better understand the benefits of AI contouring compared with manual or atlas-based contouring, generating evidence on the technologies' performance on anatomical sites other than head, neck and prostate is advised.
Also, evidence should be generated about the software's performance when contours may be challenging to obtain because, for example, a person has limited mobility or atypical anatomy. Subgroup analysis for these people can be done by collecting patient-level information on age, sex, ethnicity, relevant comorbidities or disabilities and the anatomical sites targeted by the scan.
This analysis is important if the developer expects their technology to be used for other anatomical areas and patient subgroups.