ProKnow cloud-based system for radiotherapy data storage, communication and management: early value assessment

There is a need to increase standardisation of radiotherapy treatment planning protocols across radiotherapy centres in the NHS. The clinical experts noted that peer review is done by clinical oncologists and physics and dosimetry leads using different methods, such as in person or using Microsoft Teams. But, when there is a shortage of staff to do peer reviews or a backlog in radiotherapy treatment planning, peer review may not take place. ProKnow could help address the large variation between centres through training, data collection, scorecards and linkage to national registries.

The clinical experts agreed that smaller centres with fewer radiotherapy staff have more difficulty allocating time for peer review, so they may benefit more from using a platform that allows upload of radiotherapy treatment plans for external review. For rare and complex cancers, when there may only be one specialist in a centre, ProKnow could allow communication with clinicians in other locations using a standardised format. The peer review function within ProKnow also does not need users to be online at the same time, so both clinicians do not need to be present during the peer review process, which would be the case for in-person or virtual meetings. The committee concluded that ProKnow may enable an increase in the number of radiotherapy treatment plans undergoing peer review, which would bring clinical practice more in line with guidelines from the Royal College of Radiologists.

Further evidence will be generated while ProKnow is in use to explore the impact of the technology on patient outcomes and radiotherapy treatment planning resources. The clinical experts noted that ProKnow is unlikely to have any direct effect on patient outcomes, but the technology may increase adherence to national guidelines and have a positive impact on clinical oncology training. Additional efficiency-related metrics may also be collected to estimate the impact on staffing and resources. The committee concluded that ProKnow can be used for radiotherapy data storage, communication and management while further evidence is generated.

The evidence includes 8 service evaluations or audits of treatment plan quality and 4 service evaluations or audits assessing the use of ProKnow during training. Two were done exclusively in a UK or NHS population. The evidence highlights the versatility of ProKnow to enable data sharing, including radiotherapy treatment plans, and the tools it has to enable quality assurance and review. The evidence also shows the functionality of ProKnow as a training tool for treatment planning and contouring. The clinical experts said that this could improve clinicians' skills in radiotherapy treatment planning and reduce variability between clinicians. Future uses of ProKnow may include evaluation of artificial intelligence technologies for treatment planning and contouring. It could also be used to link data to national routine datasets to determine the impact of ProKnow on patient outcomes. But the committee noted that there is a lack of prospective and comparative evidence for ProKnow. This means that it is not possible to show the impact of ProKnow on treatment planning outcomes, quality assurance, or hospital resource use. The committee concluded that the evidence base is very limited for ProKnow. See the assessment report on the NICE website for full details of the clinical evidence.

The External Assessment Group (EAG) considered the costs and resources associated with ProKnow's use in the peer review process within the care pathway. The EAG identified the key parameters affecting the cost of ProKnow. They are the length of peer review activity and the cost of purchasing the technology, which is based on the number of linear accelerators and amount of data storage needed per year. The EAG also did a scenario analysis, in which they changed the length of peer review and the number of people ProKnow was used for. A conceptual 1% increase in the proportion of plans being peer reviewed for 15 minutes, 30 minutes, and 45 minutes by a consultant oncologist would cost an additional £0.62, £1.23, and £1.85 per person, respectively.

The EAG was able to outline key cost parameters and do a scenario analysis to estimate the cost of ProKnow per year, but they noted that the broader staff and resource impact of introducing ProKnow into the process of radiotherapy treatment planning hasn't been quantified. ProKnow could increase the proportion of treatment plans undergoing peer review. This would incur an additional cost because of the increase in staff time needed to do peer review that was previously not done, but this activity could lead to improved patient outcomes. Any potential cost savings because of improved patient outcomes or more efficient resource use would also have to be included in an economic evaluation. The committee concluded that more data on these cost parameters is needed.

The key evidence gaps relate to the care pathway and key outcomes. The committee concluded that the evidence is very limited, so evidence generation is needed to address the following key evidence gaps:

• There is large variation in peer review practice across the NHS, including the proportion of treatment contours and plans undergoing peer review, who does peer review, and the tools to support how peer review is done in the NHS. Additional work is needed to quantify this variation and the uncertainty associated with the costs of delivering standard care in radiotherapy treatment planning.

• There is a lack of evidence to quantify the impact of ProKnow on patient outcomes. It is possible that increased peer review will lead to an improvement in the quality of radiotherapy treatment plans resulting in better patient outcomes, but this is difficult to quantify.

• There is limited (qualitative or non-qualitative) evidence reporting on the usability or clinician user experience of ProKnow and the ease of retrieving or storing patient data.

• There is no comparative evidence or validated competency assessment exploring the impact of ProKnow on clinical oncology training.

• There is an evidence gap relating to adverse events in large case datasets, which could be addressed as part of the NHS England commissioned pilot of ProKnow. No publications reported adverse events specifically relating to the use of ProKnow. The EAG noted that adverse events relating to patient care would not be expected for the evidence included. But adverse events may relate to confidentiality breaches or issues accessing or retrieving data, which were not reported in the included publications.

• There is limited evidence available for the economic modelling. The EAG was able to outline key cost parameters and conduct a scenario analysis to estimate the cost of ProKnow per year. But it noted that further research is needed to quantify the broader staff and resource impact of introducing ProKnow into radiotherapy treatment planning. This should include any impact on overall radiotherapy treatment planning time.