3 Committee discussion
The diagnostics advisory committee considered evidence on EarlyCDT Lung for lung cancer classification of solid lung nodules from several sources, including a diagnostics assessment report and an overview of that report. Full details of the evidence are in the project documents on the NICE website.
A test that accurately predicts lung nodule malignancy risk could reduce anxiety in people with nodules
3.1 The patient expert emphasised the importance of people understanding their options based on malignancy risk assessments. A test that correctly identifies malignant nodules in people who would otherwise be placed on surveillance, could reduce the number of people on CT surveillance and the anxiety associated with waiting. Malignant nodules could be biopsied and treated earlier, potentially leading to better health outcomes. However, patient experts were aware of the trade-off between the benefits of earlier diagnosis and treatment, and the risks of overdiagnosis leading to harms from unnecessary biopsies and treatment. The committee acknowledged the potential benefit of EarlyCDT Lung in reducing anxiety and the impact it could have in helping people, along with their clinician, to decide which further testing or treatment steps are best.
The evidence on the diagnostic accuracy of EarlyCDT Lung in people with lung nodules is limited and accuracy estimates are very uncertain
3.2 The external assessment group's (EAG) systematic review found only 5 study populations in which EarlyCDT Lung data on people with nodules was reported. Of the 5 studies, 3 were conference abstracts and only 2 were reported in full. The 2 complete publications had a high risk of bias in terms of patient selection, and the German study (Gonzalez et al. 2021) was also at high risk of bias in the flow and timing domain of the quality assessment. There were also serious concerns about the applicability of the results of these studies to NHS practice because of the position in the pathway where the test was used and how the test was used and interpreted. The EAG's meta-analysis estimated that EarlyCDT Lung has a sensitivity of 20.2% (95% confidence interval [CI] 10.5 to 35.5) and specificity of 92.2% (95% CI 86.2 to 95.8). The committee discussed that the EAG's estimate was uncertain with wide confidence intervals for the sensitivity. It noted that these results are different to the company's estimates, which were based on a single study that had 41.3% sensitivity (95% CI 35.0 to 47.6) and 90.6% specificity (95% CI 87.1 to 94.1; Healey et al. 2017). The EAG suggest that the inclusion of case-control data in Healey et al. (2017), where cases were people diagnosed with lung cancer and controls were healthy volunteers, could account for the difference in accuracy estimates. It suggested that poorer diagnostic accuracy among smaller nodules, which are less likely to be present in the confirmed cancer cases, could bias the results. The committee discussed if accuracy data from screening studies was generalisable to this decision question. The EAG commented that the malignancy risk is likely to differ between a screening population – that is people who may or may not have nodules – and a population of people who have nodules. Therefore, the screening data cannot be extrapolated reliably. The committee concluded that given the uncertainties around the accuracy estimates and the populations studied, further evidence of the accuracy of EarlyCDT Lung in people with nodules is needed (see section 4.1).
The risk model used to calculate post-test malignancy risk from the EarlyCDT Lung result and the pre-test risk needs further validation
3.3 The committee understood that positive EarlyCDT Lung results are intended to be combined with a pre-test risk (using existing malignancy risk assessment models) to give a post-test risk. The EAG's report highlighted that the risk model (Healey et al. 2017) provided by the company to recalculate risk scores, may overestimate diagnostic accuracy as it was based on evidence from case-control studies (see section 3.2). The committee agreed that combining the tests was an appropriate way to estimate malignancy risk, but also noted that the accuracy data on these test combinations is very limited. The committee recommended that further validation of the model that combines the EarlyCDT Lung result with existing risk models is needed (see section 4.1).
3.4 The EAG report identified 2 potentially relevant ongoing studies of EarlyCDT Lung: 1 case-control study in China aiming to recruit 1,000 people and the other an observational screening study in the US. The committee heard that the study in China is assessing a different version of EarlyCDT Lung that was specifically developed for the market in China and would therefore not be generalisable to the UK. They also heard that the US study is due to be published in 2022. The committee also discussed the IDx Lung study that started in May 2021. It is unclear from published literature where in the pathway EarlyCDT Lung is positioned in these trials, and whether it will be used to update risk scores as indicated. The committee concluded that further research is needed on the accuracy of EarlyCDT Lung itself (see section 3.2) and on the accuracy of combining the test results with other risk models (see section 3.3). The committee also noted that the gold standard reference test in a diagnostic accuracy study should be histological confirmation of cancer or minimum 2 years follow up with CT surveillance (see section 4.1).
3.5 The EAG's report found no evidence on how EarlyCDT Lung testing affects changes in clinical decision making for people with lung nodules. The EAG's simulation study indicated that EarlyCDT Lung is unlikely to significantly impact clinical management in low-risk nodules (below 10%). EarlyCDT Lung may improve clinical management in intermediate-risk nodules, particularly for those with a higher pre-test risk (above 48%). The data feeding into the simulations however was at high risk of bias, and the simulation had to use assumptions because of weak evidence. The committee discussed that data is needed to demonstrate if a positive EarlyCDT Lung test result changes decision making such as moving from CT surveillance to biopsy or from biopsy to immediate excision without biopsy (see section 4.1).
The likelihood and impact of unnecessary biopsy or resection of indolent and benign nodules is unknown
3.6 The EAG and committee discussed the risk and harms of false-positive test results (benign nodules incorrectly identified as malignant). The committee discussed that the safety of biopsy and surgical excision has improved in recent years, but it noted that there may still be adverse effects and anxiety for patients. It also discussed the harms of diagnoses in people with limited life expectancy or with multiple comorbidities, for whom treatment is not possible but the test could cause unnecessary anxiety. The committee discussed that data on the harms of biopsy and excision should be available through existing databases and recommended that it could be obtained as part of a large audit (see section 4.2).
3.9 The EAG's report highlighted variation in management practices and a lack of evidence around decisions in managing intermediate nodules. Under the British Thoracic Society's (BTS) guidelines for the investigation and management of pulmonary nodules (2015) people with nodules in the intermediate 10% to 70% risk group may be offered CT surveillance, biopsy or surgical excision. A clinical expert explained that in the BTS guidelines, the defined malignancy risk categories for initiating Brock and Herder risk assessment, and for guiding further testing and treatment, were mostly based on evidence graded at a level 3 (non-analytical studies; for example, case reports, case series). The committee recommended that an audit of existing data should be carried out to determine how these nodules are currently managed in NHS practice. It considered that the impact of EarlyCDT Lung and other tests would be difficult to ascertain without first understanding what happens in current clinical practice (see section 4.2).
The clinical consequences of CT surveillance of lung nodules, such as stage progression, are uncertain
3.10 The committee noted that the potential benefit of EarlyCDT Lung is that it could identify malignant lung nodules earlier than standard tests such as CT surveillance. This could lead to earlier treatment and improved patient outcomes. It also noted that there was no evidence that stage progression of malignant nodules happens in the timeframe of CT surveillance. The EAG commented that modelling studies of lung cancer screening strategies often evaluate the likelihood of pre-clinical stage progression over time, but cautioned that the generalisability of this evidence is unclear. The committee noted that it is important to understand if and how lung nodules progress during CT surveillance. This information could then be used in a linked-evidence modelling approach to understand if EarlyCDT Lung could result in meaningful earlier diagnosis of malignant lung nodules. The committee discussed that this data should be available through existing databases and recommended that it could be obtained as part of a large audit (see section 4.2).
Data on patient and nodule characteristics and how different factors impact disease progression would be helpful for future modelling
3.11 The EAG report highlighted that better characterisation of the target population and nodule characteristics is needed for future assessments. It noted that the group of people with lung nodules is heterogeneous and different factors may impact on the speed of progression of malignant lung nodules. The committee agreed that this is important data to collect because it could be used in future modelling of the clinical and cost effectiveness of EarlyCDT Lung or other new tests in this area (see section 4.2).
3.12 The committee acknowledged that the evidence to support the use of the EarlyCDT Lung to assess the malignancy risk of solid lung nodules is weak. So, the full benefits and potential harms of widespread use of EarlyCDT Lung testing cannot be reliably quantified. There is no robust data to show the clinical utility of EarlyCDT Lung testing, specifically:
how well it distinguishes between benign and malignant nodules
how it influences clinical decision making
the impact it has on longer-term clinical outcomes.
The committee concluded that it was unable to recommend the widespread use of EarlyCDT Lung testing. It recommended that further research is needed to address the limitations in the evidence (see section 4.2).
3.13 The committee noted that a better understanding of the population with lung nodules and of the current diagnostic pathway is critical for supporting a linked-evidence model for EarlyCDT Lung and for other new technologies that would be used in the same pathway (see section 2.3). These issues include:
How patient and nodule characteristics impact disease progression (see section 3.11).
How intermediate-risk nodules are currently managed (see section 3.9).
The clinical consequences of CT surveillance of lung nodules (see section 3.10).
The prevalence, likelihood and impact of false-positive test results and overdiagnosis (see section 3.6).