Tests in secondary care to identify people at high risk of ovarian cancer

The committee discussed the diagnostic accuracy data available for the included tests. It noted that the studies in the clinical-effectiveness review varied in which target condition they used (that is, what was considered a positive reference standard test result). The committee heard from clinical experts that although epithelial cancers are the most common form of ovarian malignancy, people referred to secondary care with suspected ovarian cancer will include those with non-ovarian tumours and non-epithelial ovarian tumours. The committee noted that studies which used a target condition of ovarian cancer or epithelial ovarian cancer retrospectively excluded patients from analysis based on their reference standard diagnosis, and that these studies had been assessed as having a high risk of bias by the external assessment group (EAG). The committee concluded that studies which used a target condition of all malignant (including borderline) tumours were most representative of clinical practice.

The committee considered the generalisability of the evidence to clinical practice in the NHS. It heard from clinical experts that the prevalence of malignancy in study populations was considerably higher than would be expected for people referred to secondary care with suspected ovarian cancer in the NHS. Clinical experts commented that a prevalence of less than 10% would be expected, and suggested 5% as a realistic prevalence of malignancy in this population. The committee heard from the EAG that, in addition, most studies did not report the distribution of disease stages among patients with ovarian cancer. Therefore, it noted that the spectrum of disease in the studies may not reflect that seen in secondary care in the NHS. The committee concluded that the study populations, for all tests, may not be representative of the clinical population for this assessment. In particular, the differing levels of disease severity in the study populations and in secondary care in the NHS could mean that the sensitivity and specificity estimates obtained from these studies are not accurate estimates of how the tests would perform in secondary care in the NHS.

The committee considered the test accuracy of the Simple Rules and ADNEX. It noted that studies showed that they were statistically significantly more sensitive than RMI I. The committee then considered the expertise of practitioners doing and interpreting ultrasound scans in studies assessing the Simple Rules system and the ADNEX model. The committee heard from clinical experts that practitioners in these studies had a higher level of skill and experience than is generally available in secondary care in the NHS, and noted that there were limited data available on the accuracy of tests done by less experienced operators. The committee heard that although image acquisition would generally be straightforward for NHS practitioners, additional training would be needed in interpreting the images to use the Simple Rules or ADNEX model. The committee heard from clinical experts that the local organisation of care would be likely to affect test performance in practice; and that there is uncertainty about the effect of NHS service models on how well the Simple Rules and ADNEX model would perform in secondary care. For example, if a model of delivery in which patients are seen, scanned and their condition managed in 1 setting was used, this may lead to improved performance of the tests through a concentration of services and skills. It also heard from clinical experts that no data have been published on inter-observer variation using the IOTA tests in the NHS. The committee therefore concluded that there is considerable uncertainty about the likely performance of the Simple Rules and ADNEX tests in secondary care in the NHS, and that the accuracy reported in the studies may not be achieved in clinical practice in the NHS.

The committee considered the test accuracy data for the ROMA and Overa (MIA2G). It noted that relatively few studies were identified for these tests, particularly studies with a direct comparison with RMI I. The committee concluded that there is considerable uncertainty about the diagnostic accuracy of these tests; however it is possible that the tests may offer improved accuracy relative to RMI I.

The committee considered data on the accuracy of RMI I at thresholds other than 250. It noted that most studies with a direct comparison of RMI I at 250 and another threshold used a threshold of 200, and that there were relatively few studies with other alternative thresholds. It also noted that there was very little difference in the summary estimates of sensitivity and specificity for RMI I at thresholds of 200 and 250 obtained from studies with a direct comparison. The committee therefore concluded that RMI I used with a threshold of 200 was unlikely to offer accuracy benefits over using this test with a threshold of 250, and noted that the use of RMI I at a threshold of 250 is recommended in the NICE guideline on ovarian cancer.

The committee discussed how the stage of ovarian cancer (early or advanced) could affect test accuracy. It heard from clinical experts that about 70% of people identified with ovarian cancer have advanced stage cancer. However, the main benefit of tests such as RMI I in secondary care is in identifying early stage ovarian cancer, with advanced stage ovarian cancer being more apparent from imaging. The committee heard from the EAG that very few data were available to inform estimates of test accuracy by stage of ovarian cancer. Two studies assessing the ROMA reported that sensitivity was lower for detecting early stage ovarian cancers. However most studies did not provide details on the stage of cancer of study participants included in analysis. The committee heard from clinical experts that populations in studies were likely to include more cases of advanced than early stage ovarian cancers, and that the performance of tests to detect early and advanced stage ovarian cancer could differ substantially. The committee concluded that there is uncertainty about how accurate the tests are at correctly detecting early stage ovarian cancer, potentially the most relevant group for this assessment. The committee also concluded that data on the accuracy of tests to detect early stage ovarian cancer would be important for any future assessment (see section 6.1).

The committee discussed the sensitivity and specificity estimates used in the cost-effectiveness modelling. It noted that these estimates were taken from the clinical-effectiveness review and that the concerns about the applicability of data from these studies to NHS secondary care (see sections 5.4 and 5.5) also apply to the model. The committee also noted that relatively few studies were available to inform test accuracy estimates for the ROMA and Overa (MIA2G) tests (see section 5.6).

The committee noted that tests with the highest sensitivity (ADNEX and Simple Rules) that resulted in more people with ovarian cancer being referred to a specialist MDT tended to be cost effective. It considered the parameter used in the model for the beneficial effect of a referral to a specialist MDT for people with ovarian cancer; a hazard ratio for overall and progression-free survival obtained from a Cochrane review (see section 4.31). The committee heard that estimates from this review were based on studies with a mixed cohort of early and advanced stage ovarian cancer, and that because advanced stage was likely to be predominant in this cohort, the summary estimate may not be an accurate reflection of the beneficial effect of specialist MDT treatment on people with early stage ovarian cancer. It also heard that the benefits for people with early stage ovarian cancer of having their surgery done by a gynaecological oncology specialist are potentially more difficult to assess than those for people with advanced stage cancer. An improved quality of surgery is likely to lead to more accurate staging of the cancer, which will help with subsequent treatment decisions. For example, accurate staging may show that chemotherapy is not needed (for low-risk stage I disease); but inadequate staging in a non-specialist centre could lead to inappropriate use of chemotherapy or the need for further surgery to accurately stage the cancer. The committee also heard from clinical experts that people with a false negative test result will have their condition managed in secondary care, and if their ovarian malignancy is recognised at a later date they will then be referred to a specialist MDT. The effect of this delayed referral, rather than lack of referral, to a specialist MDT on patient outcomes is unclear. The committee concluded that, because of a lack of data, there is considerable uncertainty about the effect of false negative test results on people with ovarian cancer, particularly if they have early stage ovarian cancer.

The committee discussed the costs included in the model for people referred to a specialist MDT. It noted that the cost of an MDT meeting had been included, but heard from clinical experts that additional costs may be incurred when a patient is referred to a specialist MDT for discussion; such as costs for the time taken by radiologists to review images in advance of the meeting. The committee heard from the EAG that these additional costs were not captured in the model. The committee also noted that in the base case, the model used NHS reference costs for surgery, and that this may not adequately capture the cost of extensive surgery potentially needed for people with advanced stage cancer, who make up 75% of the population with an ovarian malignancy in the model. It heard from clinical experts who suggested that the costs associated with more extensive surgery should be included in the model. The committee noted that in a scenario analysis in which additional surgery costs were assumed, RMI I (threshold 250) was cost effective at a maximum acceptable incremental cost-effectiveness ratio of £20,000 per quality-adjusted life year gained. The committee concluded that the costs of a referral to, and treatment by, a specialist MDT may have been underestimated in the model, and that this could affect the model results, such as which tests seemed to be cost effective.

The committee discussed the costs of CA125 testing included in the model. It heard from the EAG that the economic model assumed that all patients have a CA125 test in secondary care, even if they previously had one in primary care. The committee heard from clinical experts that the reasons for patients having another CA125 test in secondary care are: CA125 levels may have changed since the first test was carried out; some risk scores are only compatible with a specific brand of CA125 test; and some tests include CA125 as part of an array. The committee concluded that the assumption in the economic model in relation to CA125 testing costs was valid.

The committee considered its discussions on the clinical- and cost-effectiveness evidence. It concluded that:

• There is considerable uncertainty about the estimates of test accuracy used in modelling because: relatively few studies were found to inform estimates (for the ROMA and Overa [MIA2G]; see section 5.6); the high prevalence of malignancy in studies suggested that they were not representative of clinical populations in secondary care in the NHS (see section 5.4); and that the level of expertise of people interpreting scans for the Simple Rules and the ADNEX model in studies was higher than would be routinely available in the NHS (see section 5.5).

• There is uncertainty about the accuracy of tests to detect early stage ovarian cancer (see section 5.8), and about the likely effect on outcomes for people with early stage ovarian cancer who have a delayed referral to a specialist MDT, as a result of an initial false negative test result (see section 5.10).

• There is uncertainty about the costs of assessment and treatment at a specialist MDT, and that higher costs would impact model results (see section 5.11).

The committee discussed the accuracy of tests, noting that tests with higher sensitivity had lower specificity. The committee heard from clinical experts that tests with high sensitivity reduce the number of missed cases of ovarian cancer, but that lower test specificity will result in more false positive referrals to specialist MDTs. The committee heard from clinical experts that there is very limited specialist MDT capacity for personnel in relation to the current case demand, and that increasing the number of false positive referrals to specialist MDTs would reduce the quality of assessment by limiting time available for discussion for each patient. Clinical experts commented that this could adversely affect, or delay, decisions made in specialist MDT meetings about patient treatment. The committee concluded that using tests with lower specificity would have a large impact on specialist MDT services, but the model has not captured the impact because of a lack of data on the effect this would have on patient care and clinical outcomes and because of its structure.

The committee considered the accuracy of the tests for people who are pre- and postmenopausal. It noted that relatively few studies provided accuracy estimates stratified by menopausal status. The committee further noted that the EAG did cost-effectiveness analyses for pre- and postmenopausal subgroups; however there were relatively few data to inform the tests' performance in pre- and postmenopausal populations in this analysis. The committee heard from clinical experts that menopausal status could considerably affect the performance of the tests, and that further data are needed to assess the performance of the tests in these subgroups (see section 6.1).

The committee discussed the likely effect of test results on decisions made about patient care. It noted that no data were available on the effect of test results on decisions about patient care or referral. The committee heard from clinical experts that in practice results from tests such as the RMI I are used alongside further information, such as imaging, when making decisions about patient care and referral. It noted therefore that increased accuracy of testing may not correspond to changes in decision-making. The committee concluded that there is uncertainty about how the results of the tests included in the assessment would be used in clinical practice in the NHS, and that further research on this would be useful (see section 6.2).

The committee heard from patient experts that access to tests in primary care for people with ovarian cancer symptoms varied, and that getting a referral to secondary care could take a long time. It heard further from clinical experts that about 60% of people referred to secondary care have a late stage of cancer, and they are often referred from colorectal and urological cancer services and emergency care. The committee agreed that differences in initial assessment may lead to variation in patient outcomes. The committee noted that tests for suspected ovarian cancer in primary care were outside of the scope of this assessment and that evidence from this setting had not been reviewed.

The committee discussed the use of tests in sequence. It heard from clinical experts that the use of a highly sensitive test followed by a highly specific test may improve accuracy of referral. The committee noted that the EAG had looked for studies that assessed the use of included tests in combination or sequence in the clinical-effectiveness review; however very few data were found (1 study with a lack of detail about how test results were combined to produce a positive result). The committee considered that the use of tests in sequence could be cost effective and further research is needed to inform accuracy estimates (see section 6.3).

The committee noted that there is an ongoing National Institute for Health Research funded diagnostic test accuracy study; the ROCkeTS (Refining Ovarian Cancer Test Accuracy Scores) study, which will report in 2019 or 2020. This study will evaluate existing and new risk prediction models for people with symptoms of suspected ovarian cancer against a comparator of RMI I (threshold of 250). It will assess patients using the IOTA Simple Rules and ADNEX tests, as well as biomarker assays. Practitioners doing ultrasound scans as part of the study will have an IOTA training course; therefore the study will provide test accuracy data for NHS practitioners with a defined amount of training. The study will also report costs and resource use associated with the diagnostic tests. The recruited study population will be people referred to secondary care in the NHS; the committee noted that the results are likely to be very relevant to future updates of this guidance.