Guide for COVID-19 evidence collection
This document provides guidance on clinical evidence generation to developers of medicinal products for COVID-19 and should be read in conjunction with the NICE guide to the methods of technology appraisals 2013.
- Multicentre, randomised, pragmatic clinical trials are preferred. Pragmatic trials evaluate the effectiveness of interventions in routine clinical practice conditions.
- The design of the trial should be based on internationally agreed recommendations and guidelines (for example, on the WHO R&D Blueprint: COVID-19 therapeutic trial synopsis). The efficiency of the trials could be improved by using adaptative designs (for example, adaptive randomisation, play the winner/drop the loser designs, interim analyses).
- The sample size of the trial needs to be justified and can be optimised by using an adaptive approach.
- Interim analyses should be planned, fully described and include an appropriate control of the type I error. Interim analyses should be based on clinically relevant outcomes. The decision to stop the clinical trial early because of futility or worsening of COVID-19 should be made by an independent monitoring committee.
- Ideally the trial should be blinded for patients and investigators, although it is recognised that such blinding may be difficult to implement for institutional trials or investigator sponsored studies. If a blinding procedure cannot be implemented, preference should be given to clinical outcomes that can be assessed with as little subjectivity as possible and the adjudication of the endpoints should ideally be performed independently. Outcomes assessors should be blinded in all studies. The implementation of a blinding procedure should not delay the start of the trial.
- The primary hypothesis should be a superiority hypothesis with a clinically meaningful difference. The statistical analysis should be able to exclude a possible scenario in which the product may worsen the outcome of COVID-19. The number of hypotheses tested in the study (primary and secondary) should be minimised. The trial should focus on clinically relevant hypotheses, endpoints and effect sizes.
- The indication (prevention or curative intent) and the population (severity of disease, primary or secondary care) should be clearly defined. The COVID-19 case definitions and diagnostic criteria should be provided, with preference given to the current WHO case definitions in its interim guidance on global surveillance for COVID-19. Whenever possible, the UK case definitions for possible COVID-19 should be documented. Similarly, Public Health England’s criteria for inpatient definition should be used or documented in the trial.
- The sampling method and type of test used to confirm COVID-19 (for example, RT-PCR or serological test type) should be described. Only validated testing methods should be used and the diagnostic accuracy of the test(s) used should be recorded where possible.
- Pragmatic studies with broad eligibility criteria, which enrol patients whose results will be generalisable to a routine practice population, are preferred. In particular, the inclusion and exclusion of populations at risk should be described and justified (for example, children or older people with age range specified, older people in care homes, healthcare workers, people with poor functional status, pregnant women, people with other significant comorbidities).
- A full range of baseline patient characteristics should be collected so that the relevance of the patient population to the NHS patient population can be evaluated. This includes sex, age, ethnicity, comorbidities, smoking status, residence in care home, functional status, and confinement status.
- The analysis of important subgroups, such as those listed by the NHS as high-risk patients, should be pre-specified, as should potential coefficients in any adjusted analysis of treatment effect. This could include those with comorbidities such as lung conditions, diabetes, cardiac and renal insufficiency, underlying immune suppression or obesity.
- The nature of the intervention (for example, antiviral or product used to treat the COVID-19 cytokine release syndrome), the rationale for use and the investigated dose should be described and justified.
- The contra-indications, special precautions for use, special monitoring requirements and drug-drug interactions should be described in the protocol.
- Clinical studies should include a comparator arm reflecting the currently agreed standard of care, reflecting the WHO coronavirus disease (COVID-19) technical guidance: patient management, the NICE COVID-19 rapid guideline: critical care in adults (NG159; NICE 2020) and Public Health England and Department of Health and Social Care COVID-19: guidance for health professionals whenever possible and applicable.
- The comparator should be clearly defined and described. If a clinician’s choice comparator arm is used, the treatments permitted should be outlined in the study protocol (if possible) and all treatments that patients have had during the trial must be recorded.
- Any rescue medications allowed in the study, including investigational products or products authorised in other indications and used off-label, should be described in the protocol and their use during the trial recorded.
- For the primary endpoint of the study, preference should be given to clinical outcomes focusing on the recovery and survival of the patients, using, for example, the WHO ordinal scale of clinical improvement in its COVID-19 therapeutic trial synopsis. If a composite primary endpoint including both clinical improvement and all-cause mortality is used, the follow-up of the patients should allow recording of the full clinical recovery or death of each patient. The individual outcomes within a composite endpoint should also be disaggregated and presented separately when the trial is reported.
- Secondary endpoints should try to capture the progression of the disease (for example, COVID-19 clinical symptoms and severity, hospitalisation, need for intensive care, ventilatory support and intubation, duration of hospitalisation, adverse effects).
- If not used as the primary endpoint, the trial must capture all-cause mortality. Mortality data could be collected from the Office of National Statistics (ONS) if NHS number, date of birth and postcode are also collected.
- Health-related quality of life (HRQL) should ideally be captured whenever clinically appropriate. The collection of HRQL may not be appropriate during hospitalisation in an intensive care unit and should ideally be collected within the study immediately before and after the episode. NICE prefers the EQ-5D tool to measure HRQL. NICE uses the EQ-5D-3L UK value set within its decision making. The EQ-5D-5L version can be included within the trial and the results mapped to the EQ-5D-3L, as recommended in the NICE position statement on the use of the EQ-5D-5L value set.
- Surrogate outcomes, such as change in viral load from baseline, can be used in the trial but the primary hypothesis of the study and any interim analysis should be based on the final clinical outcomes of the patients.
- Any resistance to the intervention (primary or secondary) or SARS-CoV-2 mutation should be monitored.
- The duration of the clinical trial will depend on the nature of the intervention (for example, preventative or curative intent) and the primary hypothesis. The primary endpoint may be measured within a few weeks (for example, 28 days). Considering the usual short duration of the acute phase of COVID-19, the duration of the trial and the follow-up of the patients should allow the observation of the final clinical outcome (recovery or death) of all the patients enrolled in the study. The number lost to follow-up would be expected to be low. The investigators should endeavour to follow all the patients enrolled in the trial, including patients who withdrew from it.
- The clinical trial should only be stopped prematurely, based on planned interim analyses, for futility or safety reasons. Even if an interim analysis allows the rejection of the primary null hypothesis, the trial should be completed to minimise the uncertainty around the estimation of the treatment effect.
- To minimise the burden on the healthcare system, consideration should be given to using existing databases. In the UK, these could include the CPRD and Hospital Episode Statistics, as well as the ONS.
- If existing databases are not used, healthcare resource use data should be collected on primary care visits, hospital admissions, intensive care admissions, use of ventilation and length of stay.
Real-world evidence (RWE) collection
RWE collection should be considered before a NICE evaluation. RWE should not replace the conduct of well-designed clinical studies. RWE collection is complementary to evidence collected through comparative clinical trials to address uncertainty or evidence gaps identified during the clinical development. RWE collection should be considered when a medicine becomes available in the NHS outside of clinical trials (for example, through an early access medicines scheme).
A proposal for prospective post-appraisal RWE data collection could also be part of the submission to be assessed by NICE’s technology appraisal committees. Postappraisal RWE evidence collection and the possibility of managed access agreements should be discussed with NICE as early as possible pre-appraisal.
The key considerations for RWE collection in COVID-19 are:
- The purpose of data collection should be to address evidence gaps and areas of uncertainty (for example, to better characterise the clinical effectiveness of the technology when used in the NHS, to obtain evidence on NHS clinical practice, to collect clinical-effectiveness data on populations of patients not enrolled in the clinical studies or to obtain long-term follow-up of the patients).
- The research question should be pre-specified and a protocol for the conduct of the RWE study, including a statistical analysis plan, should be developed. The data collected in the study should be quality assured when possible and any deviations from the protocol should be reported and justified.
- As for the collection of trial data, the burden on the healthcare system should be minimised and therefore consideration should be given to using existing NHS databases.
- If new registries are being considered, their quality should be assessed by using the REQueST tool.
- Patient characteristics should be collected so that the generalisability of the findings from the clinical trial to routine clinical practice can be evaluated.
- It is important to record in real-world practice the mortality that can reasonably be attributed to COVID-19. The follow-up of patients should try to capture the time to full recovery, all-cause mortality, and deaths directly or indirectly related to COVID-19 even in the absence of formal confirmation of the diagnosis. Different scenario analyses can be presented (for example, mortality in confirmed COVID-19 cases, mortality in probable or possible COVID-19 cases). Mortality data collection from the ONS may be an option, if NHS number, date of birth and postcode are also collected.
- Self-reporting of symptoms by the patients could be considered.
- For therapeutic interventions, the SARS-CoV-2 status of all patients having the medicine in clinical practice should be established when possible (possible cases, probable cases, confirmed cases) and the testing method recorded. For preventative interventions, all patients showing symptoms of COVID-19 should be tested for SARS-CoV-2 and the testing method recorded.
The advice contained within this guide is based on the scientific and methodological knowledge publicly available at the time of writing and cannot account for future changes and developments in scientific knowledge, regulatory requirements or any referenced material from external sources.
The advice contained within this guide is provided without any warranty, express or implied of fitness for any purpose including, but not limited to, compliance with applicable legislation relating to the particulars and documents that should be submitted in support of any marketing authorisation (or other) application. It is supplied as is and the user assumes all risk and responsibility for its use.
The content of this guide may be subject to further revisions as more scientific and methodological knowledge becomes available.