NICE process and methods

6 Evidence and expert advice

6.1 Types of evidence and advice presented to the committee

In developing its draft recommendations, the committee considers:

  • the sponsor's submission, comprising clinical and cost evidence (based on the scope) and relevant cost modelling; the sponsor is responsible for ensuring that the submission contains all necessary data to properly evaluate the case for adoption

  • evidence presented by the external assessment centre (which is independent of NICE), comprising a detailed analysis and critical appraisal of the submission in the form of an assessment report

  • evidence from the programme team or other relevant organisations or working groups

  • contributions from expert advisers

  • contributions from patient and carer organisations

  • information about ongoing or future research.

6.2 Published evidence

Valid publicly available evidence that is relevant to the scope is identified with 2 aims:

  • to ensure that a comprehensive evidence base is available to the committee

  • to inform evidence synthesis (meta-analysis) and modelling studies (section 7) when these are needed.

Evidence may relate to primary clinical research or secondary research (such as evidence synthesis or modelling studies).

6.2.1 Search for published evidence

The scope informs the literature search for evidence. The sponsor carries out a literature search as part of its submission, and the external assessment centre validates this search to support its critical appraisal of the evidence in the assessment report.

The search typically covers relevant efficacy, effectiveness, usability and safety outcomes (including intermediate clinical outcomes) and available clinical and health economics studies of any type, including non-UK studies. A range of medical literature databases is systematically searched, including: primary research databases; registers or databases of systematic reviews; meta-analyses and technology assessment evaluations; registers or databases of ongoing clinical trials (including experimental or observational studies); and conference proceedings. The external assessment centre reproduces the sponsor's search to validate that all relevant evidence has been identified.

6.3 Unpublished evidence

6.3.1 Purpose and rationale

To ensure that all available relevant evidence is taken into account, the committee considers unpublished research if it is within the scope of the evaluation. As with publicly available evidence, such as that in peer-reviewed journals, unpublished evidence may relate to primary clinical or secondary research. Unpublished evidence may be included in the sponsor's submission or identified by the external assessment centre. Unpublished data may be used to support a narrative review of the evidence, as well as to inform the design and conduct of new secondary research studies (section 7).

6.3.2 Unpublished evidence sources

There are 2 main sources of unpublished evidence:

  • As part of their submission, sponsors are invited to provide unpublished evidence within the scope of the evaluation, including directly observed clinical outcomes, non-clinical studies such as in vitro research, evidence synthesis, outcomes modelling and health economics studies relating to the technology. It is the sponsor's responsibility to identify all relevant unpublished evidence as part of its submission, including studies not submitted for publication or rejected after submission.

  • In its critical appraisal of the sponsor's submission, the external assessment centre may identify other unpublished evidence, such as analysis of data from observational research sources, including professional or company-sponsored registers.

6.3.3 Unpublished evidence submitted in confidence

Unpublished evidence is not normally considered confidential and may therefore be disclosed in publicly available guidance documents. However, it may occasionally be necessary for the committee to review data provided to the programme in confidence. The committee considers such evidence in a private part of the meeting.

Unpublished evidence is accepted under agreement of confidentiality and is not made available to the public. Such evidence includes commercial-in-confidence information (confidential because its public disclosure may affect the commercial interests of a particular company) and academic-in-confidence data (confidential because the full data are yet to be published).

If the owner of any unpublished data included in the submission believes that the data should be treated as commercial- or academic-in-confidence, they should clearly state the rationale, taking into account the following principles:

  • Information and data that have been made publicly available anywhere are not considered confidential.

  • When trial results are to be published in a journal at a date later than the first public release by NICE of documentation quoting data from these trials, a structured abstract relating to the future journal publication should, as a minimum, be made available for disclosure.

NICE asks data owners to reconsider restrictions on release of data either when the reason for the restrictions is not clearly explained, or when such restrictions would make it difficult or impossible for NICE to show the evidential basis for its guidance.

6.4 Contributions from expert advisers

Expert advisers contribute to the evaluation of technologies by providing additional knowledge, opinion and experience to the committee. They provide opinions on the published evidence and supplement it with information on anecdotal or theoretical outcomes, and other information relevant to the evaluation of the technology, its comparators and the conditions for which it is used. Such information can relate to: the technical specification of the technology if this might affect its capability in delivering the claimed benefits; to the training and experience needed to use the technology; and to organisational factors that might influence the technology's technical performance or use in clinical practice.

Expert advice can also be used as part of evidence synthesis or modelling studies.

Experts advisers also contribute to the scope, give clinical advice when needed to the external assessment centre, and are involved in presenting the evidence to the committee. See the MTEP process guide for more information about how expert advisers are chosen.

6.5 Contributions from patient and carer organisations

NICE recognises that the experience of patients and carers can provide unique insights that may be of value to the committee when developing its recommendations. The public involvement programme always approaches patient and carer organisations to obtain their views on the technology (see the MTEP process guide). Patients and carers can provide information about living with the condition to which the technology relates, about any subgroups of patients who may need special consideration in relation to the technology, and about using the technology and/or comparator technologies. Patient and carer organisations can provide insight into outcomes and describe ease of use, discomfort, how the technology affects daily activities, and other aspects of quality of life.

NICE periodically reviews its experience of obtaining information on medical technologies from patient and carer organisations with the aim of refining its approach.