Process and methods

6 Developing the evidence summary

6.1 Equality and diversity

Evidence summaries are developed in line with NICE's equality scheme and declaration of interests policy.

6.2 Process overview and timelines

The key steps in developing evidence summaries are summarised below. The process reflects their status as advice products and when the advice is needed.

Timelines are agreed for each evidence summary during scoping. It usually takes 12 weeks to produce a standard evidence summary. A longer timeline may be agreed if a large volume of evidence is expected from the literature search.

NICE will consider developing a rapid evidence summary within a shorter time, if needed. Some steps in the process may be omitted or shortened and this will be explained in the published evidence review (see section 6.6). The rapid evidence summary will be concise; as a minimum, resource impact will not be assessed.

The key steps in developing an evidence summary are:

  • develop and agree final scope and population, intervention, comparator and outcomes (PICO) with clinical advisers and external topic experts as needed and/or the organisation commissioning the topic

  • send search request to information services

  • contact the company, when applicable, with data request

  • search for evidence

  • sift and select the evidence

  • appraise and summarise the evidence

  • produce initial draft documents

  • do internal technical check of revised draft documents

  • send revised draft documents to company (if applicable), external topic experts, targeted stakeholders, other relevant NICE teams (including resource impact assessment if applicable) and Medicines and Healthcare products Regulatory Agency (MHRA; if applicable) for review

  • review comments received and produce revised draft documents for sign off (including advisory statement, if applicable)

  • content sign off by associate director, clinical adviser or programme director

  • review comments received and produce revised draft documents for NICE's guidance executive or publications executive

  • if applicable, company invited to check for any factual errors and informed of publication date

  • submit final draft documents for guidance executive or publications executive sign off

  • sign off by guidance executive or publications executive

  • review comments received and produce final documents

  • submit to the organisation commissioning the topic, if applicable

  • publication on the NICE website.

6.3 Scoping

A scope is developed by either NICE or the organisation commissioning the topic, depending on how the topic has been referred to NICE. If it is developed by NICE, clinical advisers and external topic experts advise on the content as needed. The scope outlines the key review questions, PICO, and inclusion and exclusion criteria for the review. The final scope is signed off by NICE or the organisation commissioning the topic.

During scoping the following are confirmed:

  • key contacts at the company (if applicable; see section 6.4)

  • key contacts at the MHRA (if applicable)

  • external topic experts

  • stakeholder organisations

  • literature search terms (see section 6.5).

Options within the process, such as whether an advisory statement and resource impact tools will be produced, are also agreed during scoping.

6.4 Information from companies

When appropriate NICE asks companies to provide information, which may include:

  • key published clinical trials for the indication being reviewed

  • ongoing or recently completed studies that have not yet published in full

  • regulatory status, including whether or not the company (or another company) expects to hold a UK marketing authorisation for the medicine for this indication within the next 2 years

  • likely licensing and marketing timeline

  • the usual dose, or best estimate from the available data

  • the presentation of the medicine, including form, strength and pack size

  • incidence and prevalence of the indication, alternative treatments and estimated usage and cost.

6.6 Writing the evidence review

The development team drafts the evidence review document using a standard NICE template. A modified template may be used depending on the organisation commissioning the topic, for example to support development of a commissioning policy.

The evidence review will usually include the following sections (some may be omitted or shortened if a rapid evidence summary is developed):

Background

A short summary of the clinical problem, current practice and any existing related NICE guidance.

Product overview

A brief discussion of the medicine(s), including:

Effectiveness

A short narrative summary of the clinical effectiveness evidence, usually presented by outcome or by clinical question. Meta-analyses will not be carried out.

Safety

A short narrative summary of the evidence on safety outcomes. Background safety information from other sources may also be included, such as:

  • the SPC or EPAR for precautions, warnings and undesirable effects

  • published advice from medicines regulators.

Person-centred factors

A summary of person-centred factors that may be important for decision making, such as medicines adherence issues, based on a review of the evidence.

Limitations of the evidence

A critical review of the strengths and weaknesses of the relevant evidence.

Resource implications

Cost-effectiveness reviews are not carried out. Basic cost information is obtained from:

If it is agreed during scoping that resource impact tools will be developed, NICE produces these following the processes to estimate the resource impact of NICE guidance.

Development of the evidence review

A link to this process guide including any deviations from the usual process, for example if some sections have been omitted or shortened because of a rapid development timeline. Details of the external topic experts and their declarations of interest are also included.

Appendices

Appendix A: PICO table

Appendix B: summary of included studies

Appendix C: quality assessment of included studies

Appendix D: results tables

Appendix E: literature search strategy

Appendix F: excluded studies

6.7 Writing the evidence summary

The development team drafts the shorter evidence summary document, based on the content of the more detailed evidence review. This summary includes the following sections, although an advisory statement is included only if a commissioning policy is not being developed, and this has been agreed during scoping:

Advisory statement

A short evidence‑based statement about the medicine's likely place in therapy (which does not include a cost-effectiveness review), developed with input from external topic experts. It is advisory only: it is not a recommendation or formal NICE guidance.

Likely place in therapy

The development team summarises the medicine's likely place in therapy by bringing together all the key factors for decision making, in the context of the wider evidence base for managing the condition, particularly if NICE guidance is available.

The development team also takes into account the responses to questions (based on user feedback) put to the external topic experts after they review the draft evidence summary documents (including any supporting resources):

  • Which patients would be considered for this medicine?

  • Is there a particular care setting that would be best for this medicine (for example secondary care)?

  • Would the practical use of the medicine in a real-world setting differ from trial populations and settings? If so, how?

  • Would there need to be a significant change to the care pathway to support adoption of this medicine? If so, give details.

  • Are there any specific barriers or levers to the adoption of this medicine? (Other than general issues such as existing national guidance, views of opinion leaders and professional societies, and the evidence on the medicine.)

  • Are there any specific implications for people using this medicine, or their carers, compared with current practice? (For example, how convenient is it to take? Are there increased monitoring visits?)

  • Are there any specific implications around commissioning and procurement?

  • Would use of the medicine affect finance or budgets and staff capacity or resources?

  • Where do you see the place in therapy for this medicine?

The development team may finalise the likely place in therapy at a meeting with the external topic experts.

If a commissioning policy is being developed, the likely place in therapy section will not be completed. There will be a hyperlink to the policy from the NICE website.

Factors for decision making

A summary of the most important factors to be considered to help inform decision making are taken from the evidence review. This includes key information on effectiveness and safety evidence, limitations of the evidence, person-centred factors and resource impact. If the medicine is an antimicrobial, this will also include considerations about good antimicrobial stewardship.

6.8 Targeted review

The development team sends the draft evidence summary documents to:

  • targeted stakeholder organisations

  • identified external topic experts

  • the organisation commissioning the topic, if applicable

  • the company, if applicable

  • the MHRA, if applicable.

Any comments received are considered when revising the drafts. Actions are also recorded. Feedback to commentators is available on request to NICE.

6.9 Quality assurance

The evidence summary is quality assured by NICE staff who are not part of the development team. This involves a detailed check of all content, to ensure all sections contain statements and conclusions that are fair and balanced. The evidence summary must accurately reflect the evidence reviewed and be substantiated by an explicit and appropriate source of evidence.

The evidence summary is signed off by the programme director, clinical adviser or associate director.

If applicable, the company is given the opportunity to review the near-final draft to check for any factual errors (1 working day). Any necessary corrections are made by the development team.

NICE's guidance executive or publications executive reviews the evidence summary and, if appropriate, approves it for publication, ensuring that the process has been followed in its development. If applicable, the company is informed of the scheduled publication date, and may request an embargoed copy to be sent to them 24 hours before publication.

6.10 Publication

The final evidence summary documents (both the evidence review and the shorter evidence summary) are published on the NICE website. For published evidence summaries, see the NICE website. An alert to the published evidence summary is also circulated through the medicines and prescribing alert service.