NICE HealthTech programme manual

  • NICE process and methods
  • PMG48
  • Published: 
  • Last updated: 
Download (PDF)

1 Processes for developing guidance in the HealthTech programme

This section covers the process for developing guidance in the HealthTech programme for HealthTech or interventional procedures guidance. Links are made to sections in NICE technology appraisal and highly specialised technologies guidance: the manual as appropriate.

The process set out here will be used for developing interventional procedures guidance, superseding any process described in the interventional procedures programme manual.

1.1 General information

1.1.1

NICE sends correspondence for an evaluation to key contacts identified by each stakeholder organisation. Stakeholders must notify NICE of any change in contact details, or in organisation or company name, during the evaluation. This and any other correspondence should be to the email address provided by NICE.

1.1.2

Companies with a technology being assessed must inform NICE as soon as possible of any significant new information relevant to the assessment that occurs during guidance development.

1.1.3

Technologies will not be withdrawn from a scope or guidance purely because of a company request.

Information handling

Technology costs

1.1.5

The price of a technology is important for economic evaluations. Companies can provide costs relevant to using their technology in their response to a request for information or an evidence request (see section 1.3.4) and updated costs at consultation on draft guidance (see section 1.5.5). Outside of these times it may not be possible to consider new or updated prices. If companies believe there are extenuating circumstances for why the technology cost cannot be disclosed in public documents, further information on these circumstances must be provided for NICE to consider whether this is acceptable. In circumstances when NICE agrees to accept a price marked as confidential, a further price that can be publicly disclosed should also be provided.

1.1.6

Guidance can include recommendations on a technology for which no price has been provided. But if the price is needed for an economic evaluation and cannot otherwise be determined, it will impact on recommendations made about the technology. This is because it leads to uncertainty about the cost effectiveness and budget impact.

1.2 Guidance development process overview

1.2.2

It is not possible to set absolute timelines for the phases of the process. The length of time needed for each phase can vary depending on the nature of the evaluation. Illustrative lengths are shown in table 1. A shorter process can typically be used for technologies assessed for early use and interventional procedures because typically the assessment phase can be shorter.

1.2.3

Stakeholders are encouraged to input at several stages. These are described in the process details in sections 1.3 to 1.5, and a summary is provided in table 1.

Table 1 Overview of the 3 phases of HealthTech programme guidance development
Phase Overview Opportunities for stakeholder input

Scoping

Developing and finalising the assessment scope.

This phase typically takes about 10 weeks.

  • Providing responses to any requests for information or other questions from NICE.

  • Providing comments on the draft scope during consultation (if held) or a scoping workshop (if held).

Assessment

Producing an assessment report. Comment period on external assessment report (if produced).

This phase typically takes between 12 and 30 weeks.

  • Providing responses to any requests for information, evidence requests or other questions from NICE.

  • Companies can submit comments on the factual accuracy of an external assessment report and any economic model produced.

Developing recommendations

Committee meeting and producing draft guidance and final draft guidance.

  • Attending committee meetings.

  • Submitting comments during a consultation period.

1.2.4

Throughout guidance development, up-to-date information about timelines and progress is published on the NICE website.

1.2.5

NICE informs stakeholders about timeline changes during an evaluation and the reasons for these changes. When the reasons are commercially sensitive, NICE works with the company to release as much information as possible to stakeholders and on the NICE website.

Stopping guidance development

1.2.6

In exceptional circumstances, NICE may need to permanently stop guidance development. This decision is made by NICE. If guidance development is stopped, registered stakeholders are informed, and the NICE website is updated. Guidance production can be stopped for several reasons. This includes if it is no longer possible to produce recommendations, for example, if there have been changes to the regulatory status of technology.

1.3 Scoping

1.3.2

During the scoping phase, NICE will speak to individuals and organisations to gather information needed to develop the draft scope. This can include healthcare professionals, committee members, patients and carers, companies with technologies that may be relevant to the assessment, voluntary and community sector organisations, and other organisations as necessary. A key activity is identifying technologies that may be relevant to the assessment, which includes asking for suggestions and input during any scoping workshop or scope consultations (see section 1.3.18).

1.3.3

Requests for information may be sent to companies during scoping if they have technologies that could be included in the assessment or otherwise be relevant to it (for example, for interventional procedures guidance, requests can be sent to companies producing devices that may be used to do the procedure). A request for information does not mean that a technology will be included in the scope for the assessment. Information provided is often used to determine if a technology is suitable to include in the scope.

Requests for information and evidence

1.3.4

Company evidence submissions are not made for HealthTech programme assessments. Instead, companies can be asked to provide responses to requests for information and evidence requests. Requests for information may be made as needed throughout the guidance development process but are typically made during the scoping phase. Evidence requests are made after the scope publishes.

1.3.5

Unpublished evidence can be provided with a returned request for information or evidence request. See section 1.1.4 for information on how to provide confidential information to NICE.

1.3.6

A completed checklist of confidential information must be provided with a returned request for information or evidence request.

1.3.7

Economic models can be submitted as part of the response to an evidence request. But economic models may not be considered in the assessment period if they are not fully executable and using standard software, that is, Excel, DATA/Treeage, R or WinBUGs. When the company submits a fully executable electronic copy of the model, it must give NICE full access to the programming code and provide instructions on how to run the model.

1.3.8

A technology or procedure will not be withdrawn from a scope or guidance because a response to a request for information or evidence request has not been received. But not providing information needed by NICE may affect the assessment of a technology or procedure and consequently the recommendation.

Information provided by non-company stakeholders or other organisations

1.3.9

NICE can also invite non-company stakeholders or other organisations to provide evidence to inform scoping and the assessment. This can include qualitative, real-world and experiential evidence from voluntary and community sector organisations. This is to reflect the experience of patients, healthcare professionals and commissioners of current care in the NHS. It can also help understand the potential impact of using the new technology. Information on implementation issues, such as staffing and training needs, could also be provided.

Experts

1.3.10

The following experts can provide evidence, and their views and experience throughout the evaluation:

  • health and social care professionals

  • non-health and social care professionals (such as scientists, software specialists, data analysts, engineers or people with procurement or other technical experience, as needed)

  • people with a condition and their carers, who can provide information about the impact of both the condition and the technology being assessed

  • commissioning experts.

    Experts will typically be selected during the scoping period but can also be selected later in the process if needed, for example, if gaps are identified in the knowledge and expertise needed by a committee.

Identifying experts
1.3.12

Relevant NHS commissioners of the technology can be invited to nominate NHS commissioning experts if commissioning expertise is specifically needed or if the population is covered by an NHS England specialised commissioning group.

Expert eligibility and selection
1.3.14

The number of experts appointed will vary between guidance topics and will be informed by the knowledge and expertise needed by the committee. Typically, this would be up to 10 experts.

Expert participation
1.3.15

Experts help clarify issues that NICE has identified throughout guidance development (including during scoping) and can also provide further input as needed. Experts can attend committee meetings, and they may submit written evidence such as completed questionnaires.

1.3.17

Experts are asked to leave the meeting before the committee makes its decision and finalises the guidance recommendations in the private session (part 2) of the committee meeting, which is closed to the public. The chair may ask experts to remain for part of the private session (part 2A) to respond to any questions from the committee about information that cannot be discussed in the public session (part 1).

Draft scope: scoping workshops and scope consultations

1.3.18

After a draft scope is produced, NICE may hold a scoping workshop, have a consultation on the draft scope, or both.

1.3.19

A scoping workshop or draft scope consultation will not be held if NICE judges that there are no substantive uncertainties related to the scope to resolve.

Scoping for guidance updates

1.3.21

For updates of existing guidance, NICE will update the original scope. This is to make sure that the guidance update considers the care pathway and use of the technology at the time the guidance update starts. NICE can review any element in the scope, including whether to expand the scope of the guidance update to include additional technologies.

1.3.22

When changes to the original scope are made, NICE may consult on a draft scope or hold a scoping workshop.

1.3.23

Guidance updates include any procedures or technologies conditionally recommended for use while more evidence is generated that are re-evaluated once this evidence is generated. This will be done according to the NICE HealthTech programme process and methods, and in the context of the healthcare system at the time of the guidance update, rather than at the time the original recommendation was made.

Final scope

1.3.25

If the scope for an evaluation is too large for the available resources, NICE may revise it in collaboration with experts and members of the committee. Input from stakeholders, including information provided by companies, will be considered in this decision. Input from an external assessment group (EAG), if appointed, will also be important to understand the work that can be done with the resource available.

1.3.26

NICE will publish the final scope on its website.

1.3.27

A decision will be made by NICE at the end of the scoping process about what guidance will be developed in terms of the lifecycle approach to be used (for early-, routine- or existing-use guidance; see section 2.1.28 for detail). This decision will be communicated in the final scope.

Assessment protocol

1.3.28

For topics with an EAG appointed (see section 1.4.5), this group develops an assessment protocol, derived from the final scope of the evaluation. The protocol will not be consulted on.

Amending the final scope after publication on the NICE website

1.3.29

There can be circumstances when the final scope may need amending after it has been published on the NICE website. NICE decides whether to amend the scope.

1.3.30

If a final scope is amended after publication, registered stakeholders are informed. The revised scope and revised assessment protocol, if needed, are published on the NICE website. Further consultation on the scope would not usually be done.

1.4 Assessment period

1.4.1

The assessment may need to be paused. This may be because of external factors such as ongoing studies that will generate relevant evidence that will be available within or soon after the proposed guidance timeframe. NICE decides whether to pause the assessment period. Registered stakeholders are informed if the assessment period is paused.

1.4.2

An assessment report is generated to support guidance development. This report can be produced by either NICE or an EAG (see section 1.4.5). When produced by an EAG, this is an external assessment report, and the EAG is responsible for the content and quality of the report.

1.4.3

The length of the assessment period will be based on the expected amount or complexity of evidence and the extent of any economic evaluation needed. If this is more extensive than expected, the assessment period may need to be extended, and the scope may be updated (see sections 1.3.29 and 1.3.30).

1.4.4

Information provided during the assessment period that is not in response to a request for information or evidence request from NICE or agreed in advance with NICE, may not be able to be considered in the assessment report.

EAGs

1.4.6

Experts selected by NICE may also support the EAG during the evaluation. But they cannot be appointed as advisers to the EAG (that is, contribute to the EAG's work to the extent that they are authors on the assessment report). This is so they can maintain sufficient independence from the evidence and contribute to a committee's discussions on the quality of the external assessment report.

Factual accuracy checks for an external assessment report

1.4.7

NICE will share a copy of the external assessment report with companies that have a named technology in the assessment (that is, the technology name is specified in the assessment scope as an intervention or comparator) for comment in advance of committee meetings. Comments should be submitted on issues of factual accuracy in the assessment report, and model if produced. Factual accuracy would include issues such as inaccuracies in reports or models. When produced the results from a user preference assessment will also be sent to companies with a named technology in the assessment at this time.

1.4.8

If an economic model is produced as part of the assessment, NICE offers to send the economic model (in its executable form) to the same companies that receive the external assessment report (as specified in section 1.4.7). If the model contains confidential material that the data owner is unwilling to share, despite the assurances provided through the signed confidentiality agreements, NICE will ask the group who have generated the model to replace this with dummy data or redact it if this can be done without severely limiting the model's function. A request for a copy of the model must be made in writing. NICE provides the model on the basis that the recipient agrees, in writing, to the conditions of use set out in section 5.5.16 of NICE technology appraisal and highly specialised technologies guidance: the manual.

1.4.9

In exceptional circumstances it may not be possible to provide the economic model. For example, if it is not possible to do so without revealing confidential information.

1.4.10

Comments must be submitted in a 10 working day period.

1.4.11

If comments need an EAG response, NICE sends them to the EAG. Its responses will be presented at the next committee discussion.

1.5 Developing recommendations

1.5.1

The developing recommendations phase of the process has 4 possible stages:

  • consideration of the evidence at a committee meeting to discuss the content of the draft guidance

  • development of, and consultation on, the draft guidance

  • review of the draft guidance after comments from consultation

  • development of the final draft guidance.

Committee meetings

Preparing for the committee meeting
Committee meetings

Consultation on the draft guidance

1.5.5

The draft guidance and committee papers are sent to stakeholders for consultation. These documents are confidential until NICE publishes them on its website. Information designated as confidential will be redacted from the documents.

1.5.7

Stakeholders have 15 working days from the date of sending to submit comments on the draft guidance.

1.5.8

NICE publishes the draft guidance and any additional committee papers not already shared on its website with an electronic comment facility within 5 working days of sending to stakeholders. The deadline for comments on the draft guidance from non-stakeholders is the same as for stakeholders.

1.5.9

After the draft guidance has been developed, new evidence provided to NICE will not be accepted. This is unless it is specifically requested by the committee, or if a stakeholder requests that NICE considers additional evidence and NICE specifically confirms it will accept it in writing. Responses to requests for information or evidence requests (see section 1.3.4) should be used to provide evidence to NICE.

1.5.10

The committee may be unable to develop recommendations without further scrutiny or further analyses. If this is the case, the evaluation can be paused. NICE may request that a company or EAG submits specific information, further analyses or an updated economic model.

After draft guidance consultation

1.5.11

The committee chair will review the consultation comments received. When the comments will not change the recommendations, the chair can decide that another committee meeting is not needed. This decision will be made in consultation with NICE. Factual changes and corrections to the guidance are made and final draft guidance and recommendations are agreed by the committee electronically.

1.5.12

The chair's decision will be shared with stakeholders. This will be a brief statement of the decision.

1.5.14

When consultation comments are received that lead to a substantial revision of the committee's previous decision, involving a significant change in the recommendations, discussions or the evidence base, NICE and the committee chair will decide whether it is necessary to have a further draft guidance consultation. The decision to hold another consultation will extend the timelines for the evaluation. NICE will distribute any further committee papers with the second draft guidance, together with initial consultation comments. The process of a further consultation is the same as for the initial consultation.

Developing final draft guidance

1.5.16

For comments received on the draft guidance, NICE reserves the right to summarise and edit comments received during consultations. In exceptional circumstances, it can also decide to not publish them at all when, in the reasonable opinion of NICE, publication would be unlawful or otherwise inappropriate.

Resolution

1.5.18

The resolution process is a final quality-assurance step to ensure that NICE acts fairly, follows its own processes, and produces clear, accurate guidance. It happens after NICE has approved the final draft guidance for publication and before it is published. After approval, NICE sends all stakeholders the final draft guidance. Resolution does not apply to decisions about selecting technologies for evaluation. It also does not apply to the external assessment report or other documents produced during guidance development, unless the resolution request on these documents is important for an issue in the guidance itself.

1.5.19

After receiving the final draft guidance, any stakeholder can ask for factual errors to be corrected. Only consultees can raise a resolution request based on a breach of the published process.

1.5.20

If NICE either does not receive a resolution request, or receives a request that can be resolved quickly, the guidance is published as soon as possible after the resolution period ends. If NICE receives a resolution request that needs further investigation, it suspends publishing the guidance while it investigates the request and informs stakeholders of the delay to publication.

Grounds for resolution

1.5.21

NICE only considers resolution requests that clearly meet one or both of the following grounds:

  • Ground 1: Breach of NICE's published process for the development of guidance.

  • Ground 2: Factual errors in the guidance.

1.5.22

A factual error is an objective error of material fact in the final draft guidance. Conflicting scientific or clinical interpretations or judgements are not considered to be factual errors. For example, if a resolution request states that a statistic quoted in the guidance is incorrect, NICE establishes whether the final guidance misquoted the statistic, or if one statistic was preferred out of several because the committee considered it to be more reliable. The former is a factual error; the latter is a difference of scientific or clinical judgement.

Making a resolution request

1.5.23

NICE sends the final draft guidance and, when a draft guidance consultation has taken place, any consultation comments and NICE's response to those comments, to all stakeholders.

1.5.24

Eligible stakeholders must make a resolution request on one or both of the grounds within 21 days. Requests should specify the resolution they seek. NICE can then fully understand the nature of their concern and take appropriate action.

Initial scrutiny

1.5.25

All eligible resolution requests are subject to an initial scrutiny process. NICE investigates the matters raised and decides whether the request is in the scope of the resolution process. Initial scrutiny continues for 21 days after the resolution request period ends. If multiple resolution requests are made, either from the same or different sources, each request is treated separately.

Ground 1: breach of process

1.5.26

If the programme director considers that the resolution request does not meet ground 1 (breach of process), or does not have a reasonable prospect of success, NICE informs the consultee that made the request and publishes the guidance.

1.5.27

If the programme director considers that ground 1 appears to have been met, a resolution panel is convened.

Ground 2: factual errors

1.5.28

If the associate director considers that the resolution request does not meet ground 2 (factual errors), or does not have a reasonable prospect of success, the person or organisation that made the request is informed and NICE publishes the guidance.

1.5.29

If the associate director considers that the guidance contains a factual error or a point that needs clarification, but this does not affect the committee's recommendations, the guidance is amended and signed off internally without being referred to a resolution panel. NICE then publishes the final guidance.

1.5.30

If the associate director considers that there may be a major factual error that may affect the committee's recommendations, the programme director will convene a resolution panel.

1.5.31

If there are multiple resolution requests, not all requests may qualify to be referred to a resolution panel. To avoid pre-empting the outcome of resolution, NICE informs everyone who has submitted a resolution request that the panel will be convened, and that NICE will tell them the outcome of their request after the panel's decision is made.

Table 2 Initial scrutiny of resolution requests
Outcome of initial scrutiny NICE action

Ground 1 not met

Guidance is published

Ground 1 met

Resolution panel is convened

Ground 2 not met

Guidance is published

Ground 2 met, minor factual error

Guidance is amended and published

Ground 2 met, major factual error

Resolution panel is convened

The resolution panel

1.5.32

The panel consists of 2 NICE board members: 1 non-executive director and 1 executive director not previously involved in developing guidance on the technology. The panel is to decide whether there has been a breach of process or factual error and, if so, what action is appropriate.

1.5.33

The resolution panel meeting is held within 35 days after the initial scrutiny process. The meeting is usually held virtually. The NICE team prepares a briefing, which the panel uses when considering resolution requests. For ground 1, this means establishing what process was followed when developing the guidance and what events or omissions are alleged in the resolution request. In the case of ground 2, this involves setting out what evidence is behind the alleged errors.

1.5.34

The briefing is shared with the consultee making the resolution request. They have 10 days to comment on the briefing, then their comments are provided to the panel.

1.5.35

The resolution panel may hold a meeting where the panel members meet (without other parties) to consider the written evidence and make a decision. The panel may decide to hold an oral meeting where both the NICE team and the consultee attend to answer the panel questions and provide clarification. Committee members may also attend. These attendees are not members of the panel and do not contribute to the outcome of the resolution. Consultees cannot bring legal representation to the panel meeting.

Resolution outcome

Ground 1: breach of process
1.5.36

If the resolution panel decides that there has been no breach of process, NICE can publish the final guidance. If the panel decides that there has been a breach of process, it decides what action is appropriate. This may involve repeating part of the evaluation process and, if necessary, referring the guidance back to the committee or doing another consultation, or both.

Ground 2: major factual errors
1.5.37

If the resolution panel decides that there are no factual errors, NICE can publish the final guidance. If the panel decides that there are factual errors or elements to be clarified, NICE produces an amended version of the guidance. The panel decides whether the error can be corrected and the amended version of the guidance approved by NICE before publication, or whether the committee should review the wording of the amended guidance because of the error.

1.5.38

NICE considers whether to publish the amended guidance or whether there is a need for further consultation. Further consultation normally happens if:

  • NICE makes a substantive change to the wording of the recommendations, or

  • changes to the guidance not involving the recommendations are significant or likely to be of interest to the people who made the resolution request.

Table 3 Outcome of resolution panel
Outcome of resolution panel meeting NICE action

Ground 1 not met

Guidance is published

Ground 1 met

Appropriate action as decided by resolution panel

Ground 2 not met

Guidance is published

Ground 2 met

Appropriate action as decided by resolution panel
1.5.39

NICE implements the panel's decision and informs everyone who made resolution requests of the resolution outcome. This normally happens within 7 days of the panel reaching its final decision. This timescale does not apply if the committee needs to reconsider the recommendations. The resolution panel's decision is final and there are no further opportunities for redress within NICE.

Publishing the guidance

1.5.40

Once the resolution process is complete and any changes to guidance following those processes are complete, final guidance is published on the NICE website and all stakeholders are informed. NICE also publishes a lay version for patients and carers, known as 'information for the public'.

1.5.41

The following documents are available on the NICE website when guidance is published (all confidential information will be removed from the documents before publication):

  • guidance

  • external assessment report, any additional analysis and clarification questions and responses

  • any evidence submissions

  • consultation comments (anonymised) and NICE's responses

  • further analysis or correction, if any, done by NICE or the external assessment group after the external assessment report (in an addendum)

  • implementation support tools (usually at the same time as the guidance, and within 3 months of publication at the latest) when the technology is recommended (as an option)

  • equality impact assessment

  • a lay explanation of the recommendations.

1.5.42

If NICE is advised of any potential errors in the guidance or the supporting documents after publication, these are dealt with according to NICE's standard procedures.

Tools and resources

1.5.44

During guidance development for interventional procedures (and other HealthTech guidance when useful and applicable), appropriate clinical classification codes for the procedure are identified and reviewed by the committee. These codes are published with guidance on the NICE website. NICE liaises with relevant partners to identify when a new code is needed for a procedure because no appropriate codes currently exist. New codes are published on the NICE website when they become available.

Evidence generation plans
1.5.46

For technologies conditionally recommended for use while further evidence is generated in early-use HealthTech guidance, an evidence generation plan will be produced (see section 1.7).

1.6 Guidance surveillance

1.7 Evidence generation process for early-use HealthTech guidance

Overview

1.7.1

The evidence generation process is designed to help companies work with NHS sites, data custodians and analytical partners to generate evidence needed to support future NICE guidance. This process will start from the point of the decision to take a topic through early-use assessment and will support the development of guidance.

1.7.2

The evidence generation process will aim to deliver proportionate and pragmatic approaches to evidence generation. The evidence generated during the period of use in the NHS should provide the information needed for NICE to make a recommendation about routine use in the future.

Stakeholder roles for the evidence generation process

1.7.3

NICE:

  • identifies uncertainties that are essential to resolve for future decision making and that should be prioritised for further evidence generation

  • assesses the feasibility of evidence generation while the technologies are used in the NHS

  • engages with stakeholders about ongoing or planned studies and considers if and how they could address the uncertainties

  • highlights NHS real-world data sources that could support or contribute to evidence generation

  • suggests an approach to evidence generation that could address the uncertainties

  • highlights potential sources of funding when NICE is aware of these

  • highlights potential partners that could support evidence generation, such as research groups, clinical networks or implementation specialists when NICE is aware of these

  • monitors progress of evidence generation.

1.7.4

Companies:

  • are responsible for addressing identified gaps in the evidence

  • are responsible for organising funding to support evidence generation

  • engage with and support the NICE evaluation and monitoring process

  • engage with partners to support evidence generation, by:

    • choosing appropriate NHS sites to generate the evidence

    • using robust approaches to evidence generation, considering aspects such as data quality, study design, analysis, and reporting and partnering with experts in research and analysis when necessary to ensure key uncertainties are addressed

    • ensuring new evidence is generated in accordance with all applicable data protection legislation

  • ensure that safety is monitored, and signals of concern are discussed with clinical leads and reported to the Medicines and Healthcare products Regulatory Agency and NICE as appropriate

  • minimise burden of data collection whenever possible, for example, by using real-world data collections that build on existing clinical information flows

  • consider advice laid out in NICE's technology appraisal and highly specialised technologies guidance: the manual, real-world evidence framework and evidence standards framework for digital health technologies to inform evidence generation

  • make the evidence generated available to NICE in a form that can be used for decision making. For example, structuring and presenting findings as for a research publication, and ideally being able and ready to provide individual patient data if possible.

The evidence generation process

Feasibility assessment
1.7.5

The feasibility assessment considers barriers and facilitators to addressing the likely uncertainties during a standard evidence generation period. It will be finalised shortly before the first committee meeting and will use information from the EAG report as well as information already gathered from topic selection and scoping stages.

1.7.6

The feasibility assessment considers the following aspects:

  • if key uncertainties could be resolved in a fixed period of 3 years from the point of guidance publication (longer periods than this will only be allowed in exceptional circumstances)

  • the likely number and complexity of new studies needed

  • facilitators that increase the likelihood that evidence generation will be successful (based on knowledge of relevant data sources, previously completed research, or known funding opportunities).

1.7.7

The feasibility assessment will be informed by:

  • uncertainties highlighted in the EAG report

  • consideration of the evidence landscape, including:

    • ongoing or planned studies

    • real-world data sources

  • consideration of methodological approaches to address the evidence gaps, for example, those outlined in NICE's real-world evidence framework

  • knowledge of existing funding sources

  • knowledge of potentially suitable implementation partners.

1.7.8

Key conclusions from the feasibility assessment can be presented to the committee.

Evidence generation plan

1.7.9

The committee will identify the uncertainties that need to be addressed to support future NICE guidance on a technology. An evidence generation plan will be developed that describes the uncertainties and what evidence should be generated for a NICE re-evaluation of the technologies again in the future. It is not a study protocol but suggests an approach to generating the information needed to address the evidence gaps. The evidence generation plan will sit alongside the guidance.

Evidence generation monitoring

1.7.10

Once early-use guidance is published, NICE will monitor the company's evidence generation activities. The monitoring process is designed to support companies to deliver the evidence that NICE needs and to support NICE planning for a future evaluation.

1.7.11

NICE has the right to withdraw or change individual technology recommendations at any stage. Information collected through the monitoring process will inform decision making about withdrawal. Reasons a recommendation to use a technology while further evidence is generated may be withdrawn include:

  • the technology is not available to the NHS

  • NICE is unable to contact the company

  • the company volunteers to withdraw

  • there are significant regulatory or safety concerns about the technology

  • the company is not engaging in evidence generation

  • evidence generation will not address the essential uncertainties.

1.7.12

The monitoring period will begin at the date of publication of the guidance and evidence generation plan. The monitoring process includes several touchpoints, which can vary in frequency as needed, but broadly will occur:

  • 6 months after guidance publication: NICE will contact companies to confirm they are engaging with NICE processes and have begun evidence generation.

  • 12 months after guidance publication: NICE can ask for a summary of overall progress with evidence generation and the status of data collection. Ideally, companies will share their study protocol and, when relevant, evidence of engagement with implementation partners.

  • Annually from 12 months: companies will be expected to report on their data collection. At this point they can also be asked if they consider that the evidence generated is sufficient to address the essential uncertainties.

1.7.13

In addition to routine monitoring, companies should inform NICE as soon as possible of anything that may significantly affect ongoing evidence generation, including:

  • any substantial risk that the evidence will not be collected as planned

  • any safety concerns

  • the technology significantly changing in a way that affects the evidence generation process.

1.7.14

If data collection is expected to end later than planned, the company should contact NICE.