This section covers the process for developing guidance in the HealthTech programme for HealthTech or interventional procedures guidance. Links are made to sections in NICE health technology evaluations: the manual as appropriate.
The process set out here will be used for developing interventional procedures guidance, superseding any process described in the interventional procedures programme manual.
NICE sends correspondence for an evaluation to key contacts identified by each stakeholder organisation. Stakeholders must notify NICE of any change in contact details, or in organisation or company name, during the evaluation. This and any other correspondence should be to the email address provided by NICE.
Companies with a technology being assessed must inform NICE as soon as possible of any significant new information relevant to the assessment that occurs during guidance development.
Technologies will not be withdrawn from a scope or guidance purely because of a company request.
Details on information handling, including confidential information, are described in sections 5.3 and 5.4 of NICE health technology evaluations: the manual. Further detail is available from NICE.
The price of a technology is important for economic evaluations. Companies can provide costs relevant to using their technology in their response to a request for information (see section 1.3.4) and updated costs at consultation on draft guidance (see section 1.5.5). Outside of these times it may not be possible to consider new or updated prices. If companies believe there are extenuating circumstances for why the technology cost cannot be disclosed in public documents, further information on these circumstances must be provided for NICE to consider whether this is acceptable. In circumstances when NICE agrees to accept a price marked as confidential, a further price that can be publicly disclosed should also be provided.
Guidance can include recommendations on a technology for which no price has been provided. But if the price is needed for an economic evaluation and cannot otherwise be determined, it will impact on recommendations made about the technology. This is because it leads to uncertainty about the cost effectiveness and budget impact.
The guidance development process starts after a topic has been selected and scheduled for NICE guidance development. It consists of 3 phases: scoping, assessment and developing recommendations. Subsequent process for finalising and publishing the guidance are described in section 7 of NICE health technology evaluations: the manual.
It is not possible to set absolute timelines for the phases of the process. The length of time needed for each phase can vary depending on the nature of the evaluation. Illustrative lengths are shown in table 1. A shorter process can typically be used for technologies assessed for early use and interventional procedures because typically the assessment phase can be shorter.
Stakeholders are encouraged to input at several stages. These are described in the process details in sections 1.3 to 1.5, and a summary is provided in table 1.
| Phase | Overview | Opportunities for stakeholder input |
|---|---|---|
|
Scoping |
Developing and finalising the assessment scope. This phase typically takes about 10 weeks. |
|
|
Assessment |
Producing an assessment report. Comment period on external assessment report (if produced). This phase typically takes between 12 and 30 weeks. |
|
|
Developing recommendations |
Committee meeting and producing draft guidance and final draft guidance. |
|
Throughout guidance development, up-to-date information about timelines and progress is published on the NICE website.
NICE informs stakeholders about timeline changes during an evaluation and the reasons for these changes. When the reasons are commercially sensitive, NICE works with the company to release as much information as possible to stakeholders and on the NICE website.
In exceptional circumstances, NICE may need to permanently stop guidance development. This decision is made by NICE. If guidance development is stopped, registered stakeholders are informed, and the NICE website is updated. Guidance production can be stopped for several reasons. This includes if it is no longer possible to produce recommendations, for example, if there have been changes to the regulatory status of technology.
Sections 2.3 and 2.4 of NICE health technology evaluations: the manual describe the initial steps in developing the draft scope, including identifying stakeholders.
During the scoping phase, NICE will speak to individuals and organisations to gather information needed to develop the draft scope. This can include healthcare professionals, committee members, patients and carers, companies with technologies that may be relevant to the assessment, voluntary and community sector organisations, and other organisations as necessary. A key activity is identifying technologies that may be relevant to the assessment, which includes asking for suggestions and input during any scoping workshop or scope consultations (see section 1.3.18).
Requests for information may be sent to companies during scoping if they have technologies that could be included in the assessment or otherwise be relevant to it (for example, for interventional procedures guidance, requests can be sent to companies producing devices that may be used to do the procedure). A request for information does not mean that a technology will be included in the scope for the assessment. Information provided is often used to determine if a technology is suitable to include in the scope.
Company evidence submissions are not made for HealthTech programme assessments. Instead, companies can be asked to provide responses to requests for information from NICE. Requests for information may be made as needed throughout the guidance development process.
Unpublished evidence can be provided with a request for information. See section 1.1.4 for information on how to provide confidential information to NICE.
A completed checklist of confidential information must be provided with a returned request for information.
Economic models can be submitted as part of the response to a request for information. But economic models may not be considered in the assessment period if they are not fully executable and using standard software, that is, Excel, DATA/Treeage, R or WinBUGs. When the company submits a fully executable electronic copy of the model, it must give NICE full access to the programming code and provide instructions on how to run the model.
A technology or procedure will not be withdrawn from a scope or guidance because a response to a request for information has not been received. But not providing information needed by NICE may affect the assessment of a technology or procedure and consequently the recommendation.
NICE can also invite non-company stakeholders or other organisations to provide evidence to inform scoping and the assessment. This can include qualitative, real-world and experiential evidence from voluntary and community sector organisations. This is to reflect the experience of patients, healthcare professionals and commissioners of current care in the NHS. It can also help understand the potential impact of using the new technology. Information on implementation issues, such as staffing and training needs, could also be provided.
The following experts can provide evidence, and their views and experience throughout the evaluation:
health and social care professionals
non-health and social care professionals (such as scientists, software specialists, data analysts, engineers or people with procurement or other technical experience, as needed)
people with a condition and their carers, who can provide information about the impact of both the condition and the technology being assessed
commissioning experts.
Experts will typically be selected during the scoping period but can also be selected later in the process if needed, for example, if gaps are identified in the knowledge and expertise needed by a committee.
Experts are selected from those nominated by consultee organisations or by NICE, taking into account NICE's policy on declaring and managing interests for NICE advisory committees. When necessary NICE may ask for expressions of interest to identify potential experts, particularly for patient experts.
Relevant NHS commissioners of the technology can be invited to nominate NHS commissioning experts if commissioning expertise is specifically needed or if the population is covered by an NHS England specialised commissioning group.
Sections 1.3.14 and 1.3.15 of NICE health technology evaluations: the manual describe the process of selecting experts and requirements that must be met.
The number of experts appointed will vary between guidance topics and will be informed by the knowledge and expertise needed by the committee. Typically, this would be up to 10 experts.
Experts help clarify issues that NICE has identified throughout guidance development (including during scoping) and can also provide further input as needed. Experts can attend committee meetings, and they may submit written evidence such as completed questionnaires.
In committee meetings experts are expected to interact fully in the discussions with the committee, including responding to questions. Section 1.3.19 of NICE health technology evaluations: the manual further explains the role of experts in committee meetings.
Experts are asked to leave the meeting before the committee makes its decision and finalises the guidance recommendations in the private session (part 2) of the committee meeting, which is closed to the public. The chair may ask experts to remain for part of the private session (part 2A) to respond to any questions from the committee about information that cannot be discussed in the public session (part 1).
After a draft scope is produced, NICE may hold a scoping workshop, have a consultation on the draft scope, or both.
A scoping workshop or draft scope consultation will not be held if NICE judges that there are no substantive uncertainties related to the scope to resolve.
Section 2.5 of NICE health technology evaluations: the manual describes the process for consultation on the draft scope. The consultation will be 5 to 10 working days, but can be extended to 20 working days if there is a higher level of uncertainty about elements of the draft scope.
For updates of existing guidance, NICE will update the original scope. This is to make sure that the guidance update considers the care pathway and use of the technology at the time the guidance update starts. NICE can review any element in the scope, including whether to expand the scope of the guidance update to include additional technologies.
When changes to the original scope are made, NICE may consult on a draft scope or hold a scoping workshop.
Guidance updates include any procedures or technologies conditionally recommended for use while more evidence is generated that are re-evaluated once this evidence is generated. This will be done according to the NICE HealthTech programme process and methods, and in the context of the healthcare system at the time of the guidance update, rather than at the time the original recommendation was made.
After any scoping workshop or consultation held has completed, NICE agrees the final scope. Section 2.9 of NICE health technology evaluations: the manual describes the process for finalising and issuing a scope.
If the scope for an evaluation is too large for the available resources, NICE may revise it in collaboration with experts and members of the committee. Input from stakeholders, including information provided by companies, will be considered in this decision. Input from an external assessment group (EAG), if appointed, will also be important to understand the work that can be done with the resource available.
NICE will publish the final scope on its website.
A decision will be made by NICE at the end of the scoping process about what guidance will be developed in terms of the lifecycle approach to be used (for early-, routine- or existing-use guidance; see section 2.1.28 for detail). This decision will be communicated in the final scope.
For topics with an EAG appointed (see section 1.4.5), this group develops an assessment protocol, derived from the final scope of the evaluation. The protocol will not be consulted on.
There can be circumstances when the final scope may need amending after it has been published on the NICE website. NICE decides whether to amend the scope.
If a final scope is amended after publication, registered stakeholders are informed. The revised scope and revised assessment protocol, if needed, are published on the NICE website. Further consultation on the scope would not usually be done.
The assessment may need to be paused. This may be because of external factors such as ongoing studies that will generate relevant evidence that will be available within or soon after the proposed guidance timeframe. NICE decides whether to pause the assessment period. Registered stakeholders are informed if the assessment period is paused.
An assessment report is generated to support guidance development. This report can be produced by either NICE or an EAG (see section 1.4.5). When produced by an EAG, this is an external assessment report, and the EAG is responsible for the content and quality of the report.
The length of the assessment period will be based on the expected amount or complexity of evidence and the extent of any economic evaluation needed. If this is more extensive than expected, the assessment period may need to be extended, and the scope may be updated (see sections 1.3.29 and 1.3.30).
Information provided during the assessment period that is not in response to a request for information from NICE or agreed in advance with NICE, may not be able to be considered in the assessment report.
EAGs can be commissioned to produce an external assessment report to support guidance production (see section 1.3.24 of NICE health technology evaluations: the manual for further description of EAGs). They can be used when there is a larger volume or complexity of evidence, or if more complex statistical analysis or an economic evaluation is needed.
Experts selected by NICE may also support the EAG during the evaluation. But they cannot be appointed as advisers to the EAG (that is, contribute to the EAG's work to the extent that they are authors on the assessment report). This is so they can maintain sufficient independence from the evidence and contribute to a committee's discussions on the quality of the external assessment report.
NICE will share a copy of the external assessment report with companies that have a named technology in the assessment (that is, the technology name is specified in the assessment scope as an intervention or comparator) for comment in advance of committee meetings. Comments should be submitted on issues of factual accuracy in the assessment report, and model if produced. Factual accuracy would include issues such as inaccuracies in reports or models. When produced the results from a user preference assessment will also be sent to companies with a named technology in the assessment at this time.
If an economic model is produced as part of the assessment, NICE offers to send the economic model (in its executable form) to the same companies that receive the external assessment report (as specified in section 1.4.7). If the model contains confidential material that the data owner is unwilling to share, despite the assurances provided through the signed confidentiality agreements, NICE will ask the group who have generated the model to replace this with dummy data or redact it if this can be done without severely limiting the model's function. A request for a copy of the model must be made in writing. NICE provides the model on the basis that the recipient agrees, in writing, to the conditions of use set out in section 5.5.16 of NICE health technology evaluations: the manual.
In exceptional circumstances it may not be possible to provide the economic model. For example, if it is not possible to do so without revealing confidential information.
Comments must be submitted in a 10 working day period.
If comments need an EAG response, NICE sends them to the EAG. Its responses will be presented at the next committee discussion.
The developing recommendations phase of the process has 4 possible stages:
consideration of the evidence at a committee meeting to discuss the content of the draft guidance
development of, and consultation on, the draft guidance
review of the draft guidance after comments from consultation
development of the final draft guidance.
The committee is described in sections 1.2.1 to 1.2.4 of NICE health technology evaluations: the manual.
Details on committee meetings are in sections 5.8.4 to 5.8.20 of NICE health technology evaluations: the manual. Committee papers are usually shared with committee members 1 week before the meeting. For HealthTech programme guidance, experts may make a presentation to the committee.
Details on the participation of company representatives at the committee meeting are in sections 1.3.5 and 1.3.6 of NICE health technology evaluations: the manual.
The draft guidance and committee papers are sent to stakeholders for consultation. These documents are confidential until NICE publishes them on its website. Information designated as confidential will be redacted from the documents.
The committee papers and the draft guidance document are made available during consultation on draft guidance. Section 5.8.45 of NICE health technology evaluations: the manual describes draft guidance, and section 5.8.48 describes the purpose of consultation.
Stakeholders have 15 working days from the date of sending to submit comments on the draft guidance.
NICE publishes the draft guidance and any additional committee papers not already shared on its website with an electronic comment facility within 5 working days of sending to stakeholders. The deadline for comments on the draft guidance from non-stakeholders is the same as for stakeholders.
After the draft guidance has been developed, new evidence provided to NICE will not be accepted. This is unless it is specifically requested by the committee, or if a stakeholder requests that NICE considers additional evidence and NICE specifically confirms it will accept it in writing. Responses to requests for information (see section 1.3.4) should be used to provide evidence to NICE.
The committee may be unable to develop recommendations without further scrutiny or further analyses. If this is the case, the evaluation can be paused. NICE may request that a company or EAG submits specific information, further analyses or an updated economic model.
The committee chair will review the consultation comments received. When the comments will not change the recommendations, the chair can decide that another committee meeting is not needed. This decision will be made in consultation with NICE. Factual changes and corrections to the guidance are made and final draft guidance and recommendations are agreed by the committee electronically.
The chair's decision will be shared with stakeholders. This will be a brief statement of the decision.
If needed the committee can meet again to consider the preliminary recommendations in the draft guidance with comments received. Before the meeting, NICE sends the committee members the full text of the comments from stakeholders. Sections 5.8.56 and 5.8.58 of NICE health technology evaluations: the manual describe the process of a further committee meeting.
When consultation comments are received that lead to a substantial revision of the committee's previous decision, involving a significant change in the recommendations, discussions or the evidence base, NICE and the committee chair will decide whether it is necessary to have a further draft guidance consultation. The decision to hold another consultation will extend the timelines for the evaluation. NICE will distribute any further committee papers with the second draft guidance, together with initial consultation comments. The process of a further consultation is the same as for the initial consultation.
Sections 5.8.64 to 5.8.66 of NICE health technology evaluations: the manual describe the process of developing final draft guidance.
For comments received on the draft guidance, NICE reserves the right to summarise and edit comments received during consultations. In exceptional circumstances, it can also decide to not publish them at all when, in the reasonable opinion of NICE, publication would be unlawful or otherwise inappropriate.
Sections 7.2 and 7.3 of NICE health technology evaluations: the manual describes the resolution and final guidance publication processes. The resolution process is a final quality-assurance step to ensure that NICE follows its own processes, and produces clear, accurate guidance. Definitions for 'stakeholders' and 'consultees', as mentioned in this section, can be found in sections 1.2.16 and 1.2.17 of NICE health technology evaluations: the manual.
Tools and resources can be produced to support guidance. This includes resource impact tools or statements. NICE's assessing resource impact process manual has further details.
During guidance development for interventional procedures (and other HealthTech guidance when useful and applicable), appropriate clinical classification codes for the procedure are identified and reviewed by the committee. These codes are published with guidance on the NICE website. NICE liaises with relevant partners to identify when a new code is needed for a procedure because no appropriate codes currently exist. New codes are published on the NICE website when they become available.
For interventional procedures guidance, an audit tool template for procedures is available on the NICE interventional procedures guidance webpage.
For technologies conditionally recommended for use while further evidence is generated in early-use HealthTech guidance, an evidence generation plan will be produced (see section 1.7).
The process of guidance surveillance is described in processes and methods for NICE-wide guidance surveillance, and section 8 of NICE health technology evaluations: the manual. This includes monitoring activities related to the safety of recommendations and any changes to regulatory status.
The evidence generation process is designed to help companies work with NHS sites, data custodians and analytical partners to generate evidence needed to support future NICE guidance. This process will start from the point of the decision to take a topic through early-use assessment and will support the development of guidance.
The evidence generation process will aim to deliver proportionate and pragmatic approaches to evidence generation. The evidence generated during the period of use in the NHS should provide the information needed for NICE to make a recommendation about routine use in the future.
NICE:
identifies uncertainties that are essential to resolve for future decision making and that should be prioritised for further evidence generation
assesses the feasibility of evidence generation while the technologies are used in the NHS
engages with stakeholders about ongoing or planned studies and considers if and how they could address the uncertainties
highlights NHS real-world data sources that could support or contribute to evidence generation
suggests an approach to evidence generation that could address the uncertainties
highlights potential sources of funding when NICE is aware of these
highlights potential partners that could support evidence generation, such as research groups, clinical networks or implementation specialists when NICE is aware of these
monitors progress of evidence generation.
Companies:
are responsible for addressing identified gaps in the evidence
are responsible for organising funding to support evidence generation
engage with and support the NICE evaluation and monitoring process
engage with partners to support evidence generation, by:
choosing appropriate NHS sites to generate the evidence
using robust approaches to evidence generation, considering aspects such as data quality, study design, analysis, and reporting and partnering with experts in research and analysis when necessary to ensure key uncertainties are addressed
ensuring new evidence is generated in accordance with all applicable data protection legislation
ensure that safety is monitored, and signals of concern are discussed with clinical leads and reported to the Medicines and Healthcare products Regulatory Agency and NICE as appropriate
minimise burden of data collection whenever possible, for example, by using real-world data collections that build on existing clinical information flows
consider advice laid out in NICE's health technology evaluation manual, real-world evidence framework and evidence standards framework for digital health technologies to inform evidence generation
make the evidence generated available to NICE in a form that can be used for decision making. For example, structuring and presenting findings as for a research publication, and ideally being able and ready to provide individual patient data if possible.
The feasibility assessment considers barriers and facilitators to addressing the likely uncertainties during a standard evidence generation period. It will be finalised shortly before the first committee meeting and will use information from the EAG report as well as information already gathered from topic selection and scoping stages.
The feasibility assessment considers the following aspects:
if key uncertainties could be resolved in a fixed period of 3 years from the point of guidance publication (longer periods than this will only be allowed in exceptional circumstances)
the likely number and complexity of new studies needed
facilitators that increase the likelihood that evidence generation will be successful (based on knowledge of relevant data sources, previously completed research, or known funding opportunities).
The feasibility assessment will be informed by:
uncertainties highlighted in the EAG report
consideration of the evidence landscape, including:
ongoing or planned studies
real-world data sources
consideration of methodological approaches to address the evidence gaps, for example, those outlined in NICE's real-world evidence framework
knowledge of existing funding sources
knowledge of potentially suitable implementation partners.
Key conclusions from the feasibility assessment can be presented to the committee.
The committee will identify the uncertainties that need to be addressed to support future NICE guidance on a technology. An evidence generation plan will be developed that describes the uncertainties and what evidence should be generated for a NICE re-evaluation of the technologies again in the future. It is not a study protocol but suggests an approach to generating the information needed to address the evidence gaps. The evidence generation plan will sit alongside the guidance.
Once early-use guidance is published, NICE will monitor the company's evidence generation activities. The monitoring process is designed to support companies to deliver the evidence that NICE needs and to support NICE planning for a future evaluation.
NICE has the right to withdraw or change individual technology recommendations at any stage. Information collected through the monitoring process will inform decision making about withdrawal. Reasons a recommendation to use a technology while further evidence is generated may be withdrawn include:
the technology is not available to the NHS
NICE is unable to contact the company
the company volunteers to withdraw
there are significant regulatory or safety concerns about the technology
the company is not engaging in evidence generation
evidence generation will not address the essential uncertainties.
The monitoring period will begin at the date of publication of the guidance and evidence generation plan. The monitoring process includes several touchpoints, which can vary in frequency as needed, but broadly will occur:
6 months after guidance publication: NICE will contact companies to confirm they are engaging with NICE processes and have begun evidence generation.
12 months after guidance publication: NICE can ask for a summary of overall progress with evidence generation and the status of data collection. Ideally, companies will share their study protocol and, when relevant, evidence of engagement with implementation partners.
Annually from 12 months: companies will be expected to report on their data collection. At this point they can also be asked if they consider that the evidence generated is sufficient to address the essential uncertainties.
In addition to routine monitoring, companies should inform NICE as soon as possible of anything that may significantly affect ongoing evidence generation, including:
any substantial risk that the evidence will not be collected as planned
any safety concerns
the technology significantly changing in a way that affects the evidence generation process.
If data collection is expected to end later than planned, the company should contact NICE.
Technologies recommended for use while further evidence is generated that complete the evidence generation process will be re-evaluated by NICE. This is to decide whether the technology can be recommended for routine use, considering the further evidence generated. Details on the process of scoping in this scenario are described in section 1.3.23. Re-evaluation follows the process for guidance production described in section 1.
As part of monitoring done during the evidence generation process, companies can submit evidence at the touchpoints if they consider the evidence generated is sufficient to address the essential uncertainties identified in the guidance and evidence generation plan (see section 1.7.12). NICE may consider evidence provided before the end of the evidence generation period. This will follow the surveillance review process set out in sections 8.1 and 8.2 of the NICE health technology evaluations: the manual. It will consider if re-evaluation of some or all of the technologies in the guidance should start before the end of the evidence generation period. When doing the surveillance review, NICE will consider the status of evidence generation for other technologies recommended for use with evidence generation in the same guidance and how close the end of the evidence generation period is. This may lead to the surveillance review being deferred to a later date to consider evidence generated by other companies, or not being done if it is likely completion would be close to or after the end of the evidence generation period. It is expected that most re-evaluations will take place after the full evidence generation period.