Closed for comments This consultation ended on at Request commenting lead permission
This section covers the process for developing guidance in the HealthTech programme. Links are made to sections in NICE health technology evaluations: the manual as appropriate. This guide supersedes other sections in the manual, including interim statements, for guidance produced in the HealthTech programme (excluding the late stage assessment interim methods and process statement).
The process set out here will also be used for developing interventional procedures guidance, superseding any process described in the interventional procedures programme manual.
NICE sends correspondence for an evaluation to key contacts identified by each stakeholder organisation. Stakeholders (see section 1.2.16 in NICE health technology evaluations: the manual) must notify NICE of any change in contact details, or in organisation or company name, during the evaluation. This and any other correspondence should be to the email address provided by NICE.
Companies with a technology being assessed must inform NICE as soon as possible of any significant new information relevant to the assessment that occurs during guidance development.
Technologies will not be withdrawn from a scope or guidance purely because of a company request.
Details on information handling, including confidential information, is described in sections 5.3 and 5.4 of NICE health technology evaluations: the manual. Further detail is available from NICE.
The cost of a technology is important for economic evaluations. Companies can provide costs relevant to use of their technology in their response to a request for information (see section 1.3.4) and at consultation on draft guidance (see section 1.5.5). Outside of these times it may not be possible to consider new or updated prices. Because the technology price is likely to be important for decision making it should not be marked as confidential. If companies believe there are extenuating circumstances for why the technology cost cannot be disclosed in public documents, further information on these circumstances must be provided for NICE to consider whether this is acceptable. In circumstances where NICE agrees to accept a price marked as confidential, a further price that can be publicly disclosed should also be provided.
Guidance can include recommendations on a technology for which no price has been provided. But if this cost is needed for an economic evaluation and the cost cannot otherwise be determined, it can prevent the technology being recommended for use in the NHS, because it leads to uncertainty about the cost effectiveness and budget impact.
The guidance development process starts after a topic has been selected and scheduled for NICE guidance development. It consists of 3 phases: scoping, assessment and developing recommendations. Subsequent process for finalising and publishing the guidance are described in section 7 of NICE health technology evaluations: the manual.
It is not possible to set absolute timelines for the phases of the process. The length of time needed for each phase can vary depending on the nature of the evaluation. Illustrative lengths are shown in table 1. A shorter process can be used for technologies assessed for early use and interventional procedures because typically the assessment phase can be shorter.
Stakeholders are encouraged to input at several stages. These are described in the following process details, and a summary is provided in table 1.
Phase | Overview | Opportunities for stakeholder input |
|---|---|---|
Scoping | Developing and finalising the assessment scope. This phase typically takes about 10 weeks. |
|
Assessment | Producing an assessment report. This phase typically takes between 12 and 30 weeks. |
|
Developing recommendations | Committee meeting and producing draft guidance and final draft guidance. |
|
Throughout guidance development, up-to-date information about timelines and progress is published on the NICE website.
NICE informs stakeholders about timeline changes during an evaluation and the reasons for these changes. When the reasons are commercially sensitive, NICE works with the company to release as much information as possible to stakeholders and on the NICE website.
In exceptional circumstances, NICE may need to permanently stop guidance development. This decision is made by NICE. If guidance development is stopped, registered stakeholders are informed, and the NICE website is updated. Guidance production can be stopped for several reasons. This includes if producing the guidance is no longer considered useful or a priority for the healthcare system, or if it is no longer possible to produce recommendations, for example because no technologies being considered have appropriate regulatory approval.
Sections 2.3 and 2.4 of NICE health technology evaluations: the manual describe the initial steps in developing the draft scope, including identifying stakeholders.
During the scoping phase, NICE will speak to individuals and organisations to gather information needed to develop the draft scope. This can include healthcare professionals, committee members, patients and carers, companies with technologies that may be relevant to the assessment, and other organisations as necessary. A key activity is identifying technologies that may be relevant to the assessment, which includes asking for suggestions and input during any scoping workshop or scope consultations (see section 1.3.19).
Requests for information may be sent to companies during scoping if they have technologies that could be included in the assessment or otherwise be relevant to it (for example, for interventional procedures guidance, requests can be sent to companies producing devices that may be used to do the procedure). Being asked for, and providing, a request for information does not mean that a technology will be included in the scope for the assessment. Information provided is often used to determine if a technology is suitable to include in the scope.
Company evidence submissions are not made for HealthTech programme guidance. Instead, companies can be asked to provide responses to requests for information from NICE. Requests for information may be made as needed throughout the guidance development process.
Unpublished evidence can be provided with a request for information. See section 1.1.4 for information on how to provide confidential information to NICE.
A completed checklist of confidential information must be provided with a returned request for information.
Economic models can be submitted as part of the response to a request for information. But economic models may not be considered in the assessment period if they are not fully executable and using standard software, that is, Excel, DATA/Treeage, R or WinBUGs. When the company submits a fully executable electronic copy of the model, it must give NICE full access to the programming code and provide instructions on how to run the model.
It is not essential to provide a response to a request for information. Technology will not be withdrawn from a scope or guidance because a response has not been received. But not providing information needed by NICE may affect the assessment of a technology and consequently the recommendation.
Completed requests for information provided by a company can be shared with the committee, experts (see section 1.3.11) and an external assessment group (EAG; see section 1.4.5).
NICE can also invite non-company stakeholders or other organisations to provide evidence to inform scoping and the assessment. This is to reflect the experience of patients, healthcare professionals and commissioners of current care in the NHS. It can also help understand the potential impact of using the new technology. Information on implementation issues, such as staffing and training needs, could also be provided.
The following experts can provide evidence, their views and experience throughout the evaluation:
clinical experts
relevant non-clinical experts (such as scientists, software specialists, data analysts, engineers or people with procurement or other technical experience, as needed)
people with a condition and their carers, who can provide information about the impact of both the condition and the technology being assessed
commissioning experts.
Experts will typically be selected during the scoping period but can also be selected later in the process if needed, for example if gaps are identified in the knowledge and expertise needed by a committee.
Experts are selected from those nominated by consultee organisations or by NICE, taking into account the NICE policy on declaring and managing interests for NICE advisory committees.
Relevant NHS commissioners of the technology can be invited to nominate NHS commissioning experts if commissioning expertise is specifically needed or if the population is covered by an NHS England specialised commissioning group.
Sections 1.3.17 and 1.3.18 of NICE health technology evaluations: the manual describe the process of selecting experts and requirements that must be met.
The number of experts appointed will vary between guidance topics and will be informed by the knowledge and expertise needed by the committee. Typically, this would be between 3 and 10 experts.
Experts help clarify issues that NICE has identified throughout guidance development (including during scoping) and can also provide further input as needed. Experts can attend committee meetings, and they may submit written evidence such as completed questionnaires.
In committee meetings experts are expected to interact fully in the discussions with the committee, including responding to questions. Section 1.3.24 of NICE health technology evaluations: the manual further explains the role of experts in committee meetings.
Experts are asked to leave the meeting before the committee makes its decision and finalises the recommendations in the guidance in the private session (part 2) of the meeting, which is closed to the public. The chair may ask experts to remain for part of the private session (part 2A) of the committee meeting to respond to any questions from the committee about information that cannot be discussed in part 1.
After a draft scope is produced, NICE may hold a scoping workshop, have a consultation on the draft scope, or both.
A scoping workshop or draft scope consultation will not be held if NICE judges that there are no substantive uncertainties related to scope to resolve.
Section 2.5 of NICE health technology evaluations: the manual describes the process for consultation on the draft scope. The consultation will be 7 to 14 calendar days, but can be extended to 28 calendar days if there is a higher level of uncertainty about elements of the draft scope.
For updates of existing guidance, NICE will update the original scope. This is to make sure that the guidance update considers the care pathway and use of the technology at the time the guidance update starts. NICE can review any element in the scope, including whether to expand the scope of the guidance update to include additional technologies.
When changes to the original scope are made, NICE may consult on a draft scope or hold a scoping workshop.
Guidance updates include any procedures or technologies recommended for use while more evidence is generated that are re-evaluated once this evidence is generated. This will be done according to the NICE HealthTech programme process and methods, and in the context of the healthcare system at the time of the guidance update, rather than at the time the original recommendation was made.
After any scoping workshop or consultation held has completed, NICE agrees the final scope. Section 2.9 of NICE health technology evaluations: the manual describes the process for finalising and issuing a scope.
If the scope for an evaluation is too large for the available resources, NICE may revise it in collaboration with experts and members of the committee.
NICE will publish the final scope on its website at the start of an evaluation.
A decision will be made at the end of the scoping process if a topic is suitable for early use assessment. This decision will be communicated in the final scope. Topics likely to be suitable for early use assessment are those that:
address a significant unmet need
have a lower level of evidence available to support using the technologies and
have system support for using the technologies while further evidence is generated.
For topics with an EAG appointed (see section 1.4.5), this group develops an assessment protocol, derived from the final scope of the evaluation. This will be published at the same time as or soon after the scope is published. The protocol will not be consulted on.
There can be circumstances when the final scope may need amending after it has been published on the NICE website. NICE decides whether to amend the scope.
If a final scope is amended after publication, registered stakeholders are informed. The revised scope and revised assessment protocol, if needed, are published on the NICE website. Further consultation on the scope would not usually be done.
After the scoping process completes, at the start of or during the assessment period, the assessment may need to be paused. This may be because of external factors such as ongoing studies that will generate relevant evidence. NICE decides whether to pause the assessment period. Registered stakeholders are informed if the assessment period is paused.
An assessment report is generated to support guidance development. This report can either be produced by NICE or an EAG (see section 1.4.5). When produced by an EAG, this is an external assessment report, and the EAG is responsible for the content and quality of the report.
The length of the assessment period will be based on the expected amount or complexity of evidence and the extent of any economic evaluation needed. If this is more extensive than expected, the assessment period may need to be extended and the scope may be updated (see section 1.3.30).
Information provided during the assessment period that is not in response to a request for information from NICE or agreed in advance with NICE, may not be able to be considered in the assessment report.
EAGs can be commissioned to produce an external assessment report to support guidance production (see section 1.3.28 in NICE health technology evaluations: the manual for further description of EAGs). They can be used when there is a larger volume or complexity of evidence, or if more complex statistical analysis or an economic evaluation is needed.
Experts selected by NICE may also support the EAG during the evaluation. But they cannot be appointed as advisers to the EAG. This is so they can maintain sufficient independence from the evidence and contribute to a committee's discussions on the quality of the external assessment report.
NICE can share a copy of the external assessment report with companies that have a named technology in the assessment (that is, the technology name is specified in the assessment scope as an intervention) for comment in advance of committee meetings. The focus of comments should be issues of factual accuracy in the assessment report, and model if produced.
If an economic model is produced as part of the assessment, NICE offers to send the economic model (in its executable form) to stakeholders with the external assessment report. If the model contains confidential material that the data owner is unwilling to share with stakeholders, despite the assurances provided through the signed confidentiality agreements, NICE will ask the group who have generated the model to replace this with dummy data or redact it if this can be done without severely limiting the model's function. Stakeholders must make requests for a copy of the model in writing. NICE provides the model on the basis that the stakeholder agrees, in writing, to the conditions of use set out in section 5.5.18 of NICE health technology evaluations: the manual.
In exceptional circumstances it may not be possible to provide the economic model to stakeholders. For example, if it is not possible to do so without revealing confidential information.
Stakeholders have 14 calendar days to submit comments.
If comments need an EAG response, NICE sends them to the EAG. Its responses will be tabled at the next committee discussion.
The developing recommendations phase of the process has 4 possible stages:
consideration of the evidence at a committee meeting to discuss the content of the draft guidance
development of, and consultation on, the draft guidance
review of the draft guidance after comments from consultation
development of the final draft guidance.
The committee is described in sections 1.2.1 to 1.2.6 of NICE health technology evaluations: the manual.
Detail on committee meetings are in sections 5.8.4 to 5.8.21 of NICE health technology evaluations: the manual. Agendas and minutes of committee meetings are published on NICE's website.
Detail on the participation of company representatives at the committee meeting are in sections 1.3.6 and 1.3.7 of NICE health technology evaluations: the manual.
The draft guidance and committee papers are sent to stakeholders for consultation. These documents are confidential until NICE publishes them on its website. Information designated as confidential will be redacted from the documents.
The committee papers and the draft guidance document are made available during consultation on draft guidance. Section 5.8.45 of NICE health technology evaluations: the manual describes draft guidance, and section 5.8.48 describes the purpose of consultation.
Stakeholders have 21 calendar days from the date of sending to submit comments on the draft guidance.
NICE publishes the draft guidance and any additional committee papers not already shared on its website with an electronic comment facility within 7 days of circulation to stakeholders. The deadline for comments to be received is the same as for stakeholders.
The committee may be unable to develop recommendations without further scrutiny or further analyses. If this is the case, the evaluation can be paused. NICE may request that a company or EAG submits specific information, further analyses or an updated economic model.
The committee chair will review the consultation comments received. When the comments will not change the recommendations, the chair can decide that another committee meeting is not needed. Factual changes and corrections to the guidance are made and final draft guidance and recommendations are agreed by the committee electronically.
The chair's decision will be shared with stakeholders. This will be a brief statement of the decision.
If needed the committee can meet again to consider the preliminary recommendations in the draft guidance with comments received. Before the meeting, NICE sends the committee members the full text of the comments from stakeholders. Sections 5.8.57 and 5.8.58 of NICE health technology evaluations: the manual describe the process of a further committee meeting.
When consultation comments are received that lead to a substantial revision of the committee's previous decision, involving a significant change in the recommendations, discussions or the evidence base, NICE and the committee chair will decide whether it is necessary to have a further draft guidance consultation. The decision to hold another consultation will extend the timelines for the evaluation. NICE will distribute any further committee papers with the second draft guidance, together with initial consultation comments. The process of a further consultation is the same as for the initial consultation.
Sections 5.8.64 to 5.8.69 of NICE health technology evaluations: the manual describe the process of developing final draft guidance.
For comments received on the draft guidance, NICE reserves the right to summarise and edit comments received during consultations. It can also decide to not publish them at all when, in the reasonable opinion of NICE, the comments are voluminous, or publication would be unlawful or otherwise inappropriate.
Section 7.2 of NICE health technology evaluations: the manual describes the resolution process. Definitions for 'stakeholders' and 'consultees', as mentioned in this section, can be found in sections 1.2.17 and 1.2.18 of NICE health technology evaluations: the manual.
Section 5.12 of NICE health technology evaluations: the manual describes the production of tools and resources to support guidance. This includes resource impact tools or statements. NICE's assessing resource impact process manual has further details.
During guidance development for interventional procedures, appropriate OPCS codes for the procedure are identified and reviewed by the committee. These codes are published with guidance on the NICE website. NICE liaises with relevant partners to identify when a new code is needed for a procedure because no appropriate codes currently exist. New codes are published on the NICE website when they become available.
For interventional procedures guidance, an audit tool template for procedures is available on the NICE interventional procedures guidance webpage.
For technologies recommended for use while further evidence is generated in early use HealthTech guidance, an evidence generation plan can be produced (see section 1.7).
After guidance publication, the process of guidance surveillance is described in section 8 of NICE health technology evaluations: the manual.
The evidence generation process is designed to help companies work with NHS sites, data custodians and analytical partners to generate evidence needed to support future NICE guidance. This process will start from the point of the decision to take a topic through early use assessment and will support the development of guidance.
The evidence generation process will aim to deliver proportionate and pragmatic approaches to evidence generation. The evidence generated during the period of use in the NHS should provide the information needed for NICE to make a recommendation about routine use in the future.
NICE:
identifies uncertainties that are essential to resolve for future decision making and that should be prioritised for further evidence generation
assesses the feasibility of evidence generation while the technologies are used in the NHS
engages with stakeholders about ongoing or planned studies and considers if and how they could address the uncertainties
highlights NHS real-world data sources that could support or contribute to evidence generation
suggests an approach to evidence generation that could address the uncertainties
highlights potential sources of funding when NICE is aware of these
highlights potential partners that could support evidence generation, such as research groups, clinical networks or implementation specialists when NICE is aware of these
monitors progress of evidence generation.
Companies:
are responsible for delivering evidence generation
are responsible for organising funding to support evidence generation
engage with and support the NICE evaluation and monitoring process
engage with partners to implement evidence generation, by:
choosing appropriate NHS sites to develop the evidence
using robust approaches to evidence generation, considering aspects such as data quality, study design, analysis, and reporting and partnering with experts in research and analysis when necessary to ensure key uncertainties are addressed
ensuring new evidence is generated in accordance with all applicable data protection legislation
ensure that safety is monitored, and signals of concern are discussed with clinical leads and reported to the Medicines and Healthcare products Regulatory Agency and NICE as appropriate
lead response to safety signals as necessary
minimise burden of data collection whenever possible, for example, by using real-world data collections that build on existing clinical information flows
consider advice laid out in NICE's health technology evaluation manual, real-world evidence framework and evidence standards framework for digital health technologies to inform evidence generation
make the evidence generated available to NICE in a form that can be used for decision making.
The feasibility assessment considers barriers and facilitators to addressing the likely uncertainties during a standard evidence generation period. It will be finalised shortly before the first committee meeting and will use information from the EAG report as well as information already gathered from topic selection and scoping stages.
The feasibility assessment considers the following aspects:
if key uncertainties could be resolved in a fixed period of 3 years from the point of guidance publication (4 years will be allowed in exceptional circumstances)
the likely number and complexity of new studies needed
facilitators that increase the likelihood that evidence generation will be successful (based on knowledge of relevant data sources, previously completed research, or known funding opportunities).
The feasibility assessment will be informed by:
uncertainties highlighted in the EAG report
consideration of the evidence landscape, including:
ongoing or planned studies
real-world data sources
consideration of methodological approaches to address the evidence gaps, for example those outlined in NICE's real-world evidence framework
knowledge of existing funding sources
knowledge of potentially suitable implementation partners.
Key conclusions from the feasibility assessment can be presented to the committee.
The committee will identify the uncertainties that need to be addressed to support future NICE guidance of a technology. This will inform the development of an evidence generation plan. The plan describes the uncertainties and what evidence should be generated for a NICE evaluation of the technologies again in the future. It is not a study protocol but suggests an approach to generating the information needed to address the evidence gaps. The evidence generation plan will sit alongside the guidance.
Once guidance is published, NICE will monitor the companies evidence generation activities. The monitoring process is designed to support companies to deliver the evidence that NICE needs and to support NICE planning for a future evaluation.
NICE has the right to withdraw or change individual technology recommendations at any stage. Information collected through the monitoring process will inform decision making about withdrawal. Reasons a recommendation to use a technology while further evidence is generated may be withdrawn include:
the technology is not available to the NHS
NICE is unable to contact the company
the company volunteers to withdraw
there are significant safety concerns about the technology
the company is not engaging in evidence generation, or evidence generation will not address the essential uncertainties.
The monitoring period will begin at the date of publication of the guidance and evidence generation plan. The monitoring process includes several touchpoints:
Six months after guidance publication, NICE will contact companies to confirm they are engaging with NICE processes and have begun evidence generation.
Twelve months after guidance publication NICE can ask for a summary of overall progress with evidence generation and the status of data collection. Ideally, companies will share their study protocol and, when relevant, evidence of engagement with implementation partners.
Annually from 12 months, companies will be expected to report on their data collection. At this point they can also be asked if they consider that the evidence generated is sufficient to address the essential uncertainties.
In addition to routine monitoring, companies should inform NICE as soon as possible of anything that may significantly affect ongoing evidence generation, including:
any substantial risk that the evidence will not be collected as planned
any safety concerns
the technology significantly changing in a way that affects the evidence generation process.
If data collection is expected to end later than planned, the company should contact NICE.
Technologies recommended for use while further evidence is generated that complete the evidence generation process will be re-evaluated by NICE. This is to decide whether the technology can be recommended for routine use, considering the further evidence generated. Details on the process of scoping in this scenario are described in section 1.3.24.
As part of monitoring done during the evidence generation process, companies can submit evidence at the touchpoints if they consider the evidence generated is sufficient to address the essential uncertainties identified in the guidance and evidence generation plan (see section 1.7.12). NICE may consider evidence provided before the end of the evidence generation period. This will follow the surveillance review process set out in sections 8.3 and 8.4 of NICE health technology evaluations: the manual. It will consider if re-evaluation of some or all of the technologies in the guidance should start before the end of the evidence generation period. When doing the surveillance review, NICE will consider the status of evidence generation for other technologies recommended for use with evidence generation in the same guidance and how close the end of the evidence generation period is. This may lead to the surveillance review being deferred to a later date to consider evidence generated by other companies, or not being done if it is likely completion would be close to or after the end of the evidence generation period. It is expected that most re-evaluations will take place after the full evidence generation period.
How are you taking part in this consultation?
You will not be able to change how you comment later.
You must be signed in to answer questions