Process and methods
Summary of a study, which may be published alone or as an introduction to a full scientific paper.
An international collaboration of researchers and policy makers whose aim is to improve the quality and effectiveness of practice guidelines. The AGREE II instrument, developed by the group, is designed to assess the quality of guidelines.
The process by which study participants are allocated to a treatment group.
How well an observation or the results of a study or review are likely to hold true in a particular setting.
Statistical relationship between 2 or more events, characteristics or other variables. The relationship may or may not be causal.
Clear record of actions so that the reasons for the actions are apparent to a third party. For example, the reasons for changes to a draft guideline should be clearly recorded.
A set of measurements before any intervention starts (after any initial 'run‑in' period with no intervention), with which subsequent results are compared.
Systematic (as opposed to random) deviation of the results of a study from the 'true' results, caused by the way the study is designed or conducted.
An observational study to find out the possible cause(s) of a disease or condition. This is done by comparing a group of patients who have the disease or condition (cases) with a group of people who do not have it (controls) but who are otherwise as similar as possible (in characteristics thought to be unrelated to the causes of the disease or condition). This means the researcher can look for aspects of their lives that differ to see if they may have caused the condition.
For example, a group of people with lung cancer might be compared with a group of people the same age who do not have lung cancer. The researcher could compare how long both groups had been exposed to tobacco smoke. Such studies are retrospective because they look back in time from the outcome to the possible causes of a disease or condition.
Citation searching (also known as 'snowballing') can help to identify additional research. It has 2 dimensions:
Backward citation searching is reviewing references cited in studies identified for inclusion in the review.
Forward citation searching involves searching for additional studies that cite articles known to be relevant (such as those identified for inclusion in the review).
An observational study with 2 or more groups (cohorts) of people with similar characteristics. One group has a treatment, is exposed to a risk factor or has a particular symptom and the other group does not. The study follows their progress over time and records what happens.
The advisory group that considers the evidence and develops the recommendations, taking into account the views of stakeholders. NICE has standing committees (which work on multiple guidelines) and topic-specific committees (which are put together for a single guideline topic). Members include practitioners and professionals (both specialists and generalists, and/or academics), care providers and commissioners, people using health and care services and/or their family members or carers, or people from communities affected by the guideline.
A member of the committee who leads committee meetings, and ensures that the committee keeps to the scope of the guideline, works collaboratively and adheres to NICE's equality policy and principles on social value judgements. The chair completes the equality impact assessment with the developer at scoping and final guideline stages, approves the draft guideline for consultation, and advises the developer on responses to comments from registered stakeholders.
The standard (for example, another intervention or usual care) against which an intervention is compared in a study. The comparator can be no intervention (for example, best supportive care).
A theoretical structure of assumptions, principles and rules, which holds together the ideas comprising a broad concept. A conceptual model has been defined as the abstraction and representation of complex phenomena of interest in some readily expressible form, such that the individual stakeholders' understanding of the parts of the system, and/or the mathematical representation of that system, can be shared, questioned, tested and ultimately agreed.
The confidence interval is a way of expressing how certain we are about the findings from a study, using statistics. It gives a range of results that is likely to include the 'true' value for the population. A wide confidence interval indicates a lack of certainty about the true effect of the test or treatment – often because a small group of patients has been studied. A narrow confidence interval indicates a more precise estimate (for example, if a large number of patients have been studied).
A table of all the comments received by NICE during consultation on a scope or draft guideline. The committee considers the comments received, and the developer then responds to the comments in the table.
Organisations contracted to do some aspects of guideline development for NICE. This might include doing evidence reviews or fieldwork, or the developer role.
An expert invited to 1 or more meetings to contribute to formulating recommendations in a specific part of the guideline. They take part fully in discussions, but do not have voting rights or count towards quorum. Co‑opted members can include people with expertise in user, carer or community experience and views, as well as those with professional or practitioner expertise.
The core members of a standing committee include at least 1 practitioner and 1 lay member, and may include an economist. A standing committee usually has between 6 and 12 core members. They serve for an initial period of up to 3 years and work on all guidelines developed by the committee during that period.
Correlates reviews describe relationships between epidemiological factors and outcomes.
This is a type of economic evaluation in which the costs and benefits are measured using the same monetary units (for example, pounds sterling) to see whether the benefits exceed the costs.
This is a type of economic evaluation in which the costs (such as treatment and hospital care) and the consequences (such as health outcomes) of a test or treatment are compared with those for a suitable alternative. Unlike cost–benefit analysis or cost-effectiveness analysis, it does not attempt to summarise outcomes in a single measure (such as the quality-adjusted life year) or in financial terms. Instead, outcomes are shown in their natural units (some of which may be monetary) and it is left to decision-makers to determine whether, overall, the treatment is worth carrying out.
This is a type of economic evaluation in which the benefits are expressed in non-monetary terms related to health, such as symptom-free days, heart attacks avoided, deaths avoided or life years gained (that is, the number of years by which the intervention extends life). Cost-effectiveness analysis assesses the cost of achieving the same benefit by different means. Cost-effectiveness analysis is also used as an umbrella term to cover all types of economic evaluation.
In a cost-minimisation analysis, the costs of different interventions that provide the same benefits are compared. If they are equally effective, only the costs are compared because the cheapest intervention will provide the best value for money. In practice, there are relatively few cost-minimisation analyses because it is rare for 2 healthcare interventions to provide exactly the same benefits.
This is a type of economic evaluation in which the benefits are assessed in terms of both quality and duration of life, and expressed as quality-adjusted life years (QALYs).
An observational study in which a population is examined to see what proportion has a particular outcome or has been exposed to a specific risk factor, or both. Cross-sectional surveys are usually used to determine the prevalence of outcomes or exposures to risk factors in populations. This type of survey may also be called a cross-sectional study or a prevalence study. Although cross-sectional surveys often provide useful estimates of disease burden for a particular population, they are less reliable for determining the prevalence of very rare conditions or conditions of short duration. Because cross-sectional surveys are descriptive rather than analytical, they cannot be used to estimate the relationship between cause and effect.
A model of how decisions are or should be made. This could be one of several models or techniques used to help people to make better decisions (for example, when considering the trade‑off between costs, benefits and harms of diagnostic tests or interventions). See also Markov modelling.
A technique used for reaching agreement on a particular issue, without the participants meeting or interacting directly. It involves sending participants a series of questionnaires asking their views. After completing each questionnaire, participants are asked to give further views in the light of the group feedback until the group reaches a predetermined level of agreement. The judgements of the participants may be analysed statistically.
This is an explicit simplification and abstraction of the problem-oriented conceptual model, mediated by what is feasible and by the availability of evidence and data.
The team responsible for scoping the guideline, identifying and reviewing the evidence, undertaking economic analyses, supporting the committee and writing the guideline in light of the committee's discussions and decisions. The team includes administrators, coordinators and project managers who provide administrative and management support to the committee, plan and schedule the work, arrange meetings, and liaise with stakeholders, and all other people and organisations contributing to guideline development.
Costs and perhaps benefits incurred today have a higher value than costs and benefits occurring in the future. Discounting health benefits reflects individual preference for benefits to be experienced in the present rather than the future. Discounting costs reflects individual preference for costs to be experienced in the future rather than the present.
The amount of a medicine to be taken, including the size and timing of the doses.
The comparative analysis of alternative courses of action in terms of both their costs and consequences.
A person with skills in economic analysis whose role is to advise on economic aspects of the key issues or questions, review economic literature, prioritise topics for further analysis and carry out additional cost-effectiveness analyses.
The observed association between interventions and outcomes, or a statistic to summarise the strength of the observed association.
The extent to which an intervention produces an overall benefit under usual or everyday conditions. In this manual effectiveness includes cost effectiveness unless otherwise indicated.
The NICE endorsement programme formally endorses resources produced by external organisations that support the implementation of NICE guidance and the use of quality standards in part or in full.
Epidemiological reviews describe a problem in terms of its causes, distribution, control and prevention, and can be used to help focus the review questions. For example, an epidemiological review of accidents would provide information on the most common accidents, morbidity and mortality statistics, and data on inequalities in the impact of accidents.
Fair distribution of resources or benefits.
Information on which a decision or recommendation is based. Evidence can be obtained from a wide range of sources, including randomised controlled trials, observational studies and expert opinion (of practitioners, people using services, family members and carers).
Information on context and current practice, which could be in the form of published implementation trials, audit data, manuscripts of case studies or service evaluation reports, or accounts from experts.
Identifying and reviewing the evidence, and undertaking economic analyses:
The information specialist identifies relevant literature to answer the review questions, creates databases to manage the search results and keeps a log of search results and strategies.
The systematic reviewer critically appraises the evidence, distils it into tables and writes brief summaries (including GRADE tables, GRADE-CERQual or evidence statements, if used). The reviewer also summarises the main issues for the committee and contributes to its discussions.
The economist identifies potential economic issues to be considered in the guideline and performs economic analyses.
Update of a guideline carried out sooner than originally planned because new data have become available.
Explicit criteria used to decide which studies should be excluded from consideration as potential sources of evidence.
Criteria that define who is not eligible to participate in a study.
The Expert Advisers Panel provides a single repository of experts and practitioners (who have been through a robust recruitment process) for the NICE Centre for Guidelines, and can be called upon to contribute to various guideline development activities, such surveillance reviews and guideline updates.
An expert invited to attend a committee meeting to provide evidence from their experience and specific expertise. Expert witnesses answer questions from committee members and may be invited to present evidence in the form of expert testimony, which is published on the NICE website when the guideline is published. Expert witnesses are not members of the committee. They have expert knowledge of 1 or more of the following areas: experience and views of practitioners; people using services; carers or the community and voluntary sector; government and policy; or research and practice.
The degree to which the results of a study hold true in non-study situations (for example, in routine NHS practice). It may also be referred to as the generalisability of study results to non-study populations. For example, the external validity of a study that took place in Spain may be questioned if the results are applied to people in Australia.
In data analysis, predicting the value of a parameter outside the range of observed values.
Observation over a period of time of a person, group or defined population to observe changes in health status or health- and social care-related variables.
A type of graph used to display the results of a meta-analysis.
Formal consensus methods are techniques that can be used to enable a committee to reach an agreement on a particular issue. Methods include Delphi and nominal-group techniques, and consensus development conferences. These methods may be used during guideline development when there is a lack of strong research evidence in a particular area.
Terms used for searching that are not controlled vocabulary as used in the database or information source, but standard terms used in natural language.
Full update of a guideline
When a guideline is identified for a full update, the existing guideline with its recommendations, are stood down and a replacement guideline is developed with new recommendations.
The extent to which the results of a study based on measurements in a particular population or a specific context hold true for another population or in a different context.
A systematic and explicit approach to grading the quality of evidence and the strength of recommendations. GRADE is an evolving system and is continuously being adapted and extended to cover different areas and types of evidence; for example, CERQUAL for qualitative evidence and GRADE for diagnostic studies. See the GRADE working group for the latest news and publications.
A table summarising, for each important outcome, the quality of the evidence and the outcome data (used as part of the GRADE approach to assessing the quality of the evidence).
Literature that is not formally published or that has a limited distribution, such as institutional reports. Grey literature may not be easily identified through standard bibliographic retrieval systems.
The gap in health status and in access to health services between different groups, for example, those with different socioeconomic status or different ethnicity, or populations in different geographical areas. More information on health inequalities can be found on the Department of Health and Social Care website.
A combination of a person's overall physical, mental and social wellbeing; not merely the absence of disease.
Independent research about the effectiveness, costs and broader impact of healthcare (treatments and tests) for those who plan, provide or receive care in the NHS. The Health Technology Assessment (HTA) programme is part of the National Institute for Health Research (NIHR).
The process of putting guideline recommendations into practice.
Information (for example, the findings of a research project) defined as 'confidential' because its public disclosure could affect the commercial interests of a particular company ('commercial in confidence') or the academic interests of a research or professional organisation ('academic in confidence').
Explicit criteria used to decide which studies should be considered as potential sources of evidence.
The difference in the mean costs between 2 interventions, strategies or programmes in the population of interest divided by the differences in the mean outcomes between the 2 interventions, strategies or programmes in the population of interest.
The test in a study which is being compared with the best available test (the reference standard).
The defined use of a medicine as licensed by the Medicines and Healthcare products Regulatory Agency (MHRA).
An analysis to compare interventions that have not been compared directly in a head-to-head trial.
A measure of how well a research study has been designed and how well it avoids bias. That is, the extent to which the cause-and-effect relationships in a study are true for the people and conditions of the study.
Key issues are included in the scope of a guideline and broadly define aspects of care or service provision for which most advice is needed.
Key questions are included in the scope of a guideline and are broad questions related to the areas defined by the key issues. Key questions relate to the effectiveness and cost effectiveness of interventions that are being considered for a given population. Key questions are then used to develop more detailed review questions.
A member of the committee who has personal experience of using health or care services, or who is from a community affected by the guideline. A lay member can also be someone with experience as a carer, an advocate, or a member or officer of a voluntary or community organisation.
A summary of the evidence from several studies, with conclusions about the findings. It may or may not be systematically researched and developed.
A model that incorporates the assumed relationships between action and outcomes as described in the conceptual framework.
This was previously known as a product licence. Marketing authorisation is granted to medicines that meet the standards of safety, quality and efficacy set by a medicines regulator (for example, the Medicines and Healthcare products Regulatory Agency [MHRA] or the European Medicines Agency). It is normally necessary before a medicine can be prescribed or sold.
A decision-analytic technique that predicts future events occurring in a group over a period of time by assigning group members to a fixed number of health states and then modelling transitions among the health states.
All products, except medicines, used in healthcare for the diagnosis, prevention, monitoring or treatment of illness or disability.
The Executive Agency of the Department of Health and Social Care that is responsible for protecting and promoting public health and patient safety by ensuring that medicines, healthcare products and medical equipment meet appropriate standards of safety, quality, performance and effectiveness, and are used safely.
A method often used in systematic reviews to combine results from several studies of the same test, treatment or other intervention to estimate the overall effect of the treatment.
A process for sorting and combining the findings from qualitative studies.
Information needed for economic modelling. This may include information about effectiveness, adverse events, diagnostic accuracy, prognosis, quality of life, resource use and costs.
Summary of findings presented as a written description rather than, for example, as a graph or table.
In cost‑effectiveness and cost–utility analysis, the net benefit estimate can be expressed in outcomes (for example, using quality-adjusted life years [QALYs]) or monetary terms. The net health (or outcome) benefit is the difference between the total expected QALYs (or outcome) and the health (or outcomes) expected to be forgone elsewhere (the total expected costs divided by the maximum acceptable incremental cost-effectiveness ratio [ICER] value). The net monetary benefit is the difference between the monetary value of total expected QALYs (our outcome) multiplied by the maximum acceptable ICER value [ICER] and total expected costs. In cost–benefit analysis, the net benefit estimate is the estimate of the amount of money remaining after all payments made are subtracted from all payments received. This is used in the economic evidence profile for guidelines.
An analysis that compares 3 or more interventions using a combination of direct evidence (from studies that directly compare the interventions of interest) and indirect evidence (from studies that do not compare the interventions of interest directly).
Recommendations produced by NICE. There are 5 types of guidance:
guidelines covering clinical topics, medicines practice, public health and social care
interventional procedures guidance
technology appraisals guidance
medical technologies guidance
All guidance is developed by independent committees and is consulted on.
Recommendations (and the evidence they are based on) on broad topics covering health, public health and social care in England. NICE guidelines include clinical, medicines practice, public health and social care guidelines.
NICE Pathways are a tool to help find NICE guidance quickly and easily – everything NICE says on a topic in an interactive flowchart.
These are trials in which participants (or groups) are allocated to receive either the intervention or a control (or comparison intervention) but the allocation is not randomised. This type of study is often called a controlled before-and-after (CBA) study.
Retrospective or prospective study in which the investigator observes the natural course of events with or without control groups (for example, cohort studies and case–control studies).
An odds ratio compares the odds of something happening in one group with the odds of it happening in another. An odds ratio of 1 shows that the odds of the event happening (for example, a person developing a disease or a treatment working) is the same for both groups. An odds ratio of greater than 1 means that the event is more likely in the first group than the second. An odds ratio of less than 1 means that the event is less likely in the first group than in the second group.
Sometimes probability can be compared across more than 2 groups – in this case, one of the groups is chosen as the 'reference category', and the odds ratio is calculated for each group compared with the reference category.
The p value is a statistical measure that is used to indicate whether or not an effect is statistically significant.
Anyone who is using health or care services, or a member of the public affected by a guideline.
Care services for vulnerable people, including those with special needs because of old age or physical disability and children in need of care and protection. Examples are residential care homes for older people, home help and home care services, and social workers who provide help and support for a wide range of people (Department of Health and Social Care definition).
A structured approach for developing review questions about interventions. The PICO framework divides each question into 4 components: the population (the population being studied), the interventions (what is being done), the comparators (other main treatment options) and the outcomes (measures of how effective the interventions are).
In NICE quality standards, placeholder statements are used for areas of care in need of quality improvement but for which there is no evidence-based guidance available to formulate quality statements or measures.
A healthcare, social care or public health worker.
A study comparing health interventions among a randomised, diverse population representing clinical practice, and measuring a broad range of health outcomes. To ensure generalisability, pragmatic trials should represent the intended patients to whom the treatment will be applied as best as possible. (Definition from GetReal glossary)
Problem-oriented conceptual model
This is a simplified, diagrammatic representation of the framework that describes the resources, processes and interactions in the delivery of interventions.
A probable course or outcome of a disease. Prognostic factors are characteristics of a patient or disease that influence the disease course. A good prognosis is associated with a low rate of undesirable outcomes; a poor prognosis is associated with a high rate of undesirable outcomes.
The staff member who oversees and facilitates the guideline development process.
The brand name a manufacturer gives to a medicine or device it produces.
A tool for assessing the quality of studies of the accuracy of diagnostic tests.
Qualitative research explores people's beliefs, experiences, attitudes, behaviour and interactions. It asks questions about how and why, rather than how much. It generates non-numerical data, such as a person's description of their pain rather than a measure of pain. Qualitative research techniques include focus groups and in‑depth interviews.
A measure of the state of health of a person or group in which the benefits, in terms of length of life, are adjusted to reflect the quality of life. One QALY is equal to 1 year of life in perfect health.
NICE staff carry out quality assurance of the guideline, including reviews of the evidence and any economic analysis, to ensure that it is up‑to‑date, credible, robust and relevant. These staff may also be responsible for commissioning the developer.
The centre director is responsible for ensuring that the guideline is produced in accordance with this manual. The centre director is also responsible for appointing the committee chair and committee members.
The guideline lead is responsible for the development and quality assurance of the guideline (including the scope), and has delegated responsibility for approving the consultation draft, the final guideline, and other documents, before final approval by NICE's Guidance Executive. The guideline lead also advises the chair of the committee and the developer on matters of method and process. Guideline commissioning managers help them with this.
The NICE clinical, public health or social care adviser is responsible for providing advice during all stages of guideline development.
The technical lead is responsible for the technical quality assurance of the evidence reviews and other work undertaken by the developer. The technical lead commissions, coordinates and quality assures any fieldwork.
The economic lead is responsible for ensuring the technical quality of the economic evidence and any economic analysis.
Quality assurance of guideline surveillance reflects quality assurance of guideline development. The NICE associate director – surveillance is responsible for ensuring that processes are followed and that decisions to update or not update guidelines are robust and fit for approval by NICE's Guidance Executive.
Quality standards set out the priority areas for quality improvement in health and social care. They cover areas where there is variation in care. Each standard includes a set of statements to help improve quality, and information on how to measure progress.
The smallest number of group members that must be present for a valid meeting. The quorum of a committee is 50% of the total potential membership. No recommendations should be confirmed unless the quorum is reached.
Trials in which participants (or clusters) are randomly allocated to receive either intervention or control. If well implemented, randomisation should ensure that intervention and control groups differ only in their exposure to treatment.
Real world evidence
See the GetReal glossary.
Specific advice in NICE guidelines on the care and services that are suitable for most people with a specific condition or need, or for particular groups or people in particular circumstances (for example, when being discharged from hospital). Recommendations may also cover ways to promote good health or prevent ill health, or how organisations and partnerships can improve the quality of care and services.
The reference case specifies the methods considered by NICE to be the most appropriate for estimating clinical and cost effectiveness when developing guidance. These are also consistent with an NHS objective of maximising health gain from limited resources.
A method, procedure or measurement that is widely accepted as being the best available to test for or treat a disease.
Recommendations for future research that cover areas of uncertainty or gaps in the evidence identified during guideline development.
Tobacco organisations (for example, tobacco companies, those who speak for them or are funded by them) with an interest in a particular topic. The term 'respondent' acknowledges NICE's commitment to Article 5.3 of the WHO Framework Convention on Tobacco Control. This sets out an obligation to protect the development of public health policy from any vested interests of the tobacco industry.
A document that outlines the background, objectives and planned methods for an evidence review.
Review questions guide a systematic review of the literature. They address only the key issues and questions covered in the scope of the guideline, and will usually be structured with a framework (for example, using PICO or SPICE).
A search of key sources at the scoping stage to identify previous guidelines, health technology assessment reports, key systematic reviews, randomised controlled trials and economic evaluations relevant to the guideline topic. The search also includes the NICE website, government, charity, and other community and voluntary sector websites to identify relevant policies and documents.
The scoping workshop is attended by registered stakeholders and is held when key issues that need discussion have been identified by the developer. The workshop may be held before during or after consultation.
A collection of search terms designed to retrieve certain types of study (for example, those using a specific study design or on a specific topic).
This refers to how well a test detects what it is testing for. It is the proportion of people with the disease or condition that are correctly identified by the study test.
A means of exploring uncertainty in the results of economic evaluations. There may be uncertainty because data are missing, estimates are imprecise or there is controversy about methodology. Sensitivity analysis can also be used to see how applicable results are to other settings. The analysis is repeated using different assumptions to examine the effect of these assumptions on the results.
Deterministic sensitivity analysis investigates how bias in selecting data sources for key model parameters might affect the results.
One‑way sensitivity analysis (univariate analysis) varies each parameter individually to investigate how this affects the results.
Probabilistic sensitivity analysis assigns probability distributions to uncertain parameters and incorporates these into models using decision-analytic techniques (for example, Monte Carlo simulation).
These show how NICE guidance and standards have been put into practice by a range of health, local government and social care organisations.
Social care generally refers to all forms of personal care and other practical assistance for children, young people and adults who need extra support. This includes:
children and young people who are at risk of, or who are already experiencing, social and emotional problems
children, young people and adults with learning or physical disabilities or mental health problems
people who misuse drugs or alcohol
The decisions in NICE guidance are based on the best available evidence. Sometimes the available evidence is not of good quality or can be incomplete, so the committees involved have to make scientific value judgements and social value judgements. Social value judgements take account of society's expectations, preferences, culture and ethical principles when making recommendations.
This refers to how well a test detects what it is testing for. The proportion of people classified as negative by the reference standard who are correctly identified by the study test.
A structured approach for developing review questions that divides each question into 5 components: setting, perspective, intervention, comparison and evaluation (SPICE).
Stakeholders are organisations with an interest in a particular guideline topic; they may represent people whose practice or care is directly affected by the guideline.
They include: national organisations for people who use health and social care services, their families and carers, and the public; local Healthwatch organisations; national organisations that represent health and social care practitioners and other people whose practice may be affected by the guideline, or who can influence uptake of the guideline recommendations; public sector providers and commissioners of care or services; private, voluntary sector and other independent providers of care or services; companies that manufacture drugs, devices, equipment or adaptations, and commercial industries relevant to public health; organisations that fund or carry out research; government departments and national statutory agencies.
Stakeholders are encouraged get involved at all stages. Registered stakeholders comment on the draft scope and draft guideline, may provide evidence, and support implementation of the guideline.
See also respondent.
A committee consisting of core members who work on multiple guidelines. Topic expert members are brought in to work on specific guidelines.
A report that summarises the evidence and intelligence identified through the surveillance process and explains the reasons for updating or not.
The process of checking whether a guideline needs to be updated. This generally includes consideration of new evidence and intelligence such as topic expert feedback, changes to legislation or policy, and information on implementation.
A review that summarises the evidence on a clearly formulated review question according to a predefined protocol, using systematic and explicit methods to identify, select and appraise relevant studies, and to extract, analyse, collate and report their findings. It may or may not use statistical meta-analysis.
The time period over which the main differences between interventions in effects and the use of resources in health and social care are expected to be experienced, taking into account the limitations of the supporting evidence.
A member of the committee who also works closely with the developer to provide topic-specific support.
Experts on the topic of a guideline who join a standing committee to work on that guideline. They may include lay members, practitioners, providers and commissioners.
A committee consisting of members appointed for the development of a specific guideline.