NICE process and methods

6 Incorporating health economics

6.1 Introduction

This section is adapted from chapter 7 in The guidelines manual (NICE 2009).

Health economics is about using resources efficiently to improve the population's health. Health economic analysis and evaluation forms an integral part of the public health guidance development process. The Public Health Advisory Committee (PHAC) are required to make decisions informed by the best available evidence of both effectiveness and cost effectiveness.

Economic evaluation compares the costs and consequences of alternative courses of action. The cost effectiveness of an intervention or programme is assessed to ensure maximum health gain from the finite available resources. If resources are used for interventions that are not cost effective, the population as a whole gains fewer health benefits. However, a balance must be struck between efficient allocation of resources on the one hand and an equitable allocation of those resources on the other.

This chapter describes how the health economic evidence should be collated and analysed. It also sets out the principles for conducting health economic modelling analyses if there is insufficient evidence in the literature to reliably assess the cost effectiveness of interventions.

Public health recommendations should be based on the balance between the estimated cost of each intervention and the expected health benefits (that is, recommendations should be cost effective). Recommendations should not be made on the basis of the total cost or the resource impact of implementing them. So, if the evidence suggests that an intervention provides health benefits and the cost per person of doing so is acceptable, it should be recommended, even if it would be expensive to implement across the whole population.

Although commissioners do need to know the resource and cost implications of implementing NICE guidance, this assessment is not in the remit of the economic analysis. NICE undertakes a separate cost-impact analysis after the public health guidance is published and this forms part of the implementation tool set.

6.2 Reviewing economic evaluations

The review of economic studies should be systematic but focused. If it is known in advance that a high-quality economic study has been published that addresses a structured public health question and is relevant to current practice, then further modelling of that intervention will not be necessary.

6.2.1 Critical appraisal of economic studies

The search for economic studies should have identified papers that are most relevant to current public sector practice in the area under consideration and hence, likely to inform the PHACs' decision-making. It should have focused on 'full' economic evaluations that compare both the costs and the health consequences of the alternative options under consideration.

The process for sifting and selecting economic studies for critical appraisal is essentially the same as for effectiveness studies (see chapters 4 and 5). The Centre for Public Health Excellence (CPHE) project team and the review team should discuss and agree the inclusion criteria. These should include the populations and interventions that relate to the review question and should also specify:

  • An appropriate date range, because older studies may reflect outdated practices.

  • The country or setting, because studies conducted in other healthcare systems or sectors might not be relevant to the UK. In some cases, it may be appropriate to limit inclusion to UK-based or Organisation for Economic Cooperation and Development (OECD) studies.

  • The type of economic evaluation, including cost–utility, cost–benefit, cost-effectiveness, cost-minimisation and cost–consequences analyses. Non-comparative costing studies, 'burden of disease' studies and 'cost of illness' studies should usually be excluded.

6.2.2 Assessing economic evaluation studies

Estimates of resource use obtained from public health studies should be reviewed using the studies identified from the process described in section 6.2.1. Reservations about the applicability of these estimates to routine public sector practice should be noted in the evidence tables. The PHACs should take these reservations into account.

The criteria for appraising costs, cost-effectiveness ratios and net benefits are different because these estimates are usually obtained using some form of modelling. Modelling can include:

  • formal decision-analytic models

  • economic evaluations conducted alongside effectiveness trials (these usually require some additional information, for example, unit costs, health-state valuations or long-term outcome probabilities)

  • estimation procedures to predict long-term costs and outcomes.

The review team should appraise all the economic evaluation studies (both published and unpublished) used to inform public health recommendations, using the economic evaluations checklist (see appendix I). Unpublished economic evaluations may include studies submitted by stakeholders or pre-publication academic papers (see sections 4.4.4, 4.4.5 and The same criteria should be applied to any additional economic evaluations conducted for the guidance.

Given the small number of economic evaluations usually obtained through the literature searches, title and abstract screening, full-paper screening and quality appraisal should normally be undertaken by 2 reviewers. Any disagreement between the 2 reviewers, at any stage, should be resolved by discussion or recourse to a third reviewer.

The checklist includes a section on the methodological quality of the study (that is, the extent to which it fulfils its stated objectives), a section on its applicability to the review question and a section on the NICE health economics methodology used for decision-making. It can be used to judge overall methodological quality as follows:

  • Very serious limitations The study fails to meet 1 or more quality criteria and this is very likely to change the conclusions about cost effectiveness. Such studies would usually be excluded from further consideration.

  • Potentially serious limitations The study fails to meet 1 or more quality criteria and this could change the conclusions about cost effectiveness.

  • Minor limitations The study meets all quality criteria, or fails to meet 1 or more quality criteria but this is unlikely to change the conclusions about cost effectiveness.

The robustness of the study results (that is, its methodological limitations) may sometimes be apparent from reported sensitivity analyses. If not, judgement will be needed to assess whether a limitation is likely to change the results and conclusions.

The checklist can also be used to judge the overall applicability of the study in the context of the guidance:

  • Not applicable The study fails to meet 1 or more applicability criteria and this is very likely to change the conclusions about cost effectiveness. Such studies would usually be excluded from further consideration.

  • Partially applicable The study fails to meet 1 or more applicability criteria and this could change the conclusions about cost effectiveness.

  • Directly applicable The study meets all applicability criteria, or fails to meet 1 or more applicability criteria but this is unlikely to change the conclusions about cost effectiveness.

If necessary, the health technology assessment modelling checklist for decision-analytic models (Philips et al. 2004) may also be used to assess the methodological quality of modelling studies in more detail.

When assessing the evidence of effectiveness and cost effectiveness, the PHACs should consider the implications of any unexplained differences between model results.

The review team should record its judgements based on the economic evaluation checklist (and health technology assessment modelling checklist, if appropriate) in an appendix to the guidance. The checklist 'comments' column should be used to record the reasons for these judgements and, where necessary, additional details about the studies.

Where good evidence is scarce, the criteria for inclusion should be discussed (or revisited) with the CPHE project team.

6.2.3 Summarising and presenting cost-effectiveness results

The review team should present cost, cost effectiveness or net-benefit estimates from published or unpublished studies, or from economic analyses conducted for the guidance, in an economic evidence table. (See appendix K for the table template.) Where economic information is not available (or it is not thought to be relevant to the question) this should be explicitly stated. An overall assessment of any study limitations and its applicability should also be included. Issues that the PHACs should consider when assessing the economic evidence should be noted in the 'Notes' column.

The economic evaluation results should include a 'best-available' estimate or range for the incremental cost, the incremental effect and, where relevant, the incremental cost-effectiveness ratio or net-benefit estimate. A summary of the degree of uncertainty in the estimates should also be presented, reflecting the results of deterministic or probabilistic sensitivity analyses, or stochastic analyses of trial data, as appropriate.

The review team should include a short economic evidence statement at the end of the economic evidence synthesis. This should summarise key features of the evidence on cost effectiveness in a similar format to that used to summarise the effectiveness evidence (see section 5.5).

6.3 Prioritising areas for further economic analysis

Further analysis for the development of the piece of guidance under consideration will usually be required, as the health economic literature is rarely comprehensive or conclusive enough.

Additional economic analyses may involve adapting an existing model or developing a new one. For many public health interventions, it will be necessary to extrapolate effectiveness evidence over long time periods. It will also be necessary to derive long-term quality-adjusted life year (QALY) outcomes from short-term, intermediate results.

Close collaboration between the health economist contractor, the CPHE project team and the public health advisory groups is essential early in the guidance development process to ensure that:

  • the most important questions or intervention areas are selected for economic analysis

  • the overall modelling approach is appropriate

  • important health effects and resource costs are all included

  • effects and outcomes not related to health are included (if they are material for the sector whose perspective is being used, usually the public sector, local government or the NHS)

  • best available effectiveness, epidemiological and resource evidence is used

  • model assumptions are plausible

  • uncertainties are fully explored and systematically addressed

  • results are interpreted appropriately and any limitations are acknowledged.

Economic analysis is potentially useful for any question involving comparisons between interventions or programmes. This includes comparisons of prevention methods used to achieve the same health outcomes. For instance, school-based interventions to reduce under-age drinking could be compared with enforcement of current legislation restricting the sale of alcohol to under-18s. It may also be appropriate when comparing different combinations or sequences of interventions. However, given the broad scope of much public health guidance, it will not be possible to conduct original analyses for every intervention or question.

The selection of interventions or questions for further economic analysis (including modelling) should be a joint decision between the health economist, the CPHE project team and the PHACs. Selection should be based on systematic consideration of the potential value of carrying out an economic analysis across all interventions. Note that an economic analysis will be more useful if it is likely to influence a recommendation, and if the recommendation will have a significant health and financial impact. So the decision about whether or not to carry out an economic analysis depends on:

  • the expected net benefit of the recommendation (the number of individuals affected and the potential impact on costs and health outcomes per individual)

  • the degree of uncertainty in the cost effectiveness literature and the likelihood that economic analysis will clarify matters.

Economic modelling may not be warranted if:

  • it is not possible to estimate even a ball-park figure for cost effectiveness

  • the intervention is obviously cost-saving (for example, the sharing of facilities in partnership working) and the health benefits are not negative

  • the published cost effectiveness evidence is so reliable that further economic analysis would be superfluous

  • when it is obvious that the resource implications are modest in relation to the expected health gains.

6.3.1 Type of economic analysis

Because local government will (after March 2013) be largely responsible for implementing public health programmes, NICE has broadened its approach to the appraisal of public health interventions. Local government is responsible not only for the health of individuals and communities, but also for their overall welfare. It is also less centralised than the NHS. The tools of economic evaluation must reflect both of these aspects (wider remit than health and greater local element). The main change to the approach to economic evaluation is to place more emphasis on cost–consequences analysis (CCA) and cost–benefit analysis (CBA) than has been the case in previous methods manuals.

Cost-effectiveness analysis and cost–utility analysis (CUA) will still be required routinely, for several reasons:

  • CUA provides a single yardstick or 'currency' for measuring the impact of interventions on health. This form of analysis should be maintained wherever health is the sole or predominant benefit or influence.

  • CUA allows interventions in healthcare to be compared so that resources may be allocated more efficiently.

  • It should continue to be possible to compare estimates of cost effectiveness in the new institutional environment (mainly local government) with previous estimates of similar interventions made for use in the NHS.

  • All NICE programmes should include the use of a common method of cost-effectiveness analysis that allows comparisons between programmes.

  • In some circumstances, almost all benefits are health benefits. In that case, where there is a clear indication of cost effectiveness or ineffectiveness, further analysis (such as use of CCA and/or CBA) that is unlikely to change a decision about an intervention would not be required.

Performing a cost-effectiveness analysis

Nevertheless, CUA has its disadvantages. The main disadvantage is its narrowness. It measures only health benefits, and even then, does not include the benefits associated with working: a healthier person will often work longer and return-to-work sooner as a result of being healthier. It is therefore most suitable when the main or only benefit is a health benefit. It is also narrow in that it accounts only for efficiency, and not equity.

If there are not enough data to estimate QALYs gained, an alternative measure of cost effectiveness may be considered (such as life years gained, cases averted or a more disease-specific outcome).

Costs of lost production and any costs borne by patients and carers that are not reimbursed by the public sector should be included if a sufficiently wide perspective is also adopted.

A cost-effectiveness analysis could be modelled on a single well-conducted randomised controlled trial (RCT), or it could use decision-analytic techniques to analyse probability, cost and health-outcome data from a variety of published sources.

There is often a trade-off between the range of new analyses that can be conducted for the development of a given piece of guidance and the complexity of each piece of analysis. Simple methods may be used if these provide the PHACs with sufficient information on which to base a decision. For example, if an intervention is associated with better health outcomes and fewer adverse effects than its comparator and has a similar cost, then a simple decision tree may provide a sufficiently reliable estimate of cost effectiveness. In other situations a more complex approach, such as Markov modelling or discrete-event simulation, may be warranted.

In addition:

  • All health effects on individuals should be included.

  • Equity weightings should not be applied to QALYs.

  • Costs and health outcomes are discounted at 1.5% per year for both costs and benefits see section 6.3.4 for more detail). Health-related quality of life is valued using choice-based elicitation methods, a representative sample of the general population and validated, generic health-state instruments. (There is unlikely to be time to collect original quality-of-life valuations, so data collected by alternative methods may be used, but this should be suitably justified.)

  • The time horizon should be chosen so as to incorporate all important costs and effects.

In some circumstances, cost utility and cost effectiveness will not be an appropriate measure. This will normally be decided before work is commissioned, and an alternative form of analysis such as CCA, CBA or cost minimisation will be specified.

Additionally, public health has aspects that are wider than health alone, and these are more readily recognised in a local government environment. This necessitates both making the method of analysis more inclusive, and a corresponding change in perspective. CCA and CBA both fulfil this requirement.

Cost–consequences analysis (CCA)

CCA has implicitly been used at NICE since its inception. The 2 elements that have always been discussed by decision-making committees are efficiency (usually measured in terms of cost per QALY) and equity. The committees, as appropriate, also discuss adverse events, people's satisfaction with the intervention, its delivery, its ease of access and other aspects that have not been covered by efficiency and equity.

With the move to local authority responsibility, NICE's analysis will consider CCA explicitly. CCA can measure both welfare and quality of life more broadly than the health-related quality of life measure encompassed in the EuroQoL EQ-5D instrument that NICE uses to estimate the QALY. It can take many other items into account that local authorities are likely to find important, including the trade-off between long-term goals and a paucity of short-run funding, and spillover effects into other areas of local government responsibility. The extent to which these effects are material for any particular analysis will depend on circumstances. Where none of them is material, then there need be no change from the previous practice of conducting only a CUA or cost–effectiveness analysis together with a CCA to include equity considerations. In many cases, however, this will not be sufficient, and a wider and more formal form of CCA should be used.

The outcomes to be included in the CCA will depend on circumstances and should be discussed in advance with the NICE technical team. In addition, the perspective used in CCA will often be wider than that of a healthcare provider. This aspect will be covered in more detail in section 6.3.2.

The disadvantages of CCA are:

  • The outcomes are usually more difficult to interpret and to aggregate than the single CUA outcome.

  • The ranking of interventions using CCA is unlikely to be the same as that for which CUA has been used.

  • Different bodies will impose different rankings on the allocation of resources to the same set of projects evaluated in this way.

  • The method takes more time and resources to measure the range of outcomes than does CUA for its single outcome.

Cost benefit analysis (CBA)

CBA studies have been used sparingly at NICE in the past, because CUA studies have usually been adequate for interventions that involve health and healthcare alone. If cost-benefit studies occur in economic evaluation literature included in the evidence, NICE has used these studies. CBA is a common form of analysis in transport studies, capturing a wide range of health and non-health impacts such as congestion, time saving and CO2 emissions. CBA converts all benefits and costs that can be readily quantified into monetary terms. It sums the costs and benefits separately to arrive at either a net monetary benefit or a ratio of benefits to costs and consequently it usually operates with a societal perspective ('perspective' is covered in more detail below). There are advantages of using CBA, such as:

  • If it is society's interests we are interested in, then all costs and all benefits should be included. Omitting some costs or some benefits in a decision to recommend an intervention will not usually be the best decision for society, unless the omitted items are not material to the decision.

  • CBA includes benefits to individuals, such as those of a person being employed (compared with not being employed) as the result of an intervention.

  • Expressing costs and benefits in money terms avoids the difficulties of aggregating data that occur with cost–consequences analysis.

On the other hand, there are some disadvantages:

  • Some outcomes cannot readily be quantified. For example, equity and social cohesion cannot be quantified in a CBA. The inclusion of either of these concepts requires a CCA to be used.

  • If decisions are being made about what a government department (or local government) should pay for, then only the costs and benefits of interest to that sector might be required.

  • 'Willingness to pay' (WTP) for a benefit in a CBA is the sum of individually elicited WTPs that usually exceeds the WTP of the government department (or local government) that pays for the intervention out of taxation. For example, if the ratio of aggregate individual WTP to government WTP were 3 for a given intervention, then a benefit:cost ratio of at least 3, based on aggregate individual WTP, would be required before the government might be expected to fund it. Little work has been done on what value the ratio of aggregate individual WTP to government WTP should be in any particular intervention, or for an 'average' intervention, but decision-makers should be aware that it is likely to exceed unity.

  • WTP, especially the aggregation of individual WTP, as distinct from the WTP of a government department (or local government), is a measure of demand rather than of need, whereas in the NHS, healthcare is allocated according to need. This may occasionally cause contradictions with NICE's equity objectives.

  • CBA may sometimes have large data requirements over and above what would be required for a CCA, such as a survey to estimate WTP and appropriate estimates of all relevant costs.

  • Traditionally, there have been measurement issues concerning how health and non-health impacts can be valued in terms of money.

Despite this, a CBA is sometimes the most appropriate method of analysis for public health guidance.

6.3.2 Perspective

The standard perspective for the economic analysis of public health interventions is typically that of the public sector. This allows the costs and benefits of more than 1 central government department or body, and local government, to be taken into account.

In addition to the public sector perspective, the perspective of the department that administers the interventions should be used. For example, if local government is involved almost exclusively, or is the body administering the intervention, then a local government perspective should be used as well as the public sector perspective. If the NHS is involved almost exclusively, or is the body administering the intervention, then an NHS perspective should be used as well as the public sector perspective.

When appropriate, and in agreement with the CPHE project team, results may also be presented from other perspectives. For example, an employer's perspective would be taken to demonstrate the business case for a public health intervention on, say, workplace smoking. The transfer of many public health functions to local government means that a local government perspective is likely to be the most common for public health programmes.

Whenever CBA (with WTP estimated by aggregating individually elicited WTP) has been used, the appropriate perspective is a societal one.

A modified CBA (with WTP being estimated as a government-sector WTP out of its budget) would use a public sector perspective. However, this is only likely if the government sector involved is the health sector, because that is the only sector with a reasonably well-defined government-sector WTP in the form of the threshold incremental cost-effectiveness ratio (ICER) of £20,000 to £30,000 per QALY gained.

This set of perspectives differs from that used for NICE technology appraisals and clinical guidelines: these only consider the NHS and personal social services (PSS). The broadening of the perspective set for public health guidance reflects the wider remit of public health and the changes in the way that it will be delivered after March 2013.

Table 6.1 Summary of the public health reference case[8]

Element of assessment

Reference case

Defining the decision problem

The scope developed by NICE


Interventions routinely used in the public sector, including those regarded as best practice

Perspective on costs

Public sector, including the NHS and PSS, or local government

Societal perspective (where appropriate)

Perspective on outcomes

All health effects on individuals. For local government guidance, non-health benefits may also be included

Type of economic evaluation



CUA – to ensure comparability with other parts of NICE

Synthesis of evidence on outcomes

Based on a systematic review

Measure of health effects


Measure of non-health benefits

Where appropriate, to be decided on a case-by-case basis in conjunction with the CPHE technical team

Source of data for measurement of health-related quality of life (HRQL)

Reported directly by patients or carers

Source of preference data for valuation of changes in HRQL

Representative sample of the public

Discount rate

An annual rate of 1.5% on both costs and health effects (sensitivity analyses should include discount rates used by other parts of NICE)

Equity weighting

An additional QALY has the same weight, regardless of the characteristics of the individuals who gain the health benefit

6.3.3 More than one department (or local government) is involved

No standard method has been devised of apportioning costs and who should bear them when more than 1 government department (or local government) is involved in the delivery of an intervention. The problem is greatest when 1 government department (or local government) reaps the benefits of an intervention but another department is required to implement it: for example, a school being asked to find time in its timetable to deliver a health intervention. No method is suggested in this guide for dealing with such problems.

6.3.4 Discounting

There is currently some debate about the most appropriate rate of discounting future costs and benefits.

Since public health interventions usually act over the long term, it has been decided to use a discount rate of 1.5% for all costs and benefits, so as to be in line with the option used in the Guide to the methods of technology appraisal for interventions which have effects lasting for many years. In practical terms, for many interventions in public health where the intervention occurs over a relatively short period earlier in people's lives but the benefits accrue mostly at the end of their lives, the use of this set of discount rates will give approximately the same result as a 3.5% discount rate on costs and a 1.5% rate on benefits used by the Department of Health's (DH), in line with their interpretation of the Treasury Green Book.

In sensitivity analysis, an annual discount rate of 3.5% on both costs and benefits will also be routinely used in sensitivity analysis, to allow comparisons with previous public health and Centre for Health Technology Evaluation guidance and with current Centre for Health Technology Evaluation and Centre for Clinical Practice guidance where this is based on 3.5% for costs and benefits.

6.3.5 Other aspects of modelling

The general approach to modelling to be taken must be agreed with the CPHE project team prior to implementation. Where available, consideration should be given to linking the model structure to the logic model used to develop the research questions. The logic model illustrates the causal pathways between human behaviour, the social, environmental and biological determinants of health and potential interventions and outcomes (see chapter 2 and appendix A for details).

Sensitivity analysis should be used to explore the impact that potential sources of bias and uncertainty could have on model results.

Deterministic sensitivity analysis should be used to explore key structural assumptions: testing whether and how the model results change under alternative, plausible scenarios. It should also be used to test any bias resulting from the data sources selected for key model parameters.

Probabilistic sensitivity analysis should be used to explore the uncertainty arising from imprecision in model parameters. Any uncertainty associated with all parameters can be simultaneously reflected in the results. In non-linear decision models, probabilistic methods also provide the best estimates of mean costs and outcomes. However, models incorporating probabilistic methods are more time-consuming to construct and may not always be a priority for health economists working on public health guidance. In such cases, deterministic sensitivity analysis should be used, and the decision not to use probabilistic methods should be justified in the guidance.

The Guide to the methods of technology appraisal includes other useful advice for health economists developing economic models for use in public health guidance.

6.3.6 General principles

Regardless of the modelling approach adopted, economic analysis should be based on the following principles:

  • The question being asked for the economic analysis should be clearly specified and appropriate and include comparison of all relevant alternatives for specified population groups.

  • Analysis should be carried out by the health economist in collaboration and following agreement with the CPHE project team and the PHAC.

  • It should be underpinned by the best-quality publicly available effectiveness evidence.

  • There should be the highest level of transparency in the reporting of methods and results. Conventions on reporting economic studies should be followed (see Drummond and Jefferson 1996).

  • Potential sources of bias and uncertainty should be fully explored and reported using an appropriate sensitivity analysis and discussed with the PHACs. Where the average effect size at individual level is small, particular attention should be given to potential biases, especially publication bias.

  • The limitations of the approach and methods used should be discussed with the PHACs and presented in the report.

  • Whenever a probabilistic sensitivity is carried out, a value of information (VOI) analysis should be undertaken to indicate whether further research is necessary, either before recommending an intervention or in conjunction with its recommendation. In other circumstances, a decision to undertake a VOI analysis should be made in discussion with the CPHE team.

6.3.7 Identifying and selecting model inputs

The review team and the CPHE project team should discuss whether or not any additional searches are needed. Additional searches may be necessary if the effectiveness searches do not provide the information needed for economic modelling. For example, information on:

  • the relationship between short- and long-term outcomes

  • the links between behaviour and disease (for example, intermediate and longer-term outcomes)

  • quality of life

  • resource use or costs.

It is not necessary to conduct formal, systematic literature searches for all the types of information required for economic modelling (although effectiveness data used in the modelling should be taken from the effectiveness reviews). For example, it is often better to get information on unit costs from: national list prices, the Personal Social Services Research Unit (PSSRU) Unit costs of health and social care report or the DH tariff. Information on costing can also be found in the NICE document Assessing cost impact: Methods guide (2011)) and through discussion with the costing analyst in the NICE implementation team. In addition, it might be better to obtain some information about epidemiology or public services from national statistics or databases, rather than from literature studies.

Ideally, systematic literature reviews should be conducted for other model inputs, for instance to model the relative risk of coronary heart disease in relation to physical activity levels. However, this is time-consuming and the health economist should look at other options first. Possibilities include: searching the public health effectiveness evidence that was used to structure the key questions (and perhaps other relevant questions) in the scope; or liaising with the review team, the PHACs and other experts.

If an additional literature search is necessary, the review team should discuss and agree this with the information scientist and the CPHE project team. If longer-term follow-up data are required, a literature search to identify cohort studies may be appropriate. The report by Philips et al. (2004) is a useful guide to data searching methods for economic models.

6.3.8 Subgroup analysis

Where appropriate, economic model(s) should be run to estimate the cost effectiveness of an intervention in different population subgroups. The subgroups of relevance will be agreed with CPHE project team.

6.4 Economic evidence and guidance recommendations

For an economic study or analysis to be useful, it must inform the public health guidance recommendations. The PHACs should discuss cost effectiveness in parallel with general effectiveness when formulating recommendations (see chapter 7).

6.4.1 Using cost–utility analysis (CUA)

If there is strong evidence that an intervention dominates the alternatives (that is, it is both more effective and less costly), it should normally be recommended. However, if 1 intervention is more effective but also more costly than another, then the ICER should be considered. The cost per QALY gained should be calculated as the difference in mean cost divided by the difference in mean QALYs for 1 intervention compared with the other.

If 1 intervention appears to be more effective than another, the PHACs will have to decide whether it represents reasonable 'value for money' as indicated by the relevant ICER. In doing so, the committee should also refer to the principles outlined in NICE's report Social value judgements: principles for the development of NICE guidance, second edition (2008). This states:

'NICE has never identified an ICER above which interventions should not be recommended and below which they should. However, in general, interventions with an ICER of less than £20,000 per QALY gained are considered to be cost effective. Where advisory bodies consider that particular interventions with an ICER of less than £20,000 per QALY gained should not be provided by the NHS they should provide explicit reasons (for example that there are significant limitations to the generalisability of the evidence for effectiveness). Above a most plausible ICER of £20,000 per QALY gained, judgements about the acceptability of the intervention as an effective use of NHS resources will specifically take account of the following factors.

  • The degree of certainty around the ICER. In particular, advisory bodies will be more cautious about recommending a technology when they are less certain about the ICERs presented in the cost-effectiveness analysis.

  • The presence of strong reasons indicating that the assessment of the change in the quality of life has been inadequately captured, and may therefore misrepresent, the health gain.

  • When the intervention is an innovation that adds demonstrable and distinct substantial benefits that may not have been adequately captured in the measurement of health gain.

As the ICER of an intervention increases in the £20,000 to £30,000 range, an advisory body's judgement about its acceptability as an effective use of NHS resources should make explicit reference to the relevant factors considered above. Above a most plausible ICER of £30,000 per QALY gained, advisory bodies will need to make an increasingly stronger case for supporting the intervention as an effective use of NHS resources with respect to the factors considered above.'

However, such thresholds do not exist outside the health sector, making it difficult to judge whether the benefits accruing to the non-health sectors are cost effective. The PHACs should take into account the factors it considers most appropriate. These could include non-health-related outcomes that are valued by the rest of the public sector (like reduced levels of public disorder resulting from reduced alcohol consumption).

Decisions about whether to recommend interventions should not be based on cost effectiveness alone. The advisory committees responsible for developing the guidance should also take into account other factors, such as the need to prevent discrimination and to promote equity. They will consider trade-offs between efficient and equitable allocations of resources. These factors should be explained in the considerations section of the guidance (see section 7.6). If a structured public health question is not considered for further economic analysis, the PHACs should still consider the likely cost effectiveness of the associated recommendations. If available, this assessment may be based on published estimates of cost effectiveness or, if not, a qualitative judgement based on prior knowledge and undertaken in an open and transparent process.

6.4.2 Using cost–benefit analysis (CBA)

Where the WTP is aggregated individual WTP, the NICE committee should be aware that this is likely to exceed public-sector WTP, sometimes by quite a margin. If a conversion factor between these 2 forms of WTP has been established in any particular circumstance, the committee should take this into account. In the absence of a conversion factor, the committee should consider the likelihood of a discrepancy between the 2 measures in making recommendations that rely on a CBA.

The committee should also attempt to determine whether any adjustment should be made to convert 'ability-to-pay' estimates into those that prioritise on the basis of need and the ability of an intervention to meet that need.

After accounting for the above factors, committees should not recommend interventions whose net present value (NPV) is estimated to be negative unless they believe that non-economic objectives override the economic loss involved. Given a choice of interventions with positive NPVs, committees should prefer the intervention that maximises the NPV, unless non-economic objectives override the economic loss incurred by choosing an intervention that does not maximise NPV.

6.4.3 Using cost–consequences analysis (CCA)

NICE committees should ensure that, where possible, the different sets of consequences do not double count costs or benefits. The way that the sets of consequences have been implicitly weighted should be recorded as openly, transparently and as accurately as possible. CCA then requires the decision-maker to decide which interventions represent the best value, preferably using systematic and transparent process. Various tools are available to support this part of the process.

6.5 References and further reading

Briggs A, Claxton K, Sculpher M (2006) Decision modelling for health economic evaluation. Oxford: Oxford University Press

Cooper NJ, Sutton AJ, Ades AE et al. (2007) Use of evidence in economic decision models: practical issues and methodological challenges. Health Economics 16: 1277–86

Drummond MF, Jefferson TO (1996) Guidelines for authors and peer reviewers of economic submissions to the BMJ. British Medical Journal 313: 275–83

Drummond MF, McGuire A (2001) Economic evaluation in health care: merging theory with practice. Oxford: Oxford University Press

Drummond MF, Sculpher MJ, Torrance GW et al. (2005) Methods for the economic evaluation of health care programmes, 3rd edition. Oxford: Oxford University Press

Eccles M, Mason J (2001) How to develop cost-conscious guidelines. Health Technology Assessment 5: 1–69

NHS Centre for Reviews and Dissemination (2001) Improving access to cost-effectiveness information for health care decision making: the NHS Economic Evaluation Database. CRD report number 6, 2nd edition. York: NHS Centre for Reviews and Dissemination, University of York.

Philips Z, Ginnelly L, Sculpher M et al. (2004) Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technology Assessment 8 (36).

Raftery J, editor (1999–2001) Economics notes series. British Medical Journal

Wanless D (2004) Securing good health for the whole population. London: HM Treasury

[8] Adapted from table 5.1 in Guide to the methods of technology appraisal (NICE 2008).