NICE process and methods

7 Incorporating economic evaluation

7.1 The role of the economist in social care guidance development

7.1.1 The economic plan

The NICE Collaborating Centre for Social Care's (NCCSC's) economist works closely with the Guidance Development Groups (GDG) to ensure that economic evidence reviews are underpinned by the most plausible assumptions and review questions. Defining the priorities for economic evaluation should start during the scoping of social care guidance, and should proceed alongside development of the review questions.

The economic plan is a document that is prepared by the NCCSC's economist in consultation with the rest of the NCCSC, the GDG and NICE. It identifies initial priorities for further economic analysis and outlines proposed methods for addressing these questions (see sections 7.1.3 and 7.1.4). The economic plan is likely to be modified during guidance development; for example, as evidence is reviewed it may become apparent that further economic evaluation is not necessary for some aspects that were initially prioritised. The economic plan is published on the NICE website before the guidance goes out for consultation. The rationale for the final choice of priorities for economic analysis should be explained in the guidance.

7.1.2 Advising the Guidance Development Group on economic issues

The GDG should be encouraged to consider the economic consequences of the guidance recommendations as well as the implications for practice. A formal presentation outlining the basic principles of social care economics is given at the first GDG meeting. Further presentations may be useful later in the guidance development process.

It is particularly important for GDG members to understand that economic analysis is not about estimating the resource consequences of a guidance recommendation, but is concerned with the evaluation of both costs and benefits. GDG members must understand that economic evaluation should compare the costs and consequences of alternative courses of action.

Within the context of the principles outlined in Social value judgements: principles for the development of NICE guidance (second edition, 2008) (note this is currently under review, see section 1.2), the GDG should be encouraged to consider recommendations that:

  • are less effective than current practice, but free up resources that can be re-invested in public sector social care to increase the welfare of the population receiving social care, or

  • increase effectiveness at an acceptable level of increased cost (see section 7.3).

GDG members may find it useful for the economist to outline economic concepts such as incremental analysis, the public sector perspective (including NHS and personal social services[3]), and measurement of social care-related outcome measures, including social care-related quality of life, capability and wellbeing.

7.1.3 Reviewing economic evaluations

Economic analysis should be underpinned by the best available evidence. The evidence should be based on, and be consistent with, that identified when defining the review questions. If expert opinion is used in the economic analysis, this should be clearly stated and justified in the guidance.

Identifying and examining published economic information that is relevant to the review questions is an important component of social care guidance development. Processes for searching, selecting, appraising and summarising economic evaluations are discussed in sections 5.3, 6.1.3, 6.3.3 and 6.3.5.

The general approach to reviewing economic evaluations should be systematic, focused and pragmatic. If a high-quality economic analysis that addresses a key social care issue and is relevant to current practice has already been published, then further modelling is probably not necessary.

Other published economic evaluations may not be relevant, for example, costs may differ from UK costs in non-UK studies. So time should not be spent critically appraising studies that are not likely to provide useful information for guidance decision-making. Search strategies and criteria for the inclusion and exclusion of economic evaluations should be designed to filter out such studies (see section 5.3). The strategies and criteria used should be stated explicitly in the guidance and applied consistently.

7.1.4 Prioritising questions for further economic analysis

It is anticipated that the social care economic literature is unlikely to be so comprehensive and conclusive that no further analysis is necessary. Additional economic analyses are usually needed, and should be developed selectively, unless an existing analysis can easily be adapted to answer the question.

Close collaboration between the economist and the GDG is essential early in the guidance development process to ensure that:

  • the most important questions are selected for economic analysis

  • the methodological approach is appropriate

  • all important effects and resource costs are included

  • the evidence used is the best available and the assumptions are plausible

  • results of economic analysis are interpreted appropriately, limitations are acknowledged and uncertainties are systematically addressed.

The number and complexity of new analyses depends on the priority areas and the information needed for robust decision-making.

Economic analysis is potentially useful for any question in which an intervention or service is compared with another. It may also be appropriate in comparing different combinations or sequences of interventions, as well as individual components of the social care service or intervention. However, the broad scope of much social care guidance means it may not be possible to conduct original analysis for every component.

Selecting questions for further economic analysis, including modelling, should be a joint decision between the economist, other GDG members and the NICE team. Selection should be based on systematic consideration of the potential value of economic analysis across all key social care issues.

An economic analysis will be more useful if it is likely to influence a guidance recommendation, and if the social care and financial consequences of the recommendation are large. The decision about whether to carry out an economic analysis therefore depends on:

  • the expected net benefit of the recommendation (the number of individuals affected and the potential impact on costs and social care outcomes per individual)

  • the degree of uncertainty in the cost-effectiveness literature and the likelihood that economic analysis will clarify matters.

For a particular question, new economic analysis may not be warranted if the evidence is so uncertain that even providing a rough estimate of cost effectiveness is not possible. Alternatively, the published evidence on cost effectiveness may be so reliable that further economic analysis would be unnecessary. In addition, economic analysis may not be needed if it is obvious that the resource implications are modest in relation to the expected gains.

7.2 Economic reference case

Economic analysis should be undertaken in collaboration with the GDG, explicitly based on the guidance review questions, and should compare all relevant alternatives for specified groups of service users. Any differences between the review questions and the economic analysis should be clearly acknowledged, justified and explained, and be approved by the GDG.

A reference case for social care economic evaluation is outlined in table 4. Methods of social care economic evaluation are developing, and elements of the reference case reflect this; it may also be appropriate for the nature of economic evaluation to vary according to the type of social care being assessed or the quality of available evidence.

Table 4 Summary of the reference case

Element of assessment

Reference case

Section providing details

Defining the decision problem

The scope developed by NICE



Interventions routinely delivered by the public and non-public social care sector


Perspective on costs

Public sector, including the NHS, personal social services and local authorities (and other public sector agencies as appropriate)

Non-public sector funding, including family funding and the costs of unpaid care, if these contribute to outcomes


Perspective on outcomes

Effects on people for whom services are delivered (service users and/or carers)


Type of economic evaluation

Cost–utility analysis

Cost-effectiveness analysis

Cost–consequences analysis

Cost–benefit analysis


Time horizon

To reflect all important differences in costs and outcomes


Synthesis of evidence on outcomes

Based on a systematic review


Measuring and valuing effects

Quality-adjusted life year (QALY) or 'social care QALY' with parallel evaluation based on capability measures where an intervention results in both capability and health or social care outcomes.

ASCOT instruments may be used as measures of social care quality of life and ICECAP instruments may be used to measure capability


Source of data for measurement of effects

Reported directly by service users and/or carers


Source of preference data for valuation of changes in social care-related quality of life, capability, etc.

Representative sample of the UK public


Equity considerations

Equity considerations relevant to specific topics, and how these were addressed in economic evaluation, must be reported


Evidence on resource use and costs

Costs should relate to public sector (such as NHS, personal social services and local authority) resources and be valued using relevant prices; costs borne by service users and the value of unpaid care must also be included where they contribute to outcomes



An annual rate of 3.5% on both costs and effects


7.2.1 Defining the decision problem

Economic evaluation should begin with a clear statement of the decision problem. This needs a definition and justification of the interventions or programmes being assessed and the relevant service users (including carers).

7.2.2 Comparators

The interventions or services included in the analysis should be described in sufficient detail to allow stakeholders to understand exactly what is being assessed. This is particularly important when assessing the cost effectiveness of services.

For social care guidance, comparators include interventions used or services delivered in the public and non-public sectors. The contribution of unpaid social care provided by families and carers may also be important. There should be agreement with the GDG on the most appropriate choice of comparator because of its important impact on the results of economic evaluation.

7.2.3 Perspective on costs

Social care-related outcomes are delivered by a range of providers, including various public sector agencies, commercial providers, the voluntary and community sector and unpaid (family) care. The costs are borne by various public sector agencies, service users and their families. The payer-provider matrix is complex and differs across the range of social care services and programmes; some social care is provided for carers, for example. Unlike healthcare, there is no universal model of who is liable to pay for social care services, and there may be local variation in eligibility criteria, including means testing.

There may be local variation in how decisions are made about which social care is provided, for example, in terms of the use of personal social care budgets. Social care economic evaluation must reflect this complex situation and so a multi-stakeholder perspective will be adopted, with exclusions justified.

A public sector perspective (including the NHS and personal social services, or local government) is used by NICE for public health economic analysis, if this is sufficient to capture all major costs and benefits. However, the perspective is flexible, and a societal perspective can be used where appropriate.

For social care, a flexible approach may be needed to take account of costs borne by the NHS, local authorities, education authorities, relevant publicly funded voluntary organisations or criminal justice services.

Scope for 'cost shifting' should be addressed in the economic evaluation, and the results of economic evaluation may need to be presented for different public sector agencies.

The cost of social care programmes and interventions may be borne by service users and their families, for example, when access to social care is means-tested. Some social care may also be provided on an unpaid basis, often by the carers of service users. There is the potential for costs to be shifted inappropriately from the public sector to families and unpaid carers.

NICE does not usually include the costs of unpaid or informal care in cost-effectiveness analysis for health and public health interventions. However, the importance of informal care in contributing to publicly funded social care interventions (for example, care for older people) means that economic evaluation in social care should take account of the value of unpaid care associated with the services or interventions under evaluation. The inclusion of unpaid care can have a significant impact on the potential cost effectiveness of social care interventions because unpaid care can be a substantial part of the care provided and therefore impacts on the outcomes of publicly funded services. It is recognised that there is no widely accepted method for valuing unpaid care, so any methods used should be justified and agreed with NICE before economic evaluation is undertaken, and consideration must be given to the sensitivity of results to the use of alternate methods. The issue is potentially complex, for example, some service users may pay privately to ensure additional 'quality' of care. In taking account of unpaid care, economic evaluation must be designed to ensure that the focus of the economic evaluation remains on assessing the cost effectiveness of publicly funded social care interventions and services.

Economic evaluation should also recognise that social care provided by the voluntary sector may be based on public-sector funding (grants to voluntary bodies, for example).

If appropriate, and with the agreement of NICE, results may also be presented from other perspectives. For example, an employer's perspective could be taken to show the business case for a social care intervention.

It is envisaged that the analytical difficulties involved in creating clear, transparent decision rules around what costs should or should not be considered, and for which interventions and outcomes, will be particularly problematic. These should be discussed with NICE before any economic evaluation is undertaken and an approach agreed.

7.2.4 Perspective on outcomes

The outcomes focus should be the effects on people for whom services are delivered. Effects on service users and carers (whether expressed in terms of health effects, social care quality of life, capability or wellbeing) are the intended outcomes of social care interventions and programmes.

Although holistic effects on service users, families and carers may represent the ideal perspective on outcomes, a pragmatic and flexible approach is needed to address different perspectives, recognising that improved outcomes for service users and carers may not always coincide.

The economic implications of alternative service outcomes could be the focus of some economic evaluations, and where appropriate, this should be agreed with NICE at the outset.

7.2.5 Type of economic evaluation

The approach to economic evaluation will be agreed with NICE at the outset. If possible, a cost-effectiveness analysis should be undertaken, with effects measured using a non-monetary outcome indicator. A cost–utility analysis, which allows effective comparisons across different decision problems, is the form of cost-effectiveness analysis usually preferred by NICE but the use of cost–utility analysis in social care economic evaluation will present methodological challenges because currently there is no accepted social care equivalent of the healthcare QALY.

If a cost-effectiveness analysis is not appropriate, or not possible, other validated methods such as cost–consequences or cost–benefit analysis may be used. The aim is to give the GDG an opportunity to consider the cost effectiveness of social care interventions and programmes across a complex arrangement of payers and providers. Cost–benefit analysis should only be used with NICE's agreement and should not conflict with NICE's social value judgements.

For social care topics, benefits may variously be identified in terms of health or social care-related quality of life, or in terms of capability. Evaluation related to quality of life may be undertaken in parallel with evaluation based on capability measures, where an intervention results in both types of outcome.

Cost–utility analysis

A cost–utility analysis is a form of cost-effectiveness analysis in which interventions producing different effects are expressed in terms of a common measure of outcome, such as QALYs.

NICE uses cost–utility analyses for technology appraisals and clinical guidelines and, if suitable data are available, for public health guidance. This ensures baseline comparability across the UK healthcare sector and across NICE's programmes. It also helps to prioritise which recommendations should be implemented locally. The same approach should be considered for social care economic evaluation if suitable data are available.

In healthcare and public health analysis, NICE compares the efficiency of alternative interventions using the QALY as the measure of outcome. A QALY is a measure of the state of health of a person or group in which the benefits, in terms of length of life, are adjusted to reflect the quality of life. QALYs are calculated by estimating the years of life a person can expect after a particular intervention and weighting each year with a quality of life score (on a 0 to 1 scale).

Box 11 Example of how a QALY is calculated in healthcare

Patient X has a serious, life-threatening condition.

  • If he continues receiving standard treatment, he will live for 1 year, with a quality of life of 0.4 (0 or below = worst possible health, 1 = best possible health).

  • If he receives the new drug, he will live for 1 year 3 months (1.25 years), with a quality of life of 0.6.

The new treatment is compared with standard care in terms of the QALYs gained:

  • Standard treatment: 1 (year's extra life) × 0.4 = 0.4 QALY

  • New treatment: 1.25 (1 year, 3 months extra life) × 0.6 = 0.75 QALY

The new treatment leads to 0.35 additional QALYs (that is: 0.75–0.4 QALY = 0.35 QALYs).

For social care economic evaluation, there is currently no commonly accepted measure of outcome like the QALY in health economics. However, various outcome measures for social care are emerging, ranging from social care-related quality of life to broader measures of capability and wellbeing. As economic methodology develops, these measures may allow cost–utility analysis on the basis of a social care equivalent of the QALY.

A pragmatic approach to cost–utility analysis for social care should use a measure of social care-related quality of life, but parallel work using capability and wellbeing measures should also be undertaken if possible. Specific consideration may be given to identifying the nature of effects, because health effects may be relevant to some social care guidance topics and may necessitate the use of health economic approaches for elements of the economic evaluation.

Other cost-effectiveness analysis

Cost–utility analysis is a form of cost-effectiveness analysis, but if data or evidence are not sufficient to support it, an alternative may be considered. Examples include using non-generic or intervention-specific measures.

A cost-effectiveness analysis could be modelled on a single well-conducted randomised controlled trial. Or it could use decision-analytic techniques to analyse probability, cost and outcome data from a variety of published sources.

Cost–consequences analysis

Cost–consequences analysis can measure both welfare and quality of life more broadly than the health-related quality of life measure encompassed in the EQ-5D instrument that NICE uses to estimate the QALY. It can take many other items into account that public sector bodies responsible for social care are likely to find important, including the trade-off between long-term goals and a paucity of short-run funding, and spill-over effects into other areas of public sector responsibility. The extent to which these effects are material for any particular analysis will depend on circumstances. The outcomes to be included in the cost–consequences analysis will depend on circumstances and should be discussed in advance with the NICE team.

Cost–benefit analysis

Cost–benefit analysis studies have been used sparingly at NICE in the past because cost–utility analysis studies have usually been adequate for interventions that involve health and healthcare alone. If cost–benefit studies occur in economic evaluation literature included in the evidence, NICE has used these studies. Cost–benefit analysis converts all benefits and costs that can be readily quantified into monetary terms. It sums the costs and benefits separately to arrive at either a net monetary benefit or a ratio of benefits to costs and consequently it usually operates with a societal perspective.

If a cost–benefit analysis is proposed, the economist and NICE should agree how benefits are to be valued in line with the reference case and general NICE economic evaluation principles.

These analyses could be used to:

  • modify a decision based solely on cost–utility considerations

  • explain to stakeholders and the public the additional costs and benefits for organisations outside the public social care sector.

There is often a trade-off between the range of new analyses that can be conducted and the complexity of each piece of analysis. Simple methods may be used if these provide the GDG with sufficient information on which to base a decision. For example, if an intervention is associated with improved outcomes and fewer adverse effects, then a simple decision tree may provide a sufficiently reliable estimate of cost effectiveness. In other situations, a more complex approach may be warranted.

7.2.6 Time horizon

The time horizon should reflect all important differences in costs and outcomes, and the reasons for the time horizon used should be documented.

7.2.7 Measuring and valuing effects

The QALY is the measure of health effects preferred by NICE, based on patient-reported changes in health-related quality of life. The EQ-5D is the preferred measure of health-related quality of life in adults, with utility of changes being based on public preferences.

For social care economic evaluation, a flexible approach is needed, reflecting the nature of effects delivered by different social care interventions or programmes. If health effects are relevant, the EQ-5D-based QALY may be used. However, it is likely that broader, preference-weighted measures of social care outcomes, based on specific instruments, will be more appropriate.

Social care quality-of-life measures are being developed and NICE will consider using 'social care QALYs' if validated.

For example, the ASCOT (Adult Social Care Outcome Toolkit) set of instruments is used by the Department of Health in the Adult Social Care Outcomes Framework indicator on social care-related quality of life.

If a social care intervention is associated with both health and social care-related outcomes, it may be helpful for these elements to be presented separately, especially considering the emerging nature of social care economic evaluation.

Similarly, depending on the topic, and on the intended effects of social care interventions and programmes, the economic evaluation should also consider a parallel study of effects in terms of capability and wellbeing. For capability effects, use of the ICECAP (Investigating Choice Experiments for the Preferences of Older People – CAPability) instruments may be considered as part of the longer-term development of methodology.

In a field of emerging methodologies, and in the context of the variety of social care services, it is recognised that there is no single correct approach to measuring effects. The reference case, although referring to ASCOT-based social care quality of life in parallel with capability-based analysis based on ICECAP measures, should be approached with some flexibility, as long as decisions about the measurement of effects are clearly explained as part of economic evaluation reports.

Measurement of effects is as reported by service users or carers as relevant.

The source of preference data for valuing changes in quality of life and capability should be a representative sample of the UK public. However, if effects are expressed in terms of wellbeing, this must be valued by individual service users and carers.

7.2.8 Equity considerations

NICE healthcare and public health economic evaluation does not include equity weighting – a QALY normally has the same weight for all population groups.

It is important to recognise that social care provision may be means-tested, and that this affects the economic perspective in terms of who bears costs – the public sector or the service user or family. Economic evaluation should reflect the intentions of the social care system. Equity considerations relevant to specific topics, and how these were addressed in economic evaluation, must be reported.

Social care economic evaluation is unlikely to involve formal equity weighting at the outset. Economic analysis for presentation to the GDG may, however, be accompanied by a briefing on equity issues specific to social care (for example, using subgroup analysis). This approach would need to be agreed with NICE at the outset of economic evaluation.

7.2.9 Evidence on resource use and costs

Costs should relate to public sector resources (such as the NHS and personal social services) and be valued using relevant prices. Costs borne by service users and the value of unpaid care must also be included.

7.2.10 Discounting

The need to discount to a present value is widely accepted in economic evaluation, although the specific rate is variable across jurisdictions and over time. NICE considers that it is usually appropriate to discount costs and health effects at the same rate. The annual rate of 3.5%, based on the recommendations of the UK Treasury for the discounting of costs, should be applied to both costs and effects.

Sensitivity analyses using rates of 1.5% for both costs and effects may be presented alongside the reference-case analysis.

7.2.11 Identifying and selecting model inputs

If existing models are being used, or are informing new analysis, the way these studies are adapted or used should be outlined clearly.

Additional searches may be needed, for example, if effectiveness searches do not provide the information needed for economic modelling, including:

  • the relationship between short- and long-term outcomes

  • quality of life

  • resource use or costs.

It is not necessary to conduct formal, systematic literature searches for all the types of information required for economic modelling (although effectiveness data used in the modelling should be taken from the effectiveness reviews). For example, information on unit costs can be obtained from the Personal Social Services Research Unit report unit costs of health and social care or the Department of Health tariff. Information on costing can be found in the NICE Assessing cost impact: methods guide (2011) or from a costing analyst in the NICE implementation team. Some information about public services may be better obtained from national statistics or databases, rather than from literature studies.

7.2.12 Exploring uncertainty

Considerations of potential bias and limitations should be discussed by the GDG. Appropriate sensitivity analysis should be used (depending on the type of uncertainty) to explore the impact of potential sources of bias and uncertainty on the results of the economic analysis.

7.3 Economic evidence and guidance recommendations

For an economic evaluation to be useful, it must be taken into account in the guidance recommendations. The GDG should discuss cost effectiveness in parallel with general effectiveness when formulating recommendations (see section 9).

All economic analyses should be validated. The validation process should be outlined in the guidance. Useful and practical validation methods could include:

  • systematic checking of model formulae and inputs by a second modeller

  • sensitivity analysis

  • ensuring that the model results are plausible and can be explained

  • comparing end points from the model with source materials.

There should be the highest level of transparency in reporting methods and results.

In cost–utility analysis, if there is strong evidence that an intervention dominates the alternatives (that is, it is both more effective and less costly), then clearly it should be recommended. However, if an intervention is more effective but also more costly than another, then the incremental cost-effectiveness ratio (ICER) should be considered if possible. The ICER is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes. ICERs are expressed as cost (in £) per QALY gained.

If an intervention appears to be more effective than an alternative, the GDG must decide whether any increase in cost associated with the increase in effectiveness represents reasonable 'value for money'. In doing so, it should refer, as appropriate, to the principles outlined in Social value judgements: principles for the development of NICE guidance (second edition, 2008) (see section 1.2). Currently, no threshold has been established for social care and QALYs may not be an appropriate measure. So the GDG needs to make a judgement based on the economic evidence provided.

It is likely that over time, and as the methodology develops, indicative cost-effectiveness thresholds will be established for social care-related utility measures.

In using cost–consequences analysis, the GDG should ensure, where possible, that the different sets of consequences do not double-count costs or benefits. The way that the sets of consequences have been implicitly weighted should be recorded as openly, transparently and as accurately as possible. Cost–consequences analysis then requires the decision-maker to decide which interventions represent the best value, preferably using systematic and transparent process. Various tools are available to support this part of the process.

In using cost–benefit analysis, the GDG will need to take account of potential issues with the willingness to pay for a benefit in a cost–benefit analysis being the aggregation of individually elicited willingness to pay, as distinct from the willingness to pay of a public body (which may be exceeded).

Decisions about whether to recommend interventions should not be based on cost effectiveness alone. The GDG should also take into account other factors, such as the need to prevent discrimination and to promote equality. The GDG considers trade-offs between efficient and equitable allocations of resources. The issue of equity weighting in social care economic evaluation, and how any relevant means-testing in the social care system is addressed within the economic evaluation, is relevant to this.

These factors should be explained in the Evidence to recommendations section of the guidance (see section 9.2). If a structured social care question is not considered for further economic analysis, the GDG should still consider the likely cost effectiveness of the associated recommendations. This assessment may be based on published estimates of cost effectiveness if available or, if necessary, a qualitative judgement.

7.4 References and further reading

Anderson R (2010) Systematic reviews of economic evaluations: utility or futility? Health Economics 19: 350–64

Centre for Reviews and Dissemination (2007) NHS economic evaluation database handbook [online].

Cooper NJ, Sutton AJ, Ades AE et al. (2007) Use of evidence in economic decision models: practical issues and methodological challenges. Health Economics 16: 1277–86

Drummond MF, McGuire A (2001) Economic evaluation in health care: merging theory with practice. Oxford: Oxford University Press

Drummond MF, Sculpher MJ, Torrance GW et al. (2005) Methods for the economic evaluation of health care programmes, 3rd edition. Oxford: Oxford University Press

Eccles M, Mason J (2001) How to develop cost-conscious guidelines. Health Technology Assessment 5: 1–69

Jackson CH, Bojke L, Thompson G et al. (2011) A framework for addressing structural uncertainty in decision models. Medical Decision Making 31: 662–74

National Institute for Health and Clinical Excellence (2012) Social care guidance development methodology workshop December 2011: report on group discussions

National Institute for Health and Clinical Excellence Decision Support Unit (2011) Technical support document series

Philips Z, Ginnelly L, Sculpher M et al. (2004) Review of good practice in decision-analytic modelling in health technology assessment. Health Technology Assessment 8: 1–158

Raftery J, editor (1999–2001) Economics notes series. British Medical Journal [online]

[3] Department of Health definition: personal social services describes care services for vulnerable people, including those with special needs because of old age or physical disability and children in need of care and protection. Examples are residential care homes for older people, home help and home care services, and social workers who provide help and support for a wide range of people.