Process and methods

2 Topic selection and scoping the guidance

2.1 Introduction and overview of process

This chapter describes how topics for public health guidance are selected, and how the scope of the guidance is developed.

Suggestions for topics come from a variety of sources including:

  • stakeholders

  • NICE advisory committees

  • topic advisory workshops (TAWs)

  • the guidance review process

  • other NICE guidance producing centres.

Centre for Public Health Excellence (CPHE) also liaises with a range of key stakeholder and practitioner groups to identify topics, for example the Association of Directors of Public Health, the Faculty of Public Health and the Royal Colleges, as well as the key stakeholders mentioned in the consultation proposals (Department of Health [DH], Public Health England and Local Government).

Once a topic has been proposed it is considered by the NICE internal topic selection group – a cross institute group with representation from CPHE, implementation and field team leads, and quality systems.

If the internal group decides the topic is a priority for public health guidance, a background briefing paper (section 2.2) is produced. A TAW (section 2.2.4) is convened and develops a set of potential draft referrals in the topic area. The workshop is also an opportunity to suggest any additional topics that may need NICE public health guidance.

The draft referrals are discussed with policy leads at the DH, who then prepare a submission to ministers for formal consideration (see section 2.2.5). On completion of this stage of the process, a formal ministerial referral to develop public health guidance on the topic is made to NICE.

When NICE receives the ministerial referral, CPHE determines its place in the forward work programme. Once the project gets underway, CPHE develop the scope of the topic that sets out what the guidance will and will not cover (see sections 2.3, 2.4, 2.5).

2.2 Briefing paper for the TAW

2.2.1 Using the conceptual framework to construct a logic model

After holding the internal topic selection meeting and identifying a potential topic area, the CPHE, assisted by NICE information services, will prepare the briefing paper for the TAW.

The first step is to use the NICE public health conceptual framework (section 1.4) to construct a topic-specific conceptual framework (see below and appendix A). This framework may incorporate the assumed relationships between action and outcomes and, if possible, any relevant programme theory or theory of change (Pawson 2006; Weiss 1995).

The framework is used to:

  • help define the key issues involved in a broad topic area

  • specify where more focused and clearly defined topics fit into the overall strategic map of public health topics developed so far by CPHE.

Topics are mapped onto the vectors of causation outlined in the framework (population, environment, society and organisations) and cross-classified according to the potential level of intervention (population, community, organisation, family, domestic or individual).

The framework may also be used to construct a logic model. This model should incorporate the assumed relationships between action and outcomes described in the conceptual framework. Both the conceptual framework and the logic model are set out below (see figures 2.1–3).

Figure 2.1 Conceptual framework for public health guidance

Figure 2.1 Conceptual framework for public health guidance

Figure 2.2 A conceptual framework for promoting mental wellbeing at work

Figure 2.2 A conceptual framework for promoting mental wellbeing at work

Figure 2.3 A logic model for promoting mental wellbeing at work

Figure 2.3 A logic model for promoting mental wellbeing at work

2.2.2 Preliminary information exercise

Before the NICE information services team starts the preliminary search for information, it consults the logic model to determine which literature (including which type of literature) will be most relevant to the topic.

It then carries out the preliminary information gathering exercise. In some cases, this will be related to a broad topic area; in others it will be more specific. It should bring together different sources of information to provide an overview of the topic (including key issues). This is not a comprehensive search of the evidence base at this stage but rather aims to provide a background for development of the subsequent scope.

Databases and websites may be searched to identify key epidemiological reviews, policy documents and existing reviews of effectiveness. It is particularly important to identify any related NICE guidance (existing or in development) and other related guidance published in the UK (and abroad, if relevant). Key sources of relevant health inequalities information are also important.

Useful generic resources might include:

  • standard bibliographic research databases – see below (these are useful for assessing the extent of the effectiveness literature and for epidemiological assessments of the extent of the problem and its determinants)

  • relevant texts and monographs

  • government and other policy websites

  • guideline gateways in the UK and overseas

  • websites of UK and overseas agencies that produce guidance or recommendations

  • practice databases

  • routine statistics (for example, monitoring and surveillance data).

The precise details of the search will be governed by the logic model and the topic. For a list of databases and websites that the search might include, see appendix B.

2.2.3 Preparing the briefing paper for the TAWs

The briefing paper aims to:

  • map the topic onto the vectors of causation and levels of delivery (see appendix A)

  • outline the vectors and behaviours involved in the causal pathway

  • outline the systems and processes involved in delivery

  • establish the epidemiological importance of the topic

  • identify links to policy

  • describe any intervention involved as precisely as possible

  • identify the links between the intervention and its health and other outcomes using a logic model, a programme theory or a theory of change, with reference to the vectors, behaviours and organisational structures

  • identify any equity issues

  • discuss whether a universal, targeted or hybrid approach is likely to be appropriate

  • discuss the resource implications (for the NHS or other sectors) of taking or not taking action

  • highlight issues that may affect implementation

  • outline the approach for assessing effectiveness, cost effectiveness and equity

  • develop the topic-specific logic model based on the overarching conceptual framework.

On completion, briefing papers are presented to the TAWs.

2.2.4 TAWs

TAWs are held on regular basis and attendees are drawn from a range of key stakeholder and practitioner groups including:

  • Association of Directors of Public Health

  • Faculty of Public Health

  • Royal Colleges

  • DH

  • Public Health England

  • Local Government.

Representation is also sought from a range of public, private and third sector organisations including relevant patient, carer and service user groups. At the TAW, NICE will discuss with the DH which potential topic areas align with government policy.

Discussions are informed by the briefing papers and, where appropriate, by the conceptual framework discussed in section 2.2.1 (see appendix A for more detail).

The TAW also considers the extent of inequalities in health in relation to the topic area under consideration and whether or not the potential guidance would address these issues.

2.2.5 Preparing referrals

Draft referrals from the topic advisory group are refined by CPHE and an internal consistency check is carried out to ensure topics do not overlap and are properly integrated with other public health or clinical guidance. The draft referrals are discussed with policy leads at the DH, who then prepare a submission to ministers for formal consideration. Once ministers have considered the submission, 1 or more formal ministerial referrals may be made to NICE to develop public health guidance in the topic area. Final responsibility for public health guidance referrals remains with the Secretary of State for Health. If ministers make a referral CPHE then determine where the guidance development fits in the CPHE's forward programme. Once determined, work begins on developing the scope of the topic, which is a formal document that sets out what the guidance will and will not cover (see sections 2.3, 2.4, 2.5).

2.3 Purpose of the scope

The logic model constructed for the briefing paper and any considerations from the relevant TAW are the starting point for development of the scope.

The scope for guidance should either:

  • specify precisely which intervention(s), strategies or activities are covered (by describing them in some detail) or

  • specify what types of interventions, strategies or activities are covered (and include some specific examples).

In addition, the scope should:

  • Provide a clear definition of the intervention(s), strategies or activities to be addressed and, where appropriate, the relationship between them. This definition should cover the nature, content of each public health action, and the way it is delivered.

  • Describe the assumed mechanism, mediator or link between the action(s) and the outcome(s). In the case of some guidance, outcomes may be specified in terms of health or disease; an intermediate outcome could be a behaviour leading to the disease or to health improvement. For other guidance, outcomes may be more general, relating to strategy, organisational structure or service delivery.

  • Identify the causal pathway with reference to the vectors, health behaviours and means of delivery (appendix A).

  • Identify the level of delivery (individual, family, community or population level).

  • Identify the settings where the intervention(s) takes place.

  • Define what the guidance will include and exclude.

  • Identify the population(s) to be included and excluded.

  • Briefly describe the relevant epidemiology.

  • Identify the approach to equity.

  • Set the policy context.

  • Include the key, overarching questions that the guidance will address.

  • Outline the issues that the public health advisory committee (PHAC) is likely to consider.

  • Set clear parameters to ensure the guidance can be developed in the allocated time period.

  • Identify the economic approach to be taken, including any additional perspectives that will be taken into account (apart from those of the NHS), such as that of employers or the private sector.

  • Consider the extent to which evidence may be generally applicable.

  • Specify the outcomes and any comparators that will be used to judge effectiveness.

  • Indicate what kinds of evidence will be appropriate.

  • Identify links to other NICE guidance.

2.4 Drafting the scope

The following sections describe the procedures involved in developing the draft scope. A further literature search may be needed. If so, it must be based on the logic model. In addition, a list of 'considerations' should be developed (see section 2.4.2).

2.4.1 Developing key questions

The CPHE project team should refine the outline logic model constructed for the briefing paper (see figure 2.3) to draft a set of key questions that the guidance will aim to address. (These key questions, in turn, will be the starting point for developing research questions for the reviews.) The number of questions will depend on the topic and the breadth of the scope. However, it is important to ensure the total number:

  • is manageable for the PHAC

  • is feasible within the allocated budget

  • provides a sufficient focus for the guidance.

Key questions should be clear, realistic and focused. They should cover all areas of the guidance remit and should not introduce new aspects. They should be concerned with effectiveness, cost effectiveness, the relationship between interventions and outcomes, feasibility and acceptability and inequalities (also see section 2.4.2). They may cover the determinants of health, including risk and protective factors (and the relationships between them). Most important, the key questions should clearly relate to the interventions being considered and the outcomes that will be used to assess their effectiveness and cost effectiveness. Therefore, in addition to the logic model, the CPHE project team should use the population, intervention, comparison and outcome (PICO) framework to specify interventions, where appropriate, as follows.

  • Population under study

    • which populations are we most interested in?

    • how can they best be described?

    • are there subgroups that need to be considered?

    • are there any relevant inequality or exclusion issues here?

  • Intervention or approach

    • which interventions/approaches should be used?

  • Comparison

    • what are the main alternative(s), including 'usual practice' or 'do nothing'?

  • Outcome

    • what really matters to the population or individual?

    • which outcomes should be considered (for example, mortality, morbidity, relapse rates, physical and social functioning, costs, health status and so on)?

2.4.2 Considerations

The list of considerations should set out the factors that the PHAC are likely to consider when developing recommendations. They may become sub-questions for the reviews and may indicate where expert testimony will be required. They may include the various confounding factors that could influence the outcome, effectiveness and cost effectiveness of each type of activity, along with any other relevant information. The programme theory or theories of change approach and the logic model adopted may be helpful when compiling this list.

Typically, some of the following issues may be considered:

  • What factors and determinants does the intervention aim to influence (for example, does it aim to change risk factors)?

  • How valid and appropriate are the outcome measures used to assess effectiveness and cost effectiveness? (For example, how valid is self-report versus biologically validated measures?)

  • How does the content of an intervention or programme influence effectiveness and cost effectiveness?

  • How does the way the intervention or programme is carried out influence effectiveness and cost effectiveness?

  • Does effectiveness and cost effectiveness depend on who is delivering the intervention, for example their position, their age, gender or ethnicity? What are the significant features of an effective deliverer or leader?

  • Does the site or setting influence effectiveness and cost effectiveness?

  • How does the intensity, length or frequency of the intervention influence its effectiveness and cost effectiveness or duration of effect?

  • How does effectiveness and cost effectiveness vary according to the age, gender, class and ethnicity of the target population? Is there any differential impact on inequalities in health within and between different population groups?

  • How much does it cost (in terms of money, people, and time)?

  • What evidence is there on cost effectiveness?

  • Are there any factors that prevent – or support – effective implementation?

  • How acceptable is the intervention to the target population?

  • Are there any adverse or unintended outcomes?

  • Are there any trade-offs between equity and efficiency?

  • What differences are there in terms of availability and accessibility for different groups?

  • Whether it is based on an underlying theory or conceptual model.

  • Current practice.

The considerations, key questions and other parameters (for example, populations, activities and outcomes) outlined in the scope should be used to develop specific research questions for the reviews (see chapter 3). The programme theory or theories of change approach that has been adopted may also help.

The extent of inequalities in health in relation to the topic under consideration should have been considered at the briefing paper stage, during the TAWs and should be considered again as the scope is being written. Any issues identified should be used to find interested groups to invite to become stakeholders (these issues may also need to be raised with them).

2.5 Consulting on the scope

Once the scope has been drafted, it is posted on the NICE website and stakeholders are notified. Stakeholders can then register their interest and provide feedback by submitting comments via the official email address for the project during the consultation period. By exception, where a referral in a new area has been received, or where a guidance topic or an area of practice has unique complexities, a stakeholder meeting may be held. A stakeholder meeting does not replace the formal process of submitting comments.

After the public consultation, the scope should be reassessed for its potential impact on health inequities and any necessary amendments noted in the stakeholder response table. A final scope is prepared and published, along with stakeholder comments and NICE's response to those comments.

2.6 Further reading

Kelly MP, Stewart E, Morgan A et al. (2009) A conceptual framework for public health: NICE's emerging approach. Public Health 123: e14–20

Kelly MP, Morgan A, Ellis S et al. (2010) Evidence based public health: A review of the experience of the National Institute of Health and Clinical Excellence (NICE) of developing public health guidance in England, Social Science and Medicine, 71:1056–62

Pawson R (2006) Evidence based policy: a realist perspective. London: Sage

US Centers for Disease Control and Prevention (2005) The guide to community preventive services: What works to promote health? Atlanta: US Centers for Disease Control and Prevention

Weiss CH (1995) Nothing as practical as good theory: exploring theory-based evaluation for comprehensive community initiatives for children and families

Connell JP, Kubisch A, Schorr LB et al. New approaches to evaluating community initiatives: concepts, methods and context. Washington DC: Aspen Institute