Appendix B: Summary of the methods used to develop this guidance

Introduction

The reports of the reviews, expert papers and economic analysis include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it. The minutes of the PDG meetings provide further detail about the Group's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E and are available online.

Guidance development

The stages involved in developing public health programme guidance are outlined in the box below.

1. Draft scope released for consultation meeting

2. Stakeholder meeting about the draft scope

3. Stakeholder comments used to revise the scope

4. Final scope and responses to comments published on website

5. Evidence reviews and economic analysis undertaken

6. Evidence and economic analysis released for consultation

7. Comments and additional material submitted by stakeholders

8. Report of additional material produced (screened against inclusion criteria used in reviews)

9. Evidence and economic analysis submitted to the PDG

10. PDG produces draft recommendations

11. Draft guidance released for consultation and for field testing

12. PDG amends recommendations

13. Final guidance published on website

14. Responses to comments published on website

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by the PDG to help develop the recommendations. The overarching questions were:

  • What work or primary care-based interventions, programmes, policies or strategies are effective and cost-effective in

    • preventing or reducing the number of employees moving from short- to long-term sickness absence? This includes activities to prevent or reduce the re-occurrence of short-term sickness absence episodes.

    • helping employees who have been on long-term sickness absence to return to work?

    • helping to reduce the number of employees who take long-term sickness absence on a recurring basis?

  • What UK work or primary care-based interventions are effective and cost effective in helping people receiving incapacity benefit to return to full or part-time employment (paid and unpaid)? These could be delivered by a number of sectors (such as the voluntary or education sectors) in collaboration with, and/or funded by, employers and primary care services.

The subsidiary questions were:

  1. What is the frequency, content, length and duration of an effective intervention, programme, policy or strategy?

  2. Which are the most effective, cost effective and acceptable interventions, programmes, policies or strategies for different groups (for example, age, conditions, gender, ethnic groups or social classes)?

  3. Does the effectiveness of an intervention, programme, policy or strategy depend on who or what organisation is leading it (that is, internal or external occupational health provision or counselling support)?

  4. What are the barriers to, and facilitators of, effective implementation?

  5. Does the intervention, programme, policy or strategy lead to any adverse or unintended (positive and negative) outcomes?

  6. Which interventions, programmes, policies or strategies are ineffective and/or are not cost effective?

These questions were made more specific for each review (see reviews for further details).

Reviewing the evidence of effectiveness

A mapping review and three evidence reviews (covering the four main research questions) were conducted.

Identifying the evidence

The following databases were searched for review-level studies and primary studies published from 1990 onwards:

  • AMED (Allied and Complementary Medicine)

  • ASSIA (Applied Social Science Index and Abstracts)

  • Business Source Premier

  • British Nursing Index

  • CINAHL (Cumulative Index of Nursing and Allied Health Literature)

  • Cochrane Central Register of Controlled Trials

  • Cochrane Database of Systematic Reviews (CDSR)

  • Current Contents.

  • Database of Abstracts of Reviews of Effectiveness (DARE)

  • Econlit

  • EMBASE

  • Health Economics Evaluation Database (HEED)

  • HMIC (Health Management Information Consortium – King's Fund Database and DH-Data database)

  • MEDLINE

  • National Research Register

  • NHS EED (NHS Economics Evaluation Database)

  • NHS HTA (NHS Health Technology Assessment)

  • PsycINFO

  • Science Citation Index

  • SIGLE (International System for grey literature)

  • Sociological Abstracts

  • Social Science Citation Index

The following websites were also searched:

The reference lists of all review-level studies identified by the database and website searches were reviewed to identify additional potential references. Also, experts in the topic area (including PDG members) were contacted and asked to submit potentially relevant references. The reference lists for all primary studies that met the inclusion criteria were examined to identify any additional primary studies, and the citations of all included primary studies were also searched using Web of science and CINAHL.

Further details of the databases, search terms and strategies are included in the review reports.

Selection criteria

Studies were included in the two effectiveness and cost effectiveness reviews covering sickness absence if:

  • they were based in developed countries or in Organisation for Economic Co-operation and Development (OECD) countries

  • they were delivered in a primary care and/or workplace setting and/or planned, designed, delivered, managed or funded in collaboration with primary care providers and/or employers; the interventions could be delivered by a number of providers (such as voluntary, private, statutory sectors) and/or in various settings as long as they were fully or co-planned, designed, delivered, managed and/or funded in collaboration with employers and primary care settings

  • the population comprised people aged 16 or older in full- or part-time employment, both paid and unpaid and had experienced short-term sickness and/or long-term sickness (which may be defined as 'short-term absence' or 'long-term absence' or 'sickness absence' in the research)

  • they involved employers in the public, private or 'not for profit' sectors

  • they reported on work-related outcomes such as a return to work, job seeking behaviour or reduced sick absence

  • they covered an intervention that aimed to:

    • prevent or reduce the number of employees moving from short- to long-term sickness absence or prevent the recurrence of short-term absence

    • support return to work from, and/or reduce the length of, long-term sickness absence

    • help reduce recurrence of long-term sickness absence

  • the study design was randomised controlled trials (RCTs) or longitudinal intervention studies (that is, there is at least one follow-up measure after baseline) or randomised controlled trials (RCTs) with cost effectiveness, cost consequences, cost benefit, cost utility, cost minimisation or net monetary (cost) benefit data.

Studies were excluded if:

  • they were set in developing or non-OECD countries

  • the population comprised

    • self-employed people

    • pregnant women who have taken sickness absence related to their pregnancy

    • unemployed people

  • they covered an intervention that:

    • aimed to prevent the first occurrence of short- or long-term sickness absence (primary prevention)

    • targeted pregnant women exclusively and/or which focused on illnesses associated with pregnancy, during the course of a pregnancy

    • tackled workplace absences which are not reported and/or recorded as sickness absence (for example, maternity leave)

    • was delivered outside the workplace or primary care settings

    • dealt solely with the effectiveness of private health insurance schemes and/or claiming of statutory or occupational sick pay; preventing ill-health retirement ; the provision of clinical diagnosis, treatment for existing conditions (including pharmacological or therapeutic interventions) and management of conditions associated with short- and/or long-term sickness

  • if they described the relationship between health or ill health and short- or long-term absence (that is, correlate studies or non-evaluative studies of an intervention, policy, programme or strategy); descriptive studies of participants' views and experiences and cross-sectional studies (that is, with only one data collection point) were also excluded

  • was a dissertation, thesis, book or book chapter or was a non-English language study.

Studies were included in the one remaining effectiveness and cost-effectiveness review covering incapacity if:

  • they were based in the UK

  • they were delivered in a primary care setting and/or workplace setting and/or planned, designed, delivered, managed or funded in collaboration with primary care providers and/or employers; these interventions, policies, programmes or strategies could be delivered by a number of providers (such as voluntary, private, statutory sectors) and/or in various settings as long as they were fully or co-planned, designed, delivered, managed and/or funded in collaboration with employers and primary care settings

  • the population comprised people over age 16 who were unemployed because of long-term incapacity and receiving incapacity benefit or other similar benefits

  • they reported on work-related outcomes, for example, a return to work (paid/unpaid) or job seeking behaviour

  • they covered an intervention that aimed to help people (over 16) who are unemployed and in receipt of incapacity benefit (or a previous form of incapacity benefit or similar benefit) return to work (paid/unpaid) or prepare for work (paid/unpaid)

  • the study design was RCTs or longitudinal intervention studies (that is, there is at least one follow-up measure after baseline) or RCTs with cost effectiveness, cost consequences, cost-benefit, cost-utility, cost-minimisation or net monetary cost and benefit data.

For the incapacity review, studies were excluded if:

  • they were not based in the UK

  • the population was:

    • people over age 16 in full- or part-time employment, both paid and unpaid

    • people over age 16 not in receipt of incapacity benefit (or a previous version of the benefit)

  • the intervention:

    • was delivered outside a workplace or primary care setting, with no primary care or employer involvement in the planning, design, delivery, management or funding

    • dealt solely with drug treatment; the effectiveness of the incapacity benefit system, private health insurance schemes or statutory or occupational sick pay or preventing ill-health retirement

  • they described the relationship between health or ill health and incapacity (that is, correlates studies or non-evaluative studies of an intervention); descriptive studies of participants' views and experiences and cross-sectional studies (that is, with only one data collection point) were also excluded

  • it was a dissertation, thesis, book, book chapter or was not published in English.

Inclusion and exclusion criteria for each review varied slightly and details are included in the review reports.

Quality appraisal

Included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE technical manual 'Methods for development of NICE public health guidance' (see appendix E). Each study was described by study type and graded (++, +, -) to reflect the risk of potential bias arising from its design and execution.

Study type

  • RCTs (including cluster RCTs).

  • Individual, non-randomised controlled trials, case-control studies, cohort studies, controlled before-and-after (CBA) studies, interrupted time series (ITS) studies.

  • Case reports or case series.

Study quality

++ All or most of the criteria have been fulfilled. Where they have not been fulfilled the conclusions are thought very unlikely to alter.

+ Some criteria have been fulfilled. Those criteria that have not been fulfilled or not adequately described are thought unlikely to alter the conclusions.

- Few or no criteria fulfilled. The conclusions of the study are thought likely or very likely to alter.

The main reasons for studies being assessed as (-) were:

  • lack of details in terms of study design features, such as method recruitment to study or randomisation

  • lack of detail about the study participants

  • lack of details in terms of content of the intervention such as who delivered the intervention, in what setting, what the intervention involved

  • limited statistical data presented on work-related outcomes

  • short follow-up periods for the participants from baseline to post intervention.

The studies were also assessed for their applicability to the UK.

Summarising the evidence and making evidence statements

The review data was summarised in evidence tables (see full reviews).

The findings from the studies included in the reviews were synthesised and used as the basis for a number of evidence statements relating to each key question. The evidence statements were prepared by the external contractors (see appendix A). The statements reflect their judgement of the strength (quantity, type and quality) of evidence and its applicability to the populations and settings in the scope.

Economic analysis

An economic model was constructed to incorporate data from the reviews of effectiveness and cost effectiveness. The results are reported in: 'Modelling the cost effectiveness of interventions, strategies, programmes and policies to reduce the number of employees on sickness absence'.

They are available on the NICE website.

Expert papers

Five expert papers, covering expert patients' programmes; condition management; discrimination in the labour market; regional employability programmes and an evaluation of Camden GP surgery pilot were also produced.

Fieldwork

Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are for practitioners and how feasible it would be to put them into practice. It was conducted with practitioners and commissioners who are involved in managing long-term sickness absence and incapacity services. They included those working in the NHS (particularly primary care services and occupational health professionals), local authorities, community, voluntary and private sectors and workplace representatives and trade unions.

The fieldwork comprised:

  • Six stakeholder workshops involving 120 representatives from the public, private, voluntary and community sectors.

  • Eight focus groups involving 30 professionals from Jobcentre Plus, NHS HR managers and incapacity benefit and sickness absence experts.

  • Telephone interviews with three GPs and one other professional who were unable to attend a focus group.

The fieldwork was commissioned to ensure there was ample geographical coverage. The main issues arising are set out in appendix C under fieldwork findings. The full fieldwork report 'Testing NICE draft guidance: managing long-term sickness absence and incapacity to work' is available online.

How the PDG formulated the recommendations

At its meetings in February, April, May, June, September and October 2008, the PDG considered the evidence of effectiveness and cost effectiveness, expert reports and the economic analysis to determine:

  • whether there was sufficient evidence (in terms of quantity, quality and applicability) to form a judgement

  • whether, on balance, the evidence demonstrates that the intervention is effective or ineffective, or whether it is equivocal

  • where there is an effect, the typical size of effect.

The PDG developed draft recommendations through informal consensus, based on the following criteria:

  • Strength (quality and quantity) of evidence of effectiveness and its applicability to the populations/settings referred to in the scope.

  • Effect size and potential impact on the target population's health.

  • Impact on inequalities in health between different groups of the population.

  • Cost effectiveness (for the NHS and other public sector organisations).

  • Balance of risks and benefits.

  • Ease of implementation and any anticipated changes in practice.

The PDG noted that the effectiveness of some interventions could vary according to the context in which they were delivered. For example, the country in which the study was delivered may well have a different policy and fiscal structure which needs to be considered.

Where evidence was lacking, the PDG also considered whether a recommendation should only be implemented as part of a research programme.

Where possible, recommendations were linked to an evidence statement(s) (see appendix C for details). Where a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation in August 2008. At its meeting in October 2008, the PDG amended the guidance in light of comments from stakeholders and experts, the consultation report and the fieldwork. The guidance was signed off by the NICE Guidance Executive in December 2008.

  • National Institute for Health and Care Excellence (NICE)