Appendix B: Summary of the methods used to develop this guidance

Introduction

The reviews and economic analysis include full details of the methods used to select the evidence (including search strategies), assess its quality and summarise it.

The minutes of the PHIAC meetings provide further detail about the Committee's interpretation of the evidence and development of the recommendations.

All supporting documents are listed in appendix E and are available online.

The methods adopted for the development of this guidance on promoting mental wellbeing through productive and healthy working conditions are in line with the recently revised 'Methods for the development of NICE public health guidance' (2009).

The review of the evidence involved three phases, specifically:

  • Systematic review of the evidence of intervention evaluation studies (March 2008).

  • Use of a conceptual model and thematic review of a diverse range of evidence (identified by an expert reference group) to test the model and identify interventions options (July 2008).

  • Consideration of the findings of this approach as the basis for development of guidance (January and February 2009).

Guidance development

The stages involved in developing this public health guidance are outlined in the box below.

  1. Draft scope released for consultation

  2. Stakeholder meeting about the draft scope

  3. Stakeholder comments used to revise the scope

  4. Final scope and responses to comments published on website

  5. Evidence review(s) and economic analysis undertaken:

    • Phase 1: systematic review of effectiveness of interventions and economic analysis

    • Phase 2: conceptual framework and thematic review of a range of different types of evidence

    • Phase 3: draft guidance and recommendations formulated

  6. Evidence and economic analysis consultation

  7. Draft guidance released for consultation and for field testing

  8. PHIAC amends recommendations

  9. Final guidance published on website

  10. Responses to comments published on the website

Key questions

The key questions were established as part of the scope. They formed the starting point for the reviews of evidence and were used by PHIAC to help develop the recommendations. The overarching question was:

  • How can work and working conditions be used to promote mental wellbeing?

The subsidiary questions were:

  1. Which interventions are most effective and cost effective

  2. What specific characteristics of work and working conditions promote mental wellbeing effectively and cost effectively?

  3. How can organisations support employees who are coping with stress, anxiety and depression caused by external factors (for example, bereavement, family breakdown or debt)?

  4. How can healthy working conditions be created for different occupational groups and in different organisational contexts?

  5. What help do employers need to review and adapt working practices and conditions to promote the mental wellbeing of employees?

  6. What are the barriers and facilitators to the implementation of interventions to promote mental wellbeing in the workplace – for both employers and employees?

  7. Do interventions that promote health equalities also have an impact on mental wellbeing and productivity?

  8. How can the promotion of mental wellbeing at work improve both working conditions and productivity?

  9. What are the costs and economic benefits to employers: what is the business case for promoting employees' mental wellbeing?

These questions were made more specific for the reviews (see reviews for further details).

Reviewing the evidence

Phase one: systematic review of effectiveness of interventions and economic analysis

A review of effectiveness of specified interventions aimed at promoting mental wellbeing in the workplace was conducted (Review 1). This review focused on intervention evaluation studies on organisation-wide policies and approaches concerned with promoting mental wellbeing through work and working conditions.

The review followed the NICE methods for systematic review of evidence. Nineteen databases and 24 websites were searched for intervention studies and reviews. Studies were included if a specific workplace intervention had been carried out and validated outcome measures had been used. Studies were excluded if interventions focused on diagnosed mental health conditions that require pharmacological and/or psychosocial treatment.

Sixty-six primary studies met the inclusion criteria. These covered a range of organisation-wide or stress-management interventions. The included papers were assessed for methodological rigour and quality using the NICE methodology checklist, as set out in the NICE manual 'Methods for the development of NICE public health guidance' (see appendix E). The review data was summarised in evidence tables. The findings from the review were synthesised and used as the basis for a number of evidence statements relating to each key question (see full review).

Overall the review showed comparatively limited evidence on organisation-wide policies and approaches concerned with promoting mental wellbeing through work and working conditions. The strongest (although limited in number) interventions were individual interventions aimed at stress management.

In addition, this phase involved a review of economic evaluations and a cost-effectiveness analysis. An economic model was constructed to incorporate data from the review of effectiveness.

The economic databases EconLit and Health Economics Evaluation Database and the NHS Economic Evaluation Database were searched, and the effectiveness evidence (Review 1) was also scanned. Searches were conducted for full economic evaluations of workplace-based interventions that promote mental wellbeing in working adults. Of the 50 papers obtained only two were considered for full review after applying ex/inclusion criteria.

PHIAC judged that there was insufficient evidence on organisation-wide approaches (quality of work and working conditions) to make recommendations based on this initial work. However, PHIAC considered that the role of work on employees' mental wellbeing remained an important topic for NICE guidance. It therefore requested that the review of evidence should be extended to include different types of studies and literatures.

Phase two: conceptual framework and thematic review

PHIAC discussed and agreed a new approach for development of this guidance.

The development of all public health guidance is to be informed by the 'conceptual framework' set out in the Centre for Public Health Evidence revised methods manual (2009). This framework was applied to workplace mental wellbeing, to be tested and developed through review of an extended range of different types of evidence. The framework identified a range of factors that operate through population-wide institutional structures and systems, environmental agents, sociocultural mechanisms and the 'work organisational setting' to affect the mental wellbeing of employees.

A more detailed model attempted to conceptualise the main components of a healthy work organisation. It identified those work characteristics that could in principle enhance mental wellbeing and those that pose risks (act as 'stressors') to mental wellbeing. These pathways present intervention opportunities for promoting mental wellbeing.

PHIAC discussed and agreed this model at its July 2008 meeting as the basis for review of evidence.

An expert reference group (comprising a number of academics and specialists in this field) was established to provide advice. This group supported the identification of relevant evidence relating to the research questions (specific studies, references and sources) and the NICE technical team created a database of this material.

The School of Health and Related Research (ScHARR) at the University of Sheffield was commissioned to undertake a review of this evidence. Use of the 'traditional' systematic review process was not appropriate, because of the diverse nature of the evidence. A thematic review of evidence was conducted to achieve greater understanding of the work characteristics that could enhance or harm mental wellbeing, and how these factors may interact. It aimed to test and refine the conceptual model. The document set contained diverse literature including policy documents, reports and empirical work with varied study designs.

Issues relating to the economic and business case for wellbeing were part of the review process, in particular the review incorporated evidence from the Foresight report (Foresight Mental Capital Wellbeing Project 2008).

Techniques from qualitative data analysis were employed to examine the data, identify the main themes and organise these in a theoretical explanatory scheme. The review highlighted the characteristics that are associated with productive and healthy organisations. More specifically it defined the 'theoretical' pathways (causal mechanisms) between work and mental wellbeing.

Fieldwork

Fieldwork was carried out to evaluate how relevant and useful NICE's recommendations are for employers and how feasible it would be to put them into practice. It was conducted with employers and employer and employee representatives who have responsibilities for the health of employees, and also health professionals and providers of health promotion services. They included those working in public, private and voluntary sectors.

The fieldwork comprised:

  • eight workshops with employers, representatives of employers and employees, and health professionals and providers of health promotion services, held in Birmingham, London, and Manchester

  • forty telephone interviews primarily with those working in human resources or occupational health.

The workshops and telephone interviews were commissioned to ensure there was ample geographical coverage. The main issues arising from them are set out in appendix C under fieldwork findings. The full fieldwork report 'Consultation on NICE draft recommendations on the promotion of mental wellbeing in the workplace' is available online.

How PHIAC formulated its recommendations

At its meetings in January and February 2009 PHIAC considered the review of this evidence as the basis for development of recommendations. Members of the expert reference group and co-optees supported this process.

PHIAC developed draft recommendations through informal consensus.

PHIAC considered the extent to which the review had identified and provided evidence of conceptual plausibility of the associations between characteristics of work and mental wellbeing of employees and business outcomes. These represented potential stressors, and therefore areas for interventions.

The robustness of these associations was assessed using 'triangulation' methods. Supplementary work was undertaken by ScHARR to describe the range of different types of evidence reporting the associations. In addition the findings of other recent review exercises were considered; specifically:

  • 'A business case for the management standards for stress: conclusions based on meta analyses' (prepared by Goldsmith College, University of London), Health and Safety Executive (2006).

  • 'Final project report' Foresight Mental Capital and Wellbeing Project (2008).

  • 'Dame Carol Black's review of the health of Britain's working age population. Working for a healthier tomorrow' Health, Work and Wellbeing Programme (2008).

The findings of the review were judged to be consistent with other recent systematic review exercises.

Where possible, the recommendations were linked to the evidence relating to work context and work content stressors (see appendix C for details). If a recommendation was inferred from the evidence, this was indicated by the reference 'IDE' (inference derived from the evidence).

The draft guidance, including the recommendations, was released for consultation April 2009. At its meeting in June 2009, PHIAC amended the guidance in light of comments from stakeholders, experts and the fieldwork. The guidance was signed off by the NICE Guidance Executive in September 2009.

  • National Institute for Health and Care Excellence (NICE)