Appendix E: Methodology checklist: case–control studies

Appendix E: Methodology checklist: case–control studies

Checklist

Study identification

Include author, title, reference, year of publication

Guideline topic:

Review question no:

Checklist completed by:

Section 1: Internal validity

Circle or highlight one option for each question

1.1

The study addresses an appropriate and clearly focused question.

Well covered

Adequately addressed

Poorly addressed

Not addressed

Not reported

Not applicable

Selection of participants

1.2

The cases and controls are taken from comparable populations

Well covered

Adequately addressed

Poorly addressed

Not addressed

Not reported

Not applicable

1.3

The same exclusion criteria are used for both cases and controls

Well covered

Adequately addressed

Poorly addressed

Not addressed

Not reported

Not applicable

1.4

What was the participation rate for each group (cases and controls)?

Cases:

Controls:

1.5

Participants and non-participants are compared to establish their similarities or differences

Well covered

Adequately addressed

Poorly addressed

Not addressed

Not reported

Not applicable

1.6

Cases are clearly defined and differentiated from controls

Well covered

Adequately addressed

Poorly addressed

Not addressed

Not reported

Not applicable

1.7

It is clearly established that controls are not cases

Well covered

Adequately addressed

Poorly addressed

Not addressed

Not reported

Not applicable

Assessment

1.8

Measures were taken to prevent knowledge of primary exposure from influencing case ascertainment

Well covered

Adequately addressed

Poorly addressed

Not addressed

Not reported

Not applicable

1.9

Exposure status is measured in a standard, valid and reliable way

Well covered

Adequately addressed

Poorly addressed

Not addressed

Not reported

Not applicable

Confounding factors

1.10

The main potential confounders are identified and taken into account in the design and analysis

Well covered

Adequately addressed

Poorly addressed

Not addressed

Not reported

Not applicable

Statistical analysis

1.11

Have confidence intervals been provided?

Section 2: Description of the study

(This information is required for evidence tables to facilitate cross-study comparisons. Please complete all sections for which information is available.)

Please print clearly

2.1

How many people participated in the study?

List the numbers of cases and controls separately.

.

.

.

.

2.2

What are the main characteristics of the study population?

Include all characteristics used to identify both cases and controls – for example, age, sex, social class, disease status.

.

.

.

.

2.3

What environmental or prognostic factor is being investigated?

.

.

.

.

2.4

What comparisons are made?

Normally only one factor will be compared, but in some cases the extent of exposure may be stratified – for example, non-smokers vs light, moderate or heavy smokers. Note all comparisons here.

.

.

.

.

2.5

For how long are participants followed up?

This is the length of time over which participant histories are tracked in the study.

.

.

.

.

2.6

What outcome measure(s) is/are used?

List all outcomes that are used to assess the impact of the chosen environmental or prognostic factor.

.

.

.

.

.

2.7

What size of effect is identified?

Effect size should be expressed as an odds ratio. If any other measures are included, note them as well. Include p-values and any confidence intervals that are provided.

.

.

.

.

2.8

How was the study funded?

List all sources of funding quoted in the article, whether government, voluntary sector or industry.

.

.

.

.

2.9

Does this study help to answer your guideline review question?

Summarise the main conclusions of the study and indicate how it relates to the review question.

.

.

.

.

Notes on use of the Methodology checklist: case–control studies

Case–control studies are designed to answer questions of the type 'What are the factors that caused this event?'. They involve comparison of individuals who have an outcome with other individuals from the same population who do not have the outcome. These studies start after the outcome of an event, and can be used to assess multiple causes of a single event. They are generally used to assess the causes of a new problem but they may also be useful for the evaluation of population-based interventions such as screening.

The questions in section 1 are aimed at establishing the internal validity of the study under review – that is, making sure that it has been carried out carefully, and that any link between events and outcomes is clearly established. Each question covers an aspect of methodology that has been shown to make a significant difference to the conclusions of a study.

Case–control studies need to be designed very carefully – the complexity of their design is often not appreciated by investigators, and so many poor-quality studies are conducted. The questions in this checklist are designed to identify the main features that should be present in a well-designed study. There are few criteria that should, alone and unsupported, lead to rejection of a study. However, a study that fails to address or report on more than one or two of the questions in the checklist should almost certainly be rejected.

For each question in this section you should choose one of the following categories to indicate how well it has been addressed in the study:

  • well covered

  • adequately addressed

  • poorly addressed

  • not addressed (not mentioned, or this aspect of study design was ignored)

  • not reported (mentioned, but with insufficient detail to allow assessment to be made)

  • not applicable.

Question

1.1 The study addresses an appropriate and clearly focused question

Unless a clear and well-defined question is specified, it will be difficult to assess how well the study has met its objectives or how relevant it is to the question you are trying to answer.

Selection of participants

1.2 The cases and controls are taken from comparable populations

Study participants may be selected from the target population (all individuals to which the results of the study could be applied), from the source population (a defined subset of the target population from which participants are selected) or from a pool of eligible people (a clearly defined and counted group selected from the source population). A study that does not include clear definitions of the source population should be rejected.

1.3 The same exclusion criteria are used for both cases and controls

All selection and exclusion criteria should be applied equally to cases and controls. Failure to do so may introduce a significant degree of bias into the results of the study.

1.4 What was the participation rate for each group (cases and controls)?

Differences between the eligible population and the study participants are important because they may influence the validity of the study. A participation rate can be calculated by dividing the number of study participants by the number of people who are eligible to participate. It is more useful if it is calculated separately for cases and controls. If the participation rate is low, or there is a large difference in rate between cases and controls, the study results may be invalid because of differences between participants and non-participants. In these circumstances the study should be downgraded, and rejected if the differences are very large.

1.5 Participants and non-participants are compared to establish their similarities or differences

Even if participation rates are comparable and acceptable, it is still possible that the participants selected to act as cases or controls may differ from other members of the source population in some significant way. A well-conducted case–control study will look at samples of those not participating among the source population to ensure that the participants are a truly representative sample.

1.6 Cases are clearly defined and differentiated from controls

The method of selection of cases is of critical importance to the validity of the study. Investigators have to be certain that cases are truly cases, but must balance this with the need to ensure that the cases admitted into the study are representative of the eligible population. The issues involved in case selection are complex, and should ideally be evaluated by someone with a good understanding of the design of case–control studies. If there is no information on how cases were selected it is probably safest to reject the study as a source of evidence.

1.7 It is clearly established that controls are not cases

Just as it is important to be sure that cases are true cases, it is important to be sure that controls do not have the outcome under investigation. Controls should be chosen so that information on exposure status can be obtained or assessed in a similar way to that used for the selection of cases. If the methods of control selection are not described, the study should be rejected. If different methods of selection are used for cases and controls, the study should be evaluated by someone with a good understanding of the design of case–control studies.

Assessment

1.8 Measures were taken to prevent knowledge of primary exposure from influencing case ascertainment

If there is a possibility that case ascertainment was influenced by knowledge of exposure status, assessment of any association is likely to be biased. A well-conducted study should take this into account in the design of the study.

1.9 Exposure status is measured in a standard, valid and reliable way

The inclusion of evidence from other sources or previous studies that demonstrate the validity and reliability of the assessment methods, or the fact that the measurement method is a recognised procedure, should increase confidence in study quality.

Confounding factors

1.10 The main potential confounders are identified and taken into account in the design and analysis

Confounding is the distortion of a link between exposure and outcome by another factor that is associated with both exposure and outcome. The possible presence of confounding factors is one of the principal reasons why observational studies are not more highly rated as a source of evidence. The report of the study should indicate which potential confounders have been considered, and how they have been assessed or accounted for in the analysis. Clinical judgement should be used to consider whether all likely confounders have been taken into account. If the measures used to address the potential effects of confounders are considered inadequate, the study should be downgraded or rejected, depending on how serious the risk of confounding is considered to be. A study that does not address the possibility of confounding should be rejected.

Statistical analysis

1.11 Have confidence intervals been provided?

Confidence intervals are the preferred method for indicating the precision of statistical results, and can be used to differentiate between an inconclusive study and a study that shows no effect. Studies that report a single value with no assessment of precision should be treated with caution.

Section 2 of the checklist asks you to summarise key points about the study that will be added to an evidence table (see appendix J) in the next stage of the process.


This page was last updated: 30 November 2012