factor, a disease, or a preventive or therapeutic intervention. A good
epidemiologic research design should perform the following functions: □ Enable
a ...
B978-1-4557-06587.00005-0 10.1016/B978-1-45570658-7.00005-0 978-1-4557-0658-7 Elsevier Inc.
5
Common Research Designs and Issues in Epidemiology
CHAPTER OUTLINE I. FUNCTIONS OF RESEARCH DESIGN 59 II. TYPES OF RESEARCH DESIGN 59 A. Observational Designs for Generating Hypotheses 60 1. Qualitative Studies 60 2. Cross-Sectional Surveys 60 3. Cross-Sectional Ecological Studies 62 4. Longitudinal Ecological Studies 62 B. Observational Designs for Generating or Testing Hypotheses 63 1. Cohort Studies 63 2. Case-Control Studies 64 3. Nested Case-Control Studies 65 C. Experimental Designs for Testing Hypotheses 65 1. Randomized Controlled Clinical Trials 65 2. Randomized Controlled Field Trials 67 D. Techniques for Data Summary, Cost-Effectiveness Analysis, and Postapproval Surveillance 67 III. RESEARCH ISSUES IN EPIDEMIOLOGY 68 A. Dangers of Data Dredging 68 B. Ethical Issues 69 IV. SUMMARY 69 REVIEW QUESTIONS, ANSWERS, AND EXPLANATIONS
I. FUNCTIONS OF RESEARCH DESIGN Research is the process of answering a question that can be answered by appropriately collected data. The question may simply be, “What is (or was) the frequency of a disease in a certain place at a certain time?” The answer to this question is descriptive, but contrary to a common misperception, this does not mean that obtaining the answer (descriptive research) is a simple task. All research, whether quantitative or qualitative, is descriptive, and no research is better than the quality of the data obtained. To answer a question correctly, the data must be obtained and described appropriately. The rules that govern the process of collecting and arranging the data for analysis are called research designs. Another research question may be, “What caused this disease?” Hypothesis generation is the process of developing a list of possible candidates for the causes of the disease and obtaining initial evidence that supports one or more of these candidates. When one or more hypotheses are generated, the hypothesis must be tested (hypothesis testing) by making predictions from the hypotheses and examining new data to
determine if the predictions are correct (see Chapters 6 and 10). If a hypothesis is not supported, it should be discarded or modified and tested again. Some research designs are appropriate for hypothesis generation, and some are appropriate for hypothesis testing. Some designs can be used for either, depending on the circumstances. No research design is perfect, however, because each has its advantages and disadvantages. The basic function of most epidemiologic research designs is either to describe the pattern of health problems accurately or to enable a fair, unbiased comparison to be made between a group with and a group without a risk factor, a disease, or a preventive or therapeutic intervention. A good epidemiologic research design should perform the following functions: n
Enable a comparison of a variable (e.g., disease frequency) between two or more groups at one point in time or, in some cases, within one group before and after receiving an intervention or being exposed to a risk factor. n Allow the comparison to be quantified in absolute terms (as with a risk difference or rate difference) or in relative terms (as with a relative risk or odds ratio; see Chapter 6). n Permit the investigators to determine when the risk factor and the disease occurred, to determine the temporal sequence. n Minimize biases, confounding, and other problems that would complicate interpretation of the data. The research designs discussed in this chapter are the primary designs used in epidemiology. Depending on design choice, research designs can assist in developing hypotheses, testing hypotheses, or both. All designs can be used to generate hypotheses; and a few designs can be used to test them— with the caveat that hypothesis development and testing of the same hypothesis can never occur in a single study. Randomized clinical trials or randomized field trials are usually the best designs for testing hypotheses when feasible to perform.
II. TYPES OF RESEARCH DESIGN Because some research questions can be answered by more than one type of research design, the choice of design depends on a variety of considerations, including the clinical topic (e.g., whether the disease or condition is rare or common) and the cost and availability of data. Research designs are often described as either observational or experimental. 59
60
S e c t i o n 1 Epidemiology
In observational studies the investigators simply observe groups of study participants to learn about the possible effects of a treatment or risk factor; the assignment of participants to a treatment group or a control group remains outside the investigators’ control. Observational studies can be either descriptive or analytic. In descriptive observational studies, no hypotheses are specified in advance, preexisting data are often used, and associations may or may not be causal. In analytic observational studies, hypotheses are specified in advance, new data are often collected, and differences between groups are measured. In an experimental study design the investigator has more control over the assignment of participants, often placing them in treatment and control groups (e.g., by using a randomization method before the start of any treatment). Each type of research design has advantages and disadvantages, as discussed subsequently and summarized in Table 5-1 and Figure 5-1.
A. Observational Designs for Generating Hypotheses 1. Qualitative Studies Qualitative research involves an investigation of clinical issues by using anthropologic techniques such as
ethnographic observation, open-ended semistructured interviews, focus groups, and key informant interviews. The investigators attempt to listen to the participants without introducing their own bias as they gather data. They then review the results and identify patterns in the data in a structured and sometimes quantitative form. Results from qualitative research are often invaluable for informing and making sense of quantitative results and providing greater insights into clinical questions and public health problems. The two approaches (quantitative and qualitative) are complementary, with qualitative research providing rich, narrative information that tells a story beyond what reductionist statistics alone might reveal.
2. Cross-Sectional Surveys A cross-sectional survey is a survey of a population at a single point in time. Surveys may be performed by trained interviewers in people’s homes, by telephone interviewers using random-digit dialing, or by mailed, e-mailed, or Webbased questionnaires. Telephone surveys or e-mail questionnaires are often the quickest, but they typically have many nonresponders and refusals, and some people do not have telephones or e-mail access, or they may block calls or e-mails even if they do. Mailed surveys are also relatively inexpensive, but they usually have poor response rates, often 50% or less, except in the case of the U.S. Census, where
Table 5-1 Advantages and Disadvantages of Common Types of Studies Used in Epidemiology Studies
Advantages
Disadvantages
Qualitative research
Generates hypotheses and initial exploration of issues in participants’ own language without bias of investigator
Cannot test study hypotheses Can explore only what is presented or stated Has potential for bias
Cross-sectional surveys
Are fairly quick and easy to perform Are useful for hypothesis generation
Do not offer evidence of a temporal relationship between risk factors and disease Are subject to late-look bias Are not good for hypothesis testing
Ecological studies
Are fairly quick and easy to perform Are useful for hypothesis generation
Do not allow for causal conclusions to be drawn because the data are not associated with individual persons Are subject to ecological fallacy Are not good for hypothesis testing
Cohort studies
Can be performed retrospectively or prospectively Can be used to obtain a true (absolute) measure of risk Can study many disease outcomes Are good for studying rare risk factors
Are time-consuming and costly (especially prospective studies) Can study only the risk factors measured at the beginning Can be used only for common diseases May have losses to follow-up
Case-control studies
Are fairly quick and easy to perform Can study many risk factors Are good for studying rare diseases
Can obtain only a relative measure of risk Are subject to recall bias Selection of controls may be difficult Temporal relationships may be unclear Can study only one disease outcome at a time
Randomized controlled trials
Are the “gold standard” for evaluating treatment interventions (clinical trials) or preventive interventions (field trials) Allow investigator to have extensive control over research process
Are time-consuming and usually costly Can study only interventions or exposures that are controlled by investigator May have problems related to therapy changes and dropouts May be limited in generalizability Are often unethical to perform at all
Systematic reviews and meta-analysis
Decrease subjective element of literature review Increase statistical power Allow exploration of subgroups Provide quantitative estimates of effect
Mixing poor quality studies together in a review or metaanalysis does not improve the underlying quality of studies.
Cost-effectiveness analysis
Clinically important
Difficult to identify costs and payments in many health care systems
C h a p t e r 5 Common Research Designs and Issues in Epidemiology
61
Epidemiology studies
Observational studies
Descriptive studies
Ecological studies*
Crosssectional studies
Experimental studies
Casecontrol studies
Cohort studies**
Hypothesis-generating (distribution of disease)
Randomized controlled trials (RCTs) [clinical vs. field/community]
Hypothesis-generating OR testing (cause of disease) Increasing strength of epidemiologic evidence * Longitudinal ecological studies stronger than cross-sectional ecological studies ** Prospective cohort studies stronger than retrospective
Figure 5-1 Epidemiologic study designs and increasing strength of evidence.
response is required by law, and follow-up of all nonresponders is standard. Cross-sectional surveys have the advantage of being fairly quick and easy to perform. They are useful for determining the prevalence of risk factors and the frequency of prevalent cases of certain diseases for a defined population. They also are useful for measuring current health status and planning for some health services, including setting priorities for disease control. Many surveys have been undertaken to determine the knowledge, attitudes, and health practices of various populations, with the resulting data increasingly being made available to the general public (e.g., healthyamericans.org). A major disadvantage of using crosssectional surveys is that data on the exposure to risk factors and the presence or absence of disease are collected simultaneously, creating difficulties in determining the temporal relationship of a presumed cause and effect. Another disadvantage is that cross-sectional surveys are biased in favor of longer-lasting and more indolent (mild) cases of diseases. Such cases are more likely to be found by a survey because people live longer with mild cases, enabling larger numbers of affected people to survive and to be interviewed. Severe diseases that tend to be rapidly fatal are less likely to be found by a survey. This phenomenon is often called Neyman bias or late-look bias. It is known as length bias in screening programs, which tend to find (and select for) less aggressive illnesses because patients are more likely to be found by screening (see Chapter 16). Repeated cross-sectional surveys may be used to determine changes in risk factors and disease frequency in populations over time (but not the nature of the association between risk factors and diseases). Although the data derived from these surveys can be examined for such associations in order
to generate hypotheses, cross-sectional surveys are not appropriate for testing the effectiveness of interventions. In such surveys, investigators might find that participants who reported immunization against a disease had fewer cases of the disease. The investigators would not know, however, whether this finding actually meant that people who sought immunization were more concerned about their health and less likely to expose themselves to the disease, known as healthy participant bias. If the investigators randomized the participants into two groups, as in a randomized clinical trial, and immunized only one of the groups, this would exclude self-selection as a possible explanation for the association. Cross-sectional surveys are of particular value in infectious disease epidemiology, in which the prevalence of antibodies against infectious agents, when analyzed according to age or other variables, may provide evidence about when and in whom an infection has occurred. Proof of a recent acute infection can be obtained by two serum samples separated by a short interval. The first samples, the acute sera, are collected soon after symptoms appear. The second samples, the convalescent sera, are collected 10 to 28 days later. A significant increase in the serum titer of antibodies to a particular infectious agent is regarded as proof of recent infection. Even if two serum samples are not taken, important inferences can often be drawn on the basis of titers of IgG and IgM, two immunoglobulin classes, in a single serum sample. A high IgG titer without an IgM titer of antibody to a particular infectious agent suggests that the study participant has been infected, but the infection occurred in the distant past. A high IgM titer with a low IgG titer suggests a current or very recent infection. An elevated IgM titer in the presence of a high IgG titer suggests that the infection occurred fairly recently.
62
S e c t i o n 1 Epidemiology teeth has been followed by striking reductions in the rate of dental caries.2
3. Cross-Sectional Ecological Studies Cross-sectional ecological studies relate the frequency with which some characteristic (e.g., smoking) and some outcome of interest (e.g., lung cancer) occur in the same geographic area (e.g., a city, state, or country). In contrast to all other epidemiologic studies, the unit of analysis in ecological studies is populations, not individuals. These studies are often useful for suggesting hypotheses but cannot be used to draw causal conclusions. Ecological studies provide no information as to whether the people who were exposed to the characteristic were the same people who developed the disease, whether the exposure or the onset of disease came first, or whether there are other explanations for the observed association. Concerned citizens are sometimes unaware of these weaknesses (sometimes called the ecological fallacy) and use findings from cross-sectional ecological surveys to make such statements as, “There are high levels of both toxic pollution and cancer in northern New Jersey, so the toxins are causing the cancer.” Although superficially plausible, this conclusion may or may not be correct. For example, what if the individuals in the population who are exposed to the toxins are universally the people not developing cancer? Therefore the toxic pollutants would be exerting a protective effect for individuals despite the ecological evidence that may suggest the opposite conclusion. In many cases, nevertheless, important hypotheses initially suggested by cross-sectional ecological studies were later supported by other types of studies. The rate of dental caries in children was found to be much higher in areas with low levels of natural fluoridation in the water than in areas with high levels of natural fluoridation.1 Subsequent research established that this association was causal, and the introduction of water fluoridation and fluoride treatment of
4. Longitudinal Ecological Studies Longitudinal ecological studies use ongoing surveillance or frequent repeated cross-sectional survey data to measure trends in disease rates over many years in a defined population. By comparing the trends in disease rates with other changes in the society (e.g., wars, immigration, introduction of a vaccine or antibiotics), epidemiologists attempt to determine the impact of these changes on disease rates. For example, the introduction of the polio vaccine resulted in a precipitous decrease in the rate of paralytic poliomyelitis in the U.S. population (see Chapter 3 and Fig. 3-9). In this case, because of the large number of people involved in the immunization program and the relatively slow rate of change for other factors in the population, longitudinal ecological studies were useful for determining the impact of this public health intervention. Nevertheless, confounding with other factors can distort the conclusions drawn from ecological studies, so if time is available (i.e., it is not an epidemic situation), investigators should perform field studies, such as randomized controlled field trials (see section II.C.2), before pursuing a new, large-scale public health intervention. Another example of longitudinal ecological research is the study of rates of malaria in the U.S. population since 1930. As shown in Figure 5-2, the peaks in malaria rates can be readily related to social events, such as wars and immigration. The use of a logarithmic scale in the figure visually minimizes the relative decrease in disease frequency, making it less impressive to the eye, but this scale enables readers to see in detail the changes occurring when rates are low.
1000
Reported cases per 100,000 population
Relapses—overseas cases 100 Relapses from Korean War veterans 10
Returning Vietnam War veterans Foreign immigration
1
0.1
0.01 1930 1935 1940 1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 Year
Figure 5-2 Incidence rates of malaria in the United States, by year of report, 1930-1992. (From Centers for Disease Control and Prevention: Summary of notifiable diseases, United States, 1992. MMWR 41:38, 1992.)
C h a p t e r 5 Common Research Designs and Issues in Epidemiology
Important causal associations have been suggested by longitudinal ecological studies. About 20 years after an increase in the smoking rates in men, the lung cancer rate in the male population began increasing rapidly. Similarly, about 20 years after women began to smoke in large numbers, the lung cancer rate in the female population began to increase. The studies in this example were longitudinal ecological studies in the sense that they used only national data on smoking and lung cancer rates, which did not relate the individual cases of lung cancer to individual smokers. The task of establishing a causal relationship was left to cohort and case-control studies.
63
PROSPECTIVE COHORT STUDIES In a prospective cohort study, the investigator assembles the study groups in the present, collects baseline data on them, and continues to collect data for a period that can last many years. Prospective cohort studies offer three main advantages, as follows:
B. Observational Designs for Generating or Testing Hypotheses 1. Cohort Studies
1. The investigator can control and standardize data collection as the study progresses and can check the outcome events (e.g., diseases and death) carefully when these occur, ensuring the outcomes are correctly classified. 2. The estimates of risk obtained from prospective cohort studies represent true (absolute) risks for the groups studied. 3. Many different disease outcomes can be studied, including some that were not anticipated at the beginning of the study.
A cohort is a clearly identified group of people to be studied. In cohort studies, investigators begin by assembling one or more cohorts, either by choosing persons specifically because they were or were not exposed to one or more risk factors of interest, or by taking a random sample of a given population. Participants are assessed to determine whether or not they develop the diseases of interest, and whether the risk factors predict the diseases that occur. The defining characteristic of cohort studies is that groups are typically defined on the basis of exposure and are followed for outcomes. This is in contrast to case-control studies (see section II.B.2), in which groups are assembled on the basis of outcome status and are queried for exposure status. There are two general types of cohort study, prospective and retrospective; Figure 5-3 shows the time relationships of these two types.
However, any disease outcomes that were not preplanned— or supported by evidence that was available a priori (before start of the study)—would be hypothesis generating only. Sometimes studies have secondary outcomes that are determined a priori, but for which the study is not adequately powered (see Chapter 12) and thus can only be hypothesis generating. Cohort studies also have disadvantages. In such studies, only the risk factors defined and measured at the beginning of the study can be used. Other disadvantages of cohort studies are their high costs, the possible loss of study participants to follow-up, and the long wait until results are obtained. The classic cohort study is the Framingham Heart Study, initiated in 1950 and continuing today.3 Table 5-2 shows the
The passage of time Type of Study
The Past
Prospective (longitudinal) cohort study
The Present Assemble cohorts in the present and collect data on present risk factors (present exposures)
Retrospective cohort study
When exposures occur defines cohorts
Assemble cohorts in the present based on past risk factors (past exposures), and collect data on present outcomes
Case-control study
When exposures occur that may be associated with outcomes
Assemble cases and controls in the present based on present outcomes, and collect data on past risk factors (past exposures)
Cross-sectional study
The Future Collect data at a time in the future on outcomes that arise
Associations between present exposures and present outcomes (both occurring at a single point in time)
Figure 5-3 Relationship between time of assembling study participants and time of data collection. Illustration shows prospective cohort study, retrospective cohort study, case-control study, and cross-sectional study.
64
S e c t i o n 1 Epidemiology The individuals who had been exposed to x-rays in utero had a 40% increase in the risk of childhood cancers, or a risk ratio of 1.4, after adjustments for other factors.
Table 5-2 Risk that 45-year-old Man Will Have
Cardiovascular Disease within 8 Years
Risk Group Lowest
Highest Intermediate
Characteristics of Risk Group All the following factors: Nonsmoker No glucose intolerance No hypertrophy of left ventricle Low systolic blood pressure (≤105 mm Hg) Low cholesterol level (≤185 mg/dL) All the factors listed below One of the following factors: Smoker Glucose intolerance Hypertrophy of left ventricle Severe hypertension (systolic blood pressure ≥195 mm Hg) High cholesterol level (≥335 mg/dL)
Risk (%)
Ratio
2.2
—
77.8
35.4
3.8 3.9 6
1.7 1.8 2.7
8.4
3.8
8.5
3.8
Data from Pearson T, Becker D: Cardiovascular risk: computer program for IBM-compatible systems, using the Framingham Study 8-year risk equations, Johns Hopkins University; and Breslow L: Science 200:908–912, 1978. http://hp2010.nhlbihin.net/atpiii/calculator.asp
8-year risk of heart disease as calculated from the Framingham Study’s equations.4 Although these risk ratios are not based on the most recent study data, the length of follow-up and the clarity of the message still make them useful for sharing with patients. Examples of other, more recent, large cohort studies are the Nurses’ Health Study, begun in 1976 and continuing to track more than 120,000 nurses in the United States (www.nhs3.org), and the National Child Development Study, initiated after the Second World War and continuing to follow a large birth cohort in the United Kingdom.5
RETROSPECTIVE COHORT STUDIES The time and cost limitations of prospective cohort studies can be mitigated in part by conducting retrospective cohort studies. In this approach the investigator uses historical data to define a risk group (e.g., people exposed to the Hiroshima atomic bomb in August 1945) and follows group members up to the present to see what outcomes (e.g., cancer and death) have occurred. This type of study has many of the advantages of a prospective cohort study, including the ability to calculate an absolute risk. However, it lacks the ability to monitor and control data collection that characterizes a prospective cohort study. A retrospective cohort study in 1962 investigated the effects of prenatal x-ray exposure.6 In prior decades, radiographs were often used to measure the size of the pelvic outlet of pregnant women, thus exposing fetuses to x-rays in utero. The investigators identified one group of participants who had been exposed in utero and another group who had not. They determined how many participants from each group had developed cancer during childhood or early adulthood (up to the time of data collection).
2. Case-Control Studies The investigator in a case-control study selects the case group and the control group on the basis of a defined outcome (e.g., having a disease of interest versus not having a disease of interest) and compares the groups in terms of their frequency of past exposure to possible risk factors (see Fig. 5-3). This strategy can be understood as comparing “the risk of having the risk factor” in the two groups. However, the actual risk of the outcome cannot be determined from such studies because the underlying population remains unknown. Instead, case-control studies can estimate the relative risk of the outcome, known as the odds ratio. In the case-control study the cases and controls are assembled and then questioned (or their relatives or medical records are consulted) regarding past exposure to risk factors. For this reason, case-control studies were often called “retrospective studies” in previous decades; this term does not distinguish them from retrospective cohort studies and thus is no longer preferred. The time relationships in a casecontrol study are similar to those in a cross-sectional study in that investigators learn simultaneously about the current disease state and any past risk factors. In terms of assembling the participants, however, a case-control study differs from a cross-sectional study because the sample for the case-control study is chosen specifically from groups with and without the disease of interest. Often, everyone with the disease of interest in a given geographic area and time period can be selected as cases. This strategy reduces bias in case selection. Case-control studies are especially useful when a study must be performed quickly and inexpensively or when the disease under study is rare (e.g., prevalence
