must vote by mail are more or less responsive to a face-to-face mobilization message than voters who live in traditional precincts with polling places.
Get Out The Vote‐by‐Mail?
A Randomized Field Experiment Testing the Effect of Mobilization in Traditional and Vote‐by‐Mail Precincts Kevin Arceneaux, Temple University Thad Kousser, University of California, San Diego Megan Mullin, Temple University Abstract: Does the effectiveness of a Get Out The Vote (GOTV) contact depend upon the method by which a voter casts a ballot? This study investigates whether those who must vote by mail are more or less responsive to a face‐to‐face mobilization message than voters who live in traditional precincts with polling places. We combine a natural experiment designed to isolate the effects of voting by mail with a field experiment probing the impact of a door‐to‐door GOTV drive. Implementing this design in the November, 2008 general election in San Diego County, we hired professional canvassers to target a total of 29,717 randomly‐assigned registered voters. We find that a face‐to‐ face GOTV contact has a larger effect on the participation of those who vote at polling places than it does on registrants assigned to vote by mail, but only among individuals whose voting behavior is most likely to be shaped by extrinsic social rewards. Prepared for presentation at the Annual Meeting of the American Political Science Association, Toronto, Canada, September 1‐6, 2009. The authors would like to thank the JEHT Foundation for support of this research, San Diego County Registrar Deborah Seiler and her staff for their generous cooperation, Bob Glaser and the La Jolla Group for their canvassing efforts, and Jincheng Sun and Justin Levitt for data entry.
1
Does the effectiveness of a Get Out The Vote (GOTV) contact depend upon the method by which a voter casts a ballot? This study investigates whether those who must vote by mail are more responsive to a face‐to‐face mobilization message than voters who live in traditional precincts with polling places. Following pioneering work by Gerber and Green (2000, 2004), a series of field experiments has shown that the effectiveness of nonpartisan GOTV campaigns depends upon the medium of communication (telephone, mail, or door‐to‐door canvassing) as well as the message that is delivered (Gerber and Green 2000; Green, Gerber, and Nickerson 2003; Gerber, Green, and Shachar 2003; Nickerson 2007; Gerber, Green, and Larimer 2008). Our research asks instead whether the impact of a turnout drive varies according to the way that elections are conducted.
In order to test for this interaction between treatment and context, we take advantage of a
natural experiment that is conducted in every California election. Developed by Kousser and Mullin (2007) and fruitfully applied by Meredith and Malhotra (2008) and Bergman (2009), this research design compares voter behavior in traditional precincts to behavior in precincts which county registrars have assigned to vote entirely by mail. Because individuals do not self‐select into a precinct type, voters in the two types of precincts are likely to be similar in their demographic characteristics and propensity to turn out. Consequently, this design allows us to compare the behavior of similar types of voters taking part in the same election, while only their method of voting differs. To gauge the effects of door‐to‐ door canvassing , we layer a field experiment over this natural experiment, randomly assigning both traditional and vote‐by‐mail precincts to be walked by paid GOTV canvassers. We use this hybrid of a randomized and a natural experiment to study whether the method of voting shapes the effectiveness of door‐to‐door GOTV campaigns. Vote‐by‐mail (VBM) advocates tend to emphasize the ways in which mail balloting reduces the costs of voting (Karp and Banducci 2000, 2001; Gronke et al. 2008). Yet along with reducing costs, VBM removes a potential benefit of voting by making the act less socially visible. Recent work demonstrates the importance of social accountability in
2
boosting voter participation (Gerber, Green, and Larimer 2008). If GOTV canvassing stimulates turnout, in part, by keying people into the social rewards of voting, it may be less effective in areas where people must vote by mail. We also apply Arceneaux and Nickerson’s (2009a) theory of contingent mobilization to investigate whether an individual’s propensity to vote moderates the interaction between GOTV canvassing and voting method. In addition to contributing to the burgeoning literature on the effects of GOTV canvassing on voter turnout, our findings also address a debate about the wisdom of moving to an all‐VBM system. If VBM systems tend to depress turnout in high‐stimulus elections, as Kousser and Mullin (2007) find, then our evidence suggests that typical GOTV methods may not fully overcome the decrease in turnout – at least among individuals whose decision to vote is mostly motivated by attaining social rewards.
I. Previous Research on Mail Balloting Mail ballot elections occur without polling places; instead, election officials send ballots to all registrants and require the ballots to be returned by mail (or to specified drop locations) over a defined period. Proponents of mail elections contend that the convenience of voting from home will produce higher turnout rates (Bradbury 2005; Rosenfield 1995; Davis 2005), but this expectation relies in part on problematic extrapolations from the academic literature on voter participation. One basis for predictions about heightened participation is the high turnout rate among those who request absentee ballots. Studies of people who choose to vote absentee (Patterson and Caldeira 1985; Oliver 1996; Dubin and Kalsow 1996; Karp and Banducci 2001) should not be used to predict participation levels in mail ballot elections, however, because the absentee research examines a subgroup of registrants who are especially likely to turn out. Voluntary absentee voters tend to be older and better educated than other registrants (Barreto et al. 2006) and more politically active (Oliver 1996; Karp and Banducci 2001). The same holds true for those who take advantage of opportunities to vote early (Stein 1998). Simply comparing the behavior of absentee and early voters to that of precinct
3
voters may be misleading, because many of the characteristics that predict absentee and early voting are qualities that make an individual more likely to participate regardless of the voting process. A second basis for the optimism about mail balloting’s turnout effects is the body of research on VBM elections in Oregon. Studies have estimated that the introduction of mail balloting produced an increase in turnout ranging from 6 to 10 percentage points (Berinsky, Burns, and Traugott 2001; Southwell and Burchett 2000), with the strongest effects in low salience elections for ballot measures and local contests (Karp and Banducci 2000). The strength of the Oregon studies for making inferences about the effects of VBM elections is that they eliminate the selection problem inherent to analyses of absentee voting, allowing analysts to separate the act of voting at home from the characteristics of people who choose to do so. They fairly and accurately report the changes in voter participation that have occurred in Oregon. Yet the problem with using these findings to predict the effects of a shift to mail ballot elections elsewhere is that the Oregon studies do not hold constant the political context of elections and the ways in which elections are administered. It is possible that unique features of Oregon’s recent statewide elections account for the turnout effects that researchers have uncovered. The 1995 Senate primary and 1996 Senate general elections that initiated Oregon’s experiment with voting by mail featured a tight race between Portland Congressman Ron Wyden and multimillionaire Gordon Smith for what had been Senator Robert Packwood’s safe seat. The level of competition in this contest, the spending that it generated, and the intense statewide and national attention that it received could account for the fact that the mail ballot primary and general elections delivered higher turnout than previous special elections held at the polls. Moreover, Oregon election administration officials have adopted two policies which could explain the state’s heightened turnout, but which other jurisdictions may not wish to adopt. The first is to provide campaigns with continually updated lists of those who have voted so that campaigns can target their GOTV efforts most effectively. Because other jurisdictions may not have the capacity or
4
willingness to implement this policy, it is important not to combine its effects with the impact of voting by mail. Another potentially conflating factor is the new administrative practice to purge those who do not have a valid postal address from the registration rolls. This practice, which generated much controversy in Oregon because of its disproportionate impact on low‐income and homeless voters (Suo 2000), increases the turnout proportion not by adding to the numerator but by subtracting from the denominator. An ideal research design would solve these problems by randomly assigning a group of registrants to vote by mail, while others just like them are able to visit a polling place or request an absentee ballot, just like in most elections. A natural experiment quite close to this takes place during every election in California, when county election officials assign registrants in less populous precincts to vote by mail. These small mail ballot precincts, products of the intersections of California’s many jurisdictional and voting district boundaries, contain voters whose demographic and partisan characteristics are quite similar to those of voters in adjoining polling place precincts. Kousser and Mullin (2007) took advantage of California’s natural experiment to measure the impact of mandatory mail ballot elections on voter participation. Because the natural experiment does not perfectly mimic random assignment, the study used matching methods to pair each mail ballot precinct with traditional precincts containing voters with similar demographic and political attributes. It compared voter participation between the matched precincts in several different elections, estimating the effect of voting by mail while holding political context and voter characteristics constant. Results from this natural experiment challenge the conventional wisdom about the participation effects of mail ballot elections. Turnout was two to three percentage points lower in VBM precincts during the 2000 and 2002 general elections than it was in similar polling place precincts. This finding is statistically significant, consistent from election to election, and robust to multiple estimation strategies. The effect of voting by mail appears to be contingent on election context: analysis of a smaller sample of
5
precincts shows that mail balloting may heighten turnout in low‐profile, local special elections. In high‐ stakes contests for statewide and national public office, however, the overall lesson is that requiring registrants to vote by mail may suppress participation to levels lower than what might be achieved through traditional election administration. The natural experiment design provided an opportunity to measure the effect of mail voting while holding constant voter characteristics and political context. What remains unknown is whether the negative effect that Kousser and Mullin (2007) observed would disappear with a full transition to mandatory mail elections. Is the behavior of registrants assigned to an anomalous voting process a reliable indicator of what would happen under broad implementation of a VBM system? If a county, state, or the nation as a whole jettisoned polling places, campaigns would alter their mobilization strategies in order to target voters completing their ballots at home. By contrast, when only a small subset of voters cast ballots through the mail, campaigns do not have as much incentive to tailor their efforts for mail voting. Studies of absentee and early voting have demonstrated the importance of campaign mobilization activities in the context of liberalized voting laws (Oliver 1996; Stein and García‐ Monet 1997). If current mobilization strategies are failing to reach mail voters, this could help explain Kousser and Mullin’s finding. To assess whether campaign mobilization efforts might help boost turnout in VBM elections, we conduct a randomized field experiment tailored to educating people about the VBM process and encouraging them to vote by mail. As we discuss in the next section, it is not altogether clear that VBM will boost turnout even after voters are informed about the VBM process.
II. The Rewards of Voting
The Riker and Ordeshook (1968) calculus of voting model provides a useful starting point to
consider the effect of VBM on turnout. In this model, Riker and Ordeshook conceptualize the voting decision as a function of four factors: the probability that one’s vote is decisive p, the material benefits
6
one gains from their party winning B, the expressive benefits one receives from fulfilling their civic duty to vote D, and the costs that are associated with voting C. Riker and Ordeshook’s central prediction is that people vote when the benefits of voting – both material and expressive – outweigh the costs of voting: pB + D > C Of course, since p is infinitesimally small in modern democratic elections, it is unlikely that material benefits play a large role in motivating people to vote. Instead, Riker and Ordeshook’s model implies that most people vote because the expressive value of voting captured by the D‐term outweighs the costs of voting.
Much of the literature dedicated to studying the effects of VBM on turnout focuses on the C‐
term (Karp and Banducci 2000, 2001; Gronke et al. 2008). Proponents of VBM argue that it lowers the costs of voting and, therefore, if one holds the terms on the left side of the inequality constant, the introduction of VBM to an electoral system should increase turnout. In a mail ballot election, registrants do not have to learn the location of their polling place or find a way to travel to it. They do not have to worry about parking, the safety of the neighborhood, or accessibility. Individuals can fill out their ballot at any time of day or night, in the comfort of their living room, with ready access to information about candidates and ballot measures from campaign mailings, newspapers, and the Internet. The absentee voting process provides another way to enjoy these lower costs, but it requires the extra step of applying for an absentee ballot. Because mail voting seems to offer a more convenient way to cast a ballot, the expectation has been that it has a positive impact on voter participation.
Yet one need not accept the assumption that the introduction of VBM would not also affect the
other terms in Riker and Ordeshook’s model. In particular, we suspect that VBM may have a direct effect on the D‐term. Gerber, Green, and Larimer (2008) argue that the D‐term can be decomposed into two components:
7
D = f(extrinsic rewards, intrinsic rewards) The intrinsic rewards of voting come from within the individual. They include the psychic benefit that one personally places on the act of voting. Some people gain a great deal of individual satisfaction from expressing their voice in democratic elections, while others see it as a necessary chore. With intrinsic rewards, the decision to vote is much like the decision to consume any private good: people vote because they gain personal enjoyment from doing so. Extrinsic rewards come from outside the individual, and include social pressure to conform to the norm that one should vote. With extrinsic rewards, the decision to vote involves a social calculation: people vote because they believe that others want them to vote and that their social standing is improved by doing what others want them to do. Taken together, this model is consistent with the Theory of Planned Behavior (Ajzen 1991) in which behavior is a function of people’s attitude toward the behavior and their subjective estimate of social norms surrounding the behavior.
Using a clever experimental design, Gerber, Green, and Larimer (2008) demonstrate that
extrinsic rewards play a dominant role in motivating voting relative to intrinsic rewards. They randomly assigned registered voters in Michigan to receive a postcard encouraging them to vote. Some individuals received a standard GOTV mailer in which they were reminded that voting is their civic duty and encouraged them to vote on Election Day. Others received mailers in which they were informed that whether one votes is of public record in addition to the standard encouragement to vote. Moreover, these subjects were told either that researchers would be checking to see if they voted or that a postcard that listed whether they voted was going to be sent to their household or neighbors after the election. Subjects who received the traditional GOTV mailer were about 2 percentage points more likely to vote than subjects in the control group, reflecting the effect of intrinsic rewards. However, the public‐records mailer was even more effective: those who were told that their neighbors would be
8
informed about their voting behavior were 8 percentage points more likely to vote than subjects in the control group. This effect is twice the size of the effects observed in the average door‐to‐door canvassing GOTV experiments (cf. Gerber, Green, and Nickerson 2003). It also demonstrates the power that social pressure exerts on people’s decision to vote and suggests that many people vote not only because they gain personal enjoyment from voting, but because they want to receive approbation for conforming to the social norm that people should vote. Viewed from a social psychological perspective, then, VBM removes social accountability from the voting decision. People vote in the solitude of their home. There are no poll workers or neighbors to see them at the polling place. They get no sticker that announces to their co‐workers “I voted.” In short, VBM may reduce the extrinsic rewards people receive from going to a traditional polling place and casting a ballot, potentially counteracting reductions in the costs of voting. The impersonal nature of mail ballot voting may also make them less responsive to a GOTV message delivered in person. Voters in traditional precincts can tell a canvasser that they plan to vote and then do so in front of their friends and neighbors, in essence doubling their extrinsic rewards. Because VBM voters have no opportunity to receive this direct social approbation, priming their attention to the D‐term in the calculus of voting may have less of an impact on their turnout decisions. If this is true, the effect of a GOTV contact in VBM precincts, while positive, should be weaker than it is in a traditional precinct.
III. Mobilization and Voting Propensity In a recent study, Arceneaux and Nickerson (2009a) propose that the effectiveness of mobilization varies by the individual’s baseline propensity to vote and the general level of interest that the electoral contest garners from the electorate. Arceneaux and Nickerson argue that in high‐stimulus elections (e.g., presidential elections), GOTV canvassing has the largest effect on low‐propensity voters – i.e., individuals who tend not to vote – while having a minimal effect on high‐propensity voters. After all, in a highly visible election, most high‐propensity voters decide to vote early on, well before they are
9
greeted by a GOTV canvasser. Many low‐propensity voters, on the other hand, are likely to be on the cusp of voting and the face‐to‐face blandishment to vote is instrumental in nudging them to turn out. Medium‐propensity voters are likely to fall in between the two extremes, exhibiting a smaller boost in turnout in response to a GOTV message in a high‐stimulus election than low‐propensity voters but a larger boost in turnout than high‐propensity voters. Here, we consider the theory of contingent mobilization in light of the social calculus of voting discussed in the previous section. Because high‐propensity voters tend to cast a ballot under any circumstances, the impact of extrinsic social rewards would be difficult to observe in a high‐stimulus election. On the flip side of the coin, low‐propensity voters tend not to vote, which suggests that the extrinsic rewards of voting generally do not offer them much of an enticement to vote. Instead, it is among medium‐propensity voters that we are most likely to observe the effects of extrinsic social rewards. Their intermittent record of voting suggests they may be sensitive to the context in which voting takes place. Unlike high‐propensity voters who almost always vote and low‐propensity voters who have repeatedly ignored social pressure to participate in elections, medium‐propensity voters pick and choose when to vote. We think that it is among these individuals that a social accountability mechanism (e.g., the public act of voting) may have the greatest influence on the decision to turn out. By reminding individuals about the visibility of the act of voting in a polling place, a GOTV campaign can prime attention to social accountability. We summarize all of our expectations thus far as follows: H1. Turnout should be marginally higher in the precincts that are randomly assigned to receive a GOTV contact. H2. The effect of the GOTV contact on turnout should be positive in both VBM and traditional precincts, but should be weaker in VBM precincts. H3. GOTV canvassing should have a large effect on low‐propensity voters, irrespective of voting method.
10
H4. GOTV canvassing should have no effect on high‐propensity voters, irrespective of voting method. H5. The prediction in H2 should hold among medium‐propensity voters.
IV. Research Design The California natural experiment with mail balloting provides an opportunity to compare the effectiveness of a GOTV campaign at turning out individuals who have been assigned to vote using different methods—either they were required to vote by mail or they could choose between visiting a polling place and applying to cast a ballot absentee—while holding constant other features of the electoral context. By state law, county registrars may designate any precinct with fewer than 250 registered voters as a mandatory mail ballot precinct. Small precincts are most likely to exist in rural areas, but the complex overlapping boundaries of California’s many local government jurisdictions and voting districts create small pockets of voters in urban and suburban communities as well. Registrars redraw precinct boundaries for every election, so it is possible for a household to be assigned a polling place for one election but not for the next. According to county election officials, the establishment of district boundaries is a purely administrative process, with the number of registered voters rather than the rate of anticipated turnout determining the decision. Natural Experiment with Precinct Matching
To strengthen inferences drawn from the natural experiment implemented by election officials,
however, we first needed to identify sets of comparable VBM and polling place precincts. Although county election officials declare which precincts must vote by mail, the selection process is not random. We therefore took some steps to find a group of traditional precincts that most resembled the county’s VBM precincts for that election. In order to examine the location and composition of each precinct, we obtained a geographic shapefile of precinct boundaries from the San Diego County registrar prior to the election and overlaid it with a block map from the U.S. Census. We then aggregated Census block‐level
11
data up to precinct boundaries, distributing population from split blocks based on the proportion of the block’s geographic area that falls within a precinct. This allocation rule assumes that all components of a block’s population are distributed uniformly throughout the block. We also measured the proportion of a precinct’s geographic area that lay within an urbanized area. The first step in creating balance between precinct types was to throw out exceptionally small VBM precincts (those with fewer than 75 registered voters), because these were most likely to be located in remote areas. This decision left us with 103 VBM precincts containing between 75 and 300 registrants.1 Second, we looked only at traditional precincts that were adjacent to at least one VBM precinct, on the logic that any characteristic predisposing a precinct to fall below the 250‐registrant threshold likely affects neighboring precincts as well. It also allows us to employ a regression discontinuity design. Even with these constraints, VBM and traditional precincts still differ. Figure 1a illustrates these differences. As expected, the biggest difference is in location: VBM precincts are almost twice as likely to be located outside an urbanized area as precincts that offer polling place voting.2 Otherwise, in terms of racial and age composition and the percentage of households that are owner‐ occupied, the two precinct types are reasonably well balanced. [Figure 1 about here]
Because the assignment of mail voting is not random, we cannot assume that the only
systematic differences between VBM and traditional precincts are attributable to the precinct’s voting system. We know that VBM precincts contain fewer registrants, and also that they are more rural. If we simply compared turnout levels in San Diego County’s VBM precincts to turnout in all of the county’s traditional precincts, it would not be clear what accounted for any observed differences. To improve our causal inference, we follow Kousser and Mullin’s (2007) approach of using statistical matching 1
The deadline for assigning mail precincts falls before the voter registration deadline, with the result that some VBM precincts exceed the 250‐registrant threshold using election‐day registration figures. 2 The Census classifies a territory as urban if the territory’s core has a density of at least one thousand people per square mile and surrounding areas contain at least 500 people per square mile.
12
techniques pioneered by Rosenbaum and Rubin (1983) to create comparable sets of precincts that have similar values on observable covariates. We matched our 103 VBM precincts to a set of 103 traditional precincts containing voters with similar demographic characteristics, effectively holding these characteristics constant and thus ruling them out as alternative explanations of any observed turnout difference.
The specific matching technique that we used was the genetic matching algorithm created by
Diamond and Sekhon (2005). We conducted one‐to‐one matches without replacement, matching on the number of registrants in the precinct, urban location, and six demographic characteristics, as well as exactly matching on three geographic strata that distinguished the most densely populated urban portion of San Diego County from the suburbs known as “East County” (actually located in the county’s western half) and the county’s remote eastern region, which is dominated by desert and parkland. We dropped the two matched pairs of precincts located in this eastern region, due to the logistical difficulty in reaching them for a face‐to‐face GOTV campaign. Overall, this approach yielded a great improvement in the balance of demographic characteristics between VBM and traditional precincts. After matching, as Figure 1b shows, the traditional precincts remaining in the analysis matched the VBM precincts closely in terms of urban/rural location (45.7% located in urbanized areas, compared to 43.4% of VBM precincts). The two sets of precincts continue to be well balanced on demographic composition, with the largest difference being that VBM precincts in the analysis have slightly larger Hispanic populations, a difference of 2.6 percentage points.
Matching strengthens our natural experiment by narrowing the differences between our two
sets of precincts, ruling out alternative explanations of variation in turnout. Two caveats merit discussion, however. The first is that a statistical match is only as good as the variables one matches on. We used all of the relevant Census and geographic characteristics that we could obtain, but we clearly have not matched on every factor related to turnout. Nonetheless, our understanding of the precinct
13
assignment process increases our confidence that unmeasured variables are not confounding our analysis. Precincts with fewer than 250 voters occur where many jurisdictions intersect in urban areas or where voters are spread out in rural areas. Our geographic and urbanization factors capture these differences. Election officials do not make their declarations according to the political dynamics in a precinct, the income levels of a precinct, their genetic makeup, or any other factor known to predict participation. As a result, demographic and location information should be sufficient for creating comparable sets of precincts.
The second caveat is that we initially performed this matching on “consolidated” precincts, but
then conducted our field experiment on “home” precincts, which are smaller subunits. Unfortunately, geographic shapefiles of home precincts were not available in time for us to calculate their demographic composition and perform the matching routines. Instead, we used information from consolidated precincts, an aggregation of multiple home precincts created by the county registrar. The large size of consolidated precincts (they contain an average of 761 registrants) prevented us from carrying out the GOTV campaign at the consolidated precinct level. Instead, we selected a home precinct from within each consolidated precinct to canvass. If a single home precinct contained the overwhelming majority of the consolidated precinct’s population, we selected it; otherwise, we randomly selected from the home precincts with more than 100 registrants. In the few cases where all home precincts contained fewer than 100 registrants, we selected the most populous precinct, but also took into account population density. The method was consistent between mail and traditional precincts, but consolidated mail precincts were more likely to be made up of large numbers of small‐population home precincts. Figure 1c shows that matching by home precincts actually produces even better balance, with no difference of greater than 1.2 percentage points emerging between the VBM and traditional precincts.
14
GOTV Field Experiment Beginning with these 101 matched pairs of home precincts, we conducted a field experiment by randomly selecting 50 pairs to receive a non‐partisan GOTV canvassing contact. We stratified the random assignment by the geographic strata discussed above. Randomized field experiments have been used extensively in recent years to examine the effects of grassroots mobilization on voter turnout (e.g., Gerber and Green 2000; Gerber, Green, and Nickerson 2003; Green and Gerber 2008). These studies have a number of advantages over traditional observational studies of voter mobilization. Observational approaches measure the effects of voter mobilization after the fact by comparing the voting rates of individuals who were contacted by campaigns to the rates among those who were not. Because researchers cannot know for certain the selection process that caused individuals to receive contact, it is possible that any observable differences in voting rates between the contacted and un‐contacted groups may be an artifact of the selection process and not the effect of campaign contact. For instance, voters may simply be more likely than nonvoters to be home and available to receive contact from campaigns. Including a bevy of control variables does not obviate this concern, because in observational studies one can never be sure that all relevant covariates have been accounted for (Arceneaux, Gerber, and Green 2006). In contrast, field experimental designs randomly assign whether citizens receive GOTV contact before the campaign begins. Random assignment furnishes unbiased causal effect estimates of the mobilization effort, because it balances the treatment and control groups with respect to both observed and unobserved covariates.3 Every subject has an equal chance to be in the treatment or control group, which means that the distribution of subject attributes, such as voting propensity, will be identical (within sampling variability) for each group. Consequently, if the treatment were not administered, 3
Of course, randomization generates balance within sampling variability. Imbalances can occur by chance, but it is possible to calculate the probability that observed effects are due to sampling error with standard frequentist tests of statistical significance.
15
there should be no differences in voting rates between the treatment and control group after the study ends. If a statistically significant difference does exist, it can be attributed with confidence to the effect of the campaign intervention. We implemented our field experiment by contracting with a professional voter contact firm, the La Jolla Group. Bob Glaser, the group’s principal, has over 25 years of experience in San Diego campaigns and works consistently with a group of trained, professional precinct walkers. We provided him with a list of the 50 randomly‐selected traditional precincts and 50 randomly‐selected VBM precincts, and he worked with another vendor to purchase lists of registered voters and precinct maps.4 Disaggregating to home precincts produced very little loss in the quality of our match. Table 1 shows the factorial design of our study, and Figure 2 shows the location of the 100 home precincts that received the GOTV treatment. We also provided voter contact scripts and postcards with GOTV messages for each group which explained the voting process in their precinct and encouraged them to vote. Canvassers recited the script to voters who answered their doors and also left postcards with them, while simply leaving postcards at the residences where no one answered the door. [Table 1 and Figure 2 about here] We strove to make the GOTV messages as similar as possible for both the vote‐by‐mail and traditional precincts, so that we can attribute the effects of the mobilization effort to the GOTV contact and not the particular message employed. These messages, of course, could not be identical for the two different methods of voting. Our appendix reprints the text of each postcard to show the different details yet similar themes of our GOTV messages. We printed these postcards ourselves on colored cardstock and provided them directly to the La Jolla Group, while Bob Glaser trained his canvassers in the verbal scripts. We observed this training and watched him interact with canvassers at his office on numerous occasions, and coordinated with him on all aspects of the GOTV campaign. 4
Precinct population size, racial breakdown (percent black, Hispanic, and Asian), percent under 18 and over 65, and level of urbanity do not jointly predict treatment assignment (χ2[8] = 5.58, p = 0.695).
16
They carried out this campaign in the nine days leading up to the November 4th, 2008 General Election, which in addition to the presidential election featured 12 statewide propositions in California, one countywide proposition in San Diego, and many local candidate and proposition contests. Our canvassers began on Saturday, October 25th and finished their final two precincts on Monday, November 3rd. They focused their efforts on weekend days on which more voters are home, and were able to walk 28 of the VBM precincts and 22 of the traditional precincts on either a Saturday or a Sunday. Canvassers recorded the nature of their contact with each voter on a printout that included information on the voter’s precincts, home address, and voter identification number, allowing us to match this contact information with the county’s individual voter file. Using this information, we can determine whether a given voter personally talked with a canvasser, lived in the same household as someone who personally talked with the canvasser, merely had a postcard left on his or her doorstep, or lived at a residence that the canvasser could not access.5 We coded a subject as contacted if a canvasser successfully spoke to any member of the subject’s household.6 Using this definition, the contact rate was 13.75 percent in traditional treatment precincts and 13.86 percent in VBM treatment precincts. The small difference in contact rates between traditional and VBM precincts is not statistically significant (F[1, 99] = 0.001, p = 0.979). No subjects in the control group were contacted. Data for our dependent variable come from the voter file prepared by the San Diego County Registrar, an official record of the 2008 election. 5
This post‐contact information demonstrates how difficult it can be to reach voters at their residences. Sometimes, an entire precinct was inaccessible. Five of the traditional precincts in our treatment group could not be reached, four because they were located entirely within gated communities and one because it consisted of residences on boats moored in the San Diego harbor. Four of our VBM precincts were unreachable, two because they were within gated communities and two because they were in restricted land on Native American reservations. We did not treat any of these voters, but we did leave all of them in the analysis. Because we randomly selected the treatment precincts, we can assume that an equal number of precincts in our control group were also inaccessible, both to our canvassers and to other campaigns. Leaving these precincts in our treatment groups, both VBM and traditional, keeps them parallel to our control groups in everything but their method of voting. 6 Coding all members of a household as contacted even if only one member of the household spoke with a canvasser allows us to meet the Stable Unit Treatment Value Assumption (SUTVA), which holds that the outcome of one subject does not depend on the treatment of another (see Angrist, Imbens, and Rubin 1996).
17
V. Results The Effects of GOTV Canvassing
We model the effects of GOTV treatment and its interaction with VBM assignment using a linear
probability model. The results are displayed in Table 2. Columns 1 and 2 report the Intent‐to‐Treat (ITT) effects, which estimate the effect of random assignment – i.e., those we intended to treat – on the probability of voting. Columns 3 and 4 report the Average Treatment on Treated (ATT) effects, which estimate the effect of actual GOTV contact on the probability of voting. Because actual GOTV contact is not random and the sorts of people who are reachable by door‐to‐door canvassers have a higher baseline of voting, simply comparing the voting rates of contacted to uncontacted individuals would produce biased estimates of the ATT effect (Arceneaux, Gerber, and Green 2006). Consequently, we use random assignment as an instrument to estimate the effect of GOTV exposure. Since voters could only be contacted by our canvassers via random assignment, it meets the exclusion restriction as well as the other assumption underlying instrumental variables regression (cf. Angrist, Imbens, and Rubin 1996; Gerber and Green 2000). Finally, we cluster the standard errors by precinct to account for the precinct‐ level randomization (Arceneaux and Nickerson 2009b). [Table 2 about here]
We report the simple ITT model without covariates (save for an indicator for the randomization
strata) in Column 1. The coefficient for GOTV treatment shows the effect of random assignment on voter turnout in traditional polling place precincts. Voters living in traditional precincts that were assigned to be canvassed were 1.7 percentage points more likely to vote than voters in traditional precincts that were assigned to the control group (SE = 1.7). The interaction term between GOTV treatment and VBM assignment measures the degree to which the effect of GOTV on turnout differs between traditional and VBM precincts. The results show that, as predicted, the effect of GOTV
18
canvassing was somewhat weaker in VBM precincts (interaction = ‐0.8 percentage points, SE =2.9). None of these effects attain standard levels of statistical significance. We can increase the statistical power of our analysis by adding covariates that affect the dependent variable. Since treatment was randomly assigned, adding covariates should not affect the treatment effect estimates, but will improve the fit of the regression model and, thus, generate more precise estimates of the standard errors. We report the covariate‐adjusted ITT effects in Column 2 of Table 2. From the voter file, we include variables measuring subjects’ voting histories, age, and gender; we also account for major party registrants, new registrants (i.e., those who registered after the 2006 election) and permanent absentee voters. In addition, we include measures of precinct‐level racial composition, calculated using GIS from U.S. Census block‐level data. Altogether, these covariates markedly improve the fit of the ITT model; the R2 goes from 0.001 in the without‐covariates model to 0.21 in the with‐covariates model. The more precisely estimated standard errors allow us to reject the null hypothesis with respect to the effect of GOTV canvassing in traditional precincts. As Figure 3a shows, voters in treated traditional precincts were 1.9 percentage points more likely to vote than voters in traditional precincts that were not canvassed (SE = 0.7, p
