AccurAcy Assessment of predictive models of ...

6 downloads 0 Views 367KB Size Report
mus henslowii), and bobolink (Dolichonyx oryzivorus). we also developed predictive models of abundance for two addi- tional species, savannah sparrow ...
The Condor 110(4):747–755  The Cooper Ornithological Society 2008

Accuracy Assessment of predictive models of grassland bird Abundances in the prairie hardwood transition Bird conservation region Les D. Murray1,4, C hristine A. R ibic2 , Wayne E. Thogmartin3,

and

M elinda G. K nutson3

1

Department of Forest and Wildlife Ecology, University of Wisconsin–Madison, 1630 Linden Drive, Madison, WI 53706 2 U.S. Geological Survey, Wisconsin Cooperative Wildlife Research Unit, Department of Forest and Wildlife Ecology, University of Wisconsin–Madison, 1630 Linden Drive, Madison, WI 53706 3 U.S. Geological Survey, Upper Midwest Environmental Sciences Center, 2630 Fanta Reed Road, La Crosse, WI 54603 Abstract. We tested statistical models developed to predict abundances of grassland birds in the Prairie Hardwood Transition Bird Conservation Region of the upper midwestern United States. Roadside surveys were used to estimate relative abundances of grassland birds in 800 ha areas in the Iowa, Minnesota, and Wisconsin portions of the region in 2003–2005. We then compared observed abundances with predicted abundances from spatial hierarchical models for seven species. Spearman’s rho statistic for rank correlations suggested that observed abundances were positively correlated with predicted abundances for all species (rs = 0.21–0.60) except the Henslow’s Sparrow (Ammodramus henslowii; rs = 0.01). Observed abundances also were positively correlated with percent grassland in an area, and rank correlation values were similar to those obtained from the predictive models. Model accuracy was positively related to species’ abundance and niche breadth. Our accuracy assessment suggested that the spatial hierarchical models would have limited use in guiding management at a regional scale; a measure of habitat quantity performed equally as well as the models at predicting observed abundances. Future efforts to model grassland bird abundances would be improved by more accurate information on the distribution of grasslands in the region, more detailed information on grassland composition and structure, and a better understanding of the biological significance of environmental variables for grassland bird populations. Key words:  abundance, accuracy, evaluation, grassland birds, niche breadth, predictive model, validation.

Evaluación de la Exactitud de los Modelos que Predicen la Abundancia de Aves de Pastizales en la Región de Conservación de Aves de Transición Pradera-Bosque Resumen. Probamos modelos estadísticos desarrollados para predecir la abundancia de aves de pastizales en la región de conservación de aves de transición pradera-bosque en la parte alta del medio oeste de Estados Unidos. Se utilizaron censos realizados a lo largo de carreteras entre 2003 y 2005 para estimar la abundancia relativa de aves de pastizal en áreas de 800 ha en las porciones de esta región ubicadas en Iowa, Minnesota y Wisconsin. Luego comparamos las abundancias observadas con las abundancias predichas por modelos espaciales jerárquicos para siete especies. El estadístico rho de Spearman para correlaciones de rangos sugirió que las abundancias observadas estuvieron correlacionadas positivamente con las abundancias predichas para todas las especies (rs = 0.21–0.60), excepto para Ammodramus henslowii (rs = 0.01). Las abundancias observadas también estuvieron correlacionadas positivamente con el porcentaje de cobertura de pastizales en un área, y los valores de correlación de rangos fueron similares a los obtenidos a partir de los modelos. La exactitud de los modelos estuvo relacionada positivamente con la abundancia de las especies y con la amplitud del nicho. Nuestra evaluación de la exactitud sugirió que los modelos espaciales jerárquicos serían de poca utilidad para guiar el manejo a una escala regional y que una medida de la cantidad de hábitat se desempeñó igual de bien que los modelos al predecir las abundancias observadas. Los esfuerzos futuros para modelar la abundancia de las aves de pastizales podrían ser mejorados contando con información más exacta acerca de la distribución de los pastizales en la región, con información más detallada sobre la composición y estructura de los pastizales y con un mejor entendimiento de la significancia biológica de las variables ambientales para las aves de estos ambientes.

INTRODUCTION Grassland bird populations have declined in the midwestern United States (Knopf 1994, Herkert 1995, Peterjohn and Sauer 1999), and most grassland species are considered to be of high

conservation concern (Knutson et al. 2001). Unfortunately, management of grassland birds at a regional level is limited, because sparse data on geographical patterns of grassland bird abundances within a region make selection of focal species and priority management areas difficult.

Manuscript received 22 April 2008; accepted 21 October 2008. 4 Present address: School of Environment and Natural Resources, The Ohio State University, 2021 Coffey Road, Columbus, OH 43210. E-mail: [email protected] The Condor, Vol. 110, Number 4, pages 747–755. ISSN 0010-5422, electronic ISSN 1938-5129.  2008 by The Cooper Ornithological Society. All rights reserved. Please direct all requests for permission to photocopy or reproduce article content through the University of California Press’s Rights and Permissions website, http://www.ucpressjournals.com/ reprintInfo.asp. DOI: 10.1525/cond.2008.8610

747

15_MS8610.indd 747

11/28/08 6:00:57 PM

748   LES D. MURRAY

et al .

Thogmartin et al. (2006) developed hierarchical statistical models to predict abundances of five species of grassland bird throughout the Prairie Hardwood Transition Bird Conservation Region of the upper midwestern United States (Bird Conservation Region 23; U.S. North American Bird Conservation Committee 2000). The goal of these modeled distributions was to guide management efforts by identifying target areas or focal species within the region. Potential uses of the models within the region are to: (1) identify the proportion of a species’ population that occurs on conserved lands versus private lands, (2) identify grassland ‘focal areas’ in the region, and (3) inform agency decisions related to habitat conservation efforts. Obviously, the accuracy and usefulness of predicted species’ distributions should be assessed before management resources are allocated based on models. Thogmartin et al. (2006) tested the predictions from their models against a subset of the original data not used to develop the models. In general, their models showed a positive correlation between predicted and observed abundances across the region. This evaluation of the models’ predictive ability, however, used data collected in the same time period and at the same scale as the data used to construct the models. Johnson (2001) suggested that accuracy of model predictions should be tested against an independent dataset. Thogmartin et al. (2006) did use an independent set of point counts from the region to evaluate the accuracy of their models, but these point counts were conducted mostly on federal lands and did not assess model accuracy outside federal areas. Grassland birds also commonly occur on state and privately owned lands (Frawley and Best 1991, Patterson and Best 1996, Sample and Mossman 1997, Temple et al. 1999, McCoy et al. 2001); therefore, it is important to assess the accuracy of these models for lands not in federal ownership. Accuracy of model predictions is dependent on model assumptions, the modeling technique used, and the accuracy of the data used to construct the model. Species’ distributions and model accuracy also have been shown to be related to species abundance and niche characteristics (Gaston et al. 1997, Brändle and Brandl 2001, Marsden and Whiffin 2003). Several studies have found a positive relationship beween model accuracy and species abundance (Garrison and Lupo 2002, Karl et al. 2002, Kadmon et al. 2003), and model accuracy has been found to be lower for generalist species than for specialist species (Hepinstall et al. 2002, Brotons et al. 2004). The hierarchical modeling and mapping process of Thogmartin et al. (2006) required extensive statistical expertise and time to predict abundances of grassland birds. Therefore, the usefulness and accuracy of these models should be compared with the ability to predict abundance relative to simpler methods. Percent grassland in an area is a relatively easy measure of habitat quantity that can be easily calculated using available land-use datasets (i.e., National Land Cover Dataset; Vogelmann et al. 2001). Consequently, the usefulness of the models could be assessed by comparing the accuracy of

15_MS8610.indd 748

predictions of grassland bird abundance made by the models versus percent grassland in the area. We assessed the accuracy of model predictions of grassland bird abundances in the Prairie Hardwood Transition against independently collected data from roadside surveys. Our objectives were to: (1) assess correlations between predicted abundances and observed abundances in 800 ha areas, (2) examine the relationships between model accuracy and species’ niche breadth and abundance, and (3) assess the usefulness of the predictive models compared with a simple measure of grassland quantity (i.e., percent grassland) in 800 ha areas. We chose to use 800 ha areas because this was the finest spatial scale at which the models were designed to be used (Thogmartin et al. 2006). METHODS Study area

The majority of the Prairie Hardwood Transition Bird Conservation Region occurs in Michigan, Minnesota, and Wisconsin, but it includes portions of Illinois, Iowa, and Indiana (U.S. North American Bird Conservation Committee 2000). Historically, the Prairie Hardwood Transition consisted of a mosaic of oak savanna, tall-grass prairie, and wetlands bounded by hardwood forest to the north and east and prairie to the south and west (Transeau 1935, Nuzzo 1986). Current land cover is composed of a mixture of row crop agriculture (42%), deciduous forest (21%), and grasslands (20%), and less than 1% of the prairie and savanna in the area remains intact (Johnson 1986, Sample 1989, Johnson and Temple 1990). Predictive bird abundance models

Spatially explicit predictions of relative abundances of birds in the Prairie Hardwood Transition were developed by Thogmartin et al. (2006) for five grassland-obligate species of management concern: Upland Sandpiper (Bartramia longicauda), Sedge Wren (Cistothorus platensis), Grasshopper Sparrow (Ammodramus savannarum), Henslow’s Sparrow (Ammodramus henslowii), and Bobolink (Dolichonyx oryzivorus). We also developed predictive models of abundance for two additional species, Savannah Sparrow (Passerculus sandwichensis) and Eastern Meadowlark (Sturnella magna), using the technique of Thogmartin et al. (2006; see Murray [2006] for details). Details of the methods used to construct the predictive models are given by Thogmartin, Sauer et al. (2004) and Thogmartin et al. (2006); a summary of the modeling methods is given below. Models were developed with North American Breeding Bird Survey (BBS) data from 1981 to 2001 using Bayesian hierarchical modeling with flat priors and iterative simulation (i.e., Markov chain Monte Carlo). Explanatory variables for each model were chosen a priori from 80 possible variables (Thogmartin, Sauer et al. 2004). Environmental variables were calculated within buffers of 0.1, 1, and

11/28/08 6:00:58 PM

ACCURACY OF GRASSLAND BIRD ABUNDANCE MODELS   749

10 km around BBS routes to accommodate potential scaling relationships between avian responses and environmental variables. Grids of 800, 8000, and 100 000 ha cells were used to calculate and map environmental variables for the 0.1, 1, and 10 km buffers, respectively. Land-cover variables were calculated from the 1992 National Land Cover Dataset. All models also included five variables not related to habitat: the spatial neighborhood, a year effect, a temporal trend effect, an observer effect, and a novice observer effect. Model averaging (Burnham and Anderson 2002) was used to predict abundances of the focal species. The average of all models within five deviance information criterion (DIC) units of the minimum DIC model (Burnham and Anderson 2002) was used to predict abundance for all species except the Savannah Sparrow and Eastern Meadowlark. Due to the large number of models within five DIC units for these two species, models within 2 DIC units were averaged for the Eastern Meadowlark, while the model with the minimum DIC value was used for the Savannah Sparrow. The number of variables included in models used to predict abundances ranged from five to 11 and included climate and habitat variables (Table 1). Predicted abundances were mapped as the 20-year expected mean (i.e., mean predicted counts for the period 1981−2001) on a 50-stop BBS route centered on a 1 ha pixel based on environmental variables measured in 800, 8000, and 100 000 ha grid cells. Independent dataset

An independent dataset of relative bird abundances was collected at 525 locations in the Iowa, Minnesota, and Wisconsin portions of the Prairie Hardwood Transition in 2003–2005. The same grid cells used to measure environmental variables for the mapping process were used as sampling units for independent data collection. Bird surveys were conducted in 800 ha cells because this was the smallest scale at which environmental variables used in the modeling process were measured. In addition, the scale at which variables would be important in each species’ model was not known prior to data collection, so the smallest scale was used. A nested sampling scheme was used to select the 800 ha cells in which surveys were conducted. We divided the study area into five subregions of equal size to disperse samples across the study area and randomly selected 8000 ha cells within each of the subregions (Fig. 1). The 8000 ha cells were selected independently of the 100 000 ha cells used in the mapping process to allow for greater dispersion of sampling points across the study area. Within each 8000 ha survey area, three 800 ha cells that were greater than 50% contained within the 8000 ha cell were selected to be surveyed. Independent data collected in any one year might not be an accurate assessment of mean expected abundance at a location because of year-to-year differences in bird numbers related to annual variation and trends in demographic and environmental factors (e.g., climate, food availability, survival, or reproductive success). To examine the effects of annual

15_MS8610.indd 749

TABLE 1. List of explanatory variables used to predict abundances of grassland birds in the Prairie Hardwood Transition Bird Conservation Region of the upper midwestern United States (Murray 2006, Thogmartin et al. 2006). The spatial extent at which habitat variables were included in the models are given in parentheses. Climate variables were included in the models at the smallest spatial extent of habitat variables included in the model. Variables are arranged in alphabetical order. Species are arranged in descending order of relative abundance in the study area (Table 2). Species Explanatory variablea Savannah Sparrow (Passerculus sandwichensis)   Area-weighted mean grassland patch size (800 ha)  Mean spring temperature  Percent grassland*Percent forest (800 ha)  Percent row crop (800 ha)  Total summer precipitation Bobolink (Dolichonyx oryzivorus)   Area-weighted mean grassland patch size (800 and 8000 ha)  Coefficient of variation in annual precipitation  Coefficient of variation in grass patch contiguity (8000 ha)  Mean temperature during coldest quarter  Percent forest (800, 8000, and 80 000 ha)  Percent forest*Area-weighted mean grassland patch size (8000 ha) Sedge Wren (Cistothorus platensis)  Disjunct core area of wetlands (8000 ha)  Mean January temperature  Percent forest (800 ha)  Percent grassland (800 ha)  Percent mucky soils (800 ha) Eastern Meadowlark (Sturnella magna)   Area-weighted mean grassland patch size (800 ha)  Mean summer temperature  Mean temperature during wettest quarter  Percent forest (800 ha)  Percent grassland (800 ha)  Percent grassland*static wetness index (800 ha)  Percent row crop (800 ha)  Percent urban grass (800 ha)  Static wetness index (800 ha)  Total precipitation in coldest quarter  Total precipitation in driest quarter Grasshopper Sparrow (Ammodramus savannarum)  Mean autumn precipitation  Mean summer precipitation  Mean temperature during warmest quarter  Percent forest (800 ha)  Range in growing season temperature  Static wetness index (800 ha)  Variation in autumn precipitation  Variation in summer precipitation Henslow’s Sparrow (Ammodramus henslowii)   Area-weighted mean grassland patch size (80 000 ha)  Coefficient of variation in annual precipitation  Land-cover diversity (modified Simpson’s index; 800 ha)  Mean temperature during driest season  Percent forest (8000 ha)  Percent forest*Area-weighted mean grassland patch size (8000 ha)  Total warm season precipitation Upland Sandpiper (Bartramia longicauda)   Area-weighted mean grassland patch size (800, 8000, and 80 000 ha)  Coefficient of variation in mean spring precipitation  Mean spring temperature  Percent forest (800, 8000, and 80 000 ha)  Percent sandy soil (8000 ha) a  Variables and model details are described by Thogmartin, Sauer et al. (2004) and Thogmartin et al. (2006).

11/28/08 6:00:59 PM

750   LES D. MURRAY

et al .

FIGURE 1. Map of 175 ~8000 ha cells surveyed for grassland birds in 2003, 2004, or 2005 in the Iowa, Minnesota, and Wisconsin portions of the Prairie Hardwood Transition Bird Conservation Region of the upper midwestern United States. An equal number of cells were randomly selected in each of five subregions (solid lines) to disperse samples across the study area. Inset shows a representative 8000 ha cell divided into a grid of 800 ha cells, from which three (hatched areas) were randomly selected to be surveyed. The 800 ha and 8000 ha grids of cells were the same as those used by Thogmartin et al. (2006) to measure environmental variables used to predict abundances of birds in the study area.

variation in bird abundances on model accuracy, we surveyed five 8000 ha cells in each subregion for three years. Each year an additional ten 8000 ha cells from each subregion were selected to be surveyed in only a single year. We used these data to assess differences in accuracy between independent data collected over one year and multiple years. We chose not to survey the same cells in all years because we thought that a more spatially dispersed sample would better assess the accuracy of the model over the entire study area. In total, 25 8000 ha cells were surveyed all three years, and 150 8000 ha cells were surveyed in a single year. Therefore, 75 800 ha cells were surveyed all three years and 450 800 ha cells were surveyed in a single year, for a total of 525 800 ha cells surveyed. Roadside survey routes. Roadside survey routes consisting of three stops for each 800 ha cell were conducted by 46

15_MS8610.indd 750

qualified individuals from 1 June to 4 July in 2003, 2004, and 2005. Survey methods and dates of surveys were designed to be similar to BBS survey protocol to minimize the effects of date and techniques on model accuracy assessment. Surveys were not conducted during rain, fog, or high winds (>19 kph). Surveys could be started no more than 0.5 hr before sunrise and finished no later than 4.5 hr after sunrise. Observers recorded the number of birds seen or heard within 400 m of each survey point within a 3 min period. The three 800 ha cells located within each 8000 ha cell were surveyed on the same day by the same observer. Survey routes were located along secondary roads, and the starting point of each route was determined by randomly selecting a road that intersected the cell boundary and placing the first point ≥0.4 km from the cell’s edge. Subsequent stops were placed approximately 0.8 km from the previous stop. If the distance between points was greater than the distance to an intersection of two secondary roads, the direction in which to travel at the intersection (right, left, or straight) was chosen randomly. An effort was made to have all routes independently surveyed by two different observers within a year to reduce the effects of detection biases associated with observers, weather conditions (e.g., lower than normal temperatures), or low bird activity at the time of a single survey. Roadside surveys were completed at least once a year for all areas in 2003 and 2004 and for 99% of areas in 2005. Most areas (85%–93%) were surveyed twice each year. We directed observers to record only 15 predetermined species to reduce biases related to individual knowledge, experience, species affinities, and difficulties counting large numbers of birds. Recording a subset of species allowed observers to focus identification efforts on a limited number of songs and ignore abundant species not of interest to the project (e.g., Red-winged Blackbird [Agelaius phoeniceus]). This approach differed from BBS surveys, which ask observers to count all bird species at a stop, and our surveys were probably more effective at detecting target species than BBS surveys. Abundances and data standardization

The relative abundances for a route from the independent surveys are presented as the maximum number of birds observed from the two repeated surveys within a year. The abundance data used to create the models, however, was only collected once per year along a survey route. We chose to use the maximum number rather than the mean number of birds observed because territorial birds are assumed to stay in the same area for most of the breeding season. Therefore, the maximum number of birds should give a more accurate representation of the number of territories in the area. The maximum number along a route was divided by the number of stops surveyed for each route because not all stops were surveyed for some

11/28/08 6:01:02 PM

ACCURACY OF GRASSLAND BIRD ABUNDANCE MODELS   751

routes. A minimum of two stops on a route needed to be surveyed for a route to be included in analyses. For routes surveyed in all three years, we averaged the maximum number of birds per stop for a route across years. Predicted abundances were arbitrarily mapped at a resolution of 1 ha, but the independent data were collected in 800 ha cells. Therefore, we compared the independent data collected for each 800 ha cell with the mean predicted abundance per pixel for the entire cell. Using the central pixel of a cell to calculate predicted abundances did not improve model accuracy relative to using the cell mean (Murray 2006), so these results are not presented here. Predicted abundances represented a mean of 20-year counts collected annually and summed over 50-stop routes. Observed counts represented either single-year or three-year mean maximum counts from routes with two or three stops. Because of these differences in scale, both observed and predicted abundances were standardized by subtracting the mean and dividing by the standard deviation to allow for direct comparison between datasets. Standardized abundances thus represent the number of standard deviations from the mean of the respective dataset. The mean and standard deviation for predicted abundances were calculated using only those cells being compared to observed abundances. Statistical analyses

Spearman’s rho statistics for rank correlation (rs) were calculated in R (R Development Core Team 2006) to give a general measure of correlation between relative predicted and observed abundances. Other accuracy measurements are available to compare absolute abundances (Mayer and Butler 1993, Murray 2006), but comparison of relative abundances better assessed the usefulness of models for management purposes. In addition, the differences in the number of points surveyed

along BBS routes and independent routes made comparison of absolute abundances difficult. We first compared rs for observed data from routes surveyed in all three years to data for routes surveyed in only a single year. All other analyses were conducted using the survey data (one or three years) that was most highly correlated with the predicted abundances. If correlations were similar for data collected over three years and in a single year then all data were used for further analyses. We also calculated rs to measure the correlation between percent total grassland in an 800 ha area calculated from the 1992 National Land Cover Dataset and the observed abundance of birds in that area. The rs for percent grassland allowed for assessment of the usefulness of the predictive models relative to a simple index of habitat quantity. Standardized observed abundances were also plotted against standardized predicted abundances to provide visual assessments of their general relationships. A fitted leastsquares regression line with 95% confidence limits and a 45° line representing perfect model fit also were plotted to aid visual assessment of the model. We used least-squares regression to determine if variation among species in correlation between observed and predicted abundances was explained by mean observed abundances and niche breadth. Niche breadth was calculated as the proportion of habitats in which each species was documented or presumed to breed based on information provided by Best et al. (1995). RESULTS The Savannah Sparrow was the most frequently recorded grassland bird during roadside surveys (Table 2). Bobolinks were second-most abundant, followed by Sedge Wrens, Eastern Meadowlarks, and Grasshopper Sparrows. Henslow’s

TABLE 2. Total number of detections and mean relative abundance ± SE from independent roadside surveys, niche breadth (proportion of habitats in which a species was documented or presumed to breed [Best et al. 1995]), and Spearman’s rho measuring the correlation between observed abundances and predicted abundances (Murray 2006, Thogmartin et al. 2006) and between observed abundances and percent grassland measured from the 1992 National Land Cover Dataset for seven species of grassland bird in 800 ha areas in the Prairie Hardwood Transition Bird Conservation Region of the upper midwestern United States. Measures were compared for observed abundances estimated over one year (n = 450) or three years (n = 75). Species are ordered by decreasing abundance in the study area. Model rs Species Savannah Sparrow Bobolink Sedge Wren Eastern Meadowlark Grasshopper Sparrow Henslow’s Sparrow Upland Sandpiper

Detections

Relative abundancea

Niche breadth

1 year

3 years

1964 933 443 471 174 21 13

0.70 ± 0.92 0.36 ± 0.74 0.19 ± 0.47 0.18 ± 0.35 0.09 ± 0.26 0.01 ± 0.06 0.01 ± 0.06

0.50 0.40 0.40 0.60 0.45 0.15 0.25

0.30 0.32 0.06 0.28 0.05 –0.05 0.05

0.60 0.51 0.21 0.38 0.35 0.01 0.29

Grassland rs 3 years 0.53 0.45 0.12 0.46 0.28 0.05 0.17

a

 Mean maximum number of birds seen within 400 m of a roadside survey stop during a 3 min period.

15_MS8610.indd 751

11/28/08 6:01:03 PM

et al .

15_MS8610.indd 752

Bobolink 4

5

Observed abundance

Observed abundance

Savannah Sparrow

4 3 2 1 0 −1

3 2 1 0

0 1 2 3 4 Predicted abundance

−1

3 2 1 0 0

0 1 2 3 Predicted abundance

Eastern Meadowlark Observed abundance

Observed abundance

Sedge Wren

2 1 0

2 4 6 8 Predicted abundance

0 1 2 3 Predicted abundance

Grasshopper Sparrow

Henslow’s Sparrow

4

5 Observed abundance

Observed abundance

Sparrows and Upland Sandpipers were rare, with only 21 and 13 observations over three years, respectively. Spearman’s rank correlation values were higher for data collected over three years than in only one year for all species (Table 2). Thus, we used observed abundances only from routes surveyed in all three years for further analyses. Correlations between predicted and observed abundances were positive for all species; however, correlation for the Henslow’s Sparrow, a species we rarely recorded, was near zero (Table 2). The highest correlations were for the Savannah Sparrow and Bobolink, species we recorded most frequently. Plots of observed vs. predicted abundances for five species (Upland Sandpiper, Grasshopper Sparrow, Savannah Sparrow, Bobolink, and Eastern Meadowlark) also suggested a positive relationship between model predictions and actual abundances (Fig. 2). Visual assessment suggested no relationship between predicted and observed abundances for the Sedge Wren, but a single outlier with very high predicted abundance (standardized predicted abundance = 7.8) and no observed birds likely contributed to the lack of a positive trend. The rank correlation between predicted and observed abundances for the Sedge Wren when the outlier was removed, however, was only 0.24, compared with 0.21 including the outlier. Visual assessment of plots of predicted and observed abundances found cases of underpredicting and overpredicting for all species (Fig. 2). The models underpredicted abundances for the Sedge Wren, Eastern Meadowlark, Grasshopper Sparrow, and Henslow’s Sparrow, all of which had high observed abundances in areas with predicted abundances near zero. Although overprediction occurred for all species, low abundances were rarely observed in areas with the highest predicted abundances, except in the case of the Henslow’s Sparrow. Correlations between percent grassland and observed abundances also were positive for all species (Table 2). The correlations of observed abundances with predicted abundances from the models were only slightly higher than correlations with percent grassland for five species (Savannah Sparrow, Bobolink, Sedge Wren, Grasshopper Sparrow, and Upland Sandpiper). For the Eastern Meadowlark and Henslow’s Sparrow, observed abundances were more closely correlated with percent grassland in an 800 ha area than with predicted abundances. The difference in correlation values between predicted abundances and percent grassland was greatest for the Upland Sandpiper (0.12); differences between correlations for the other species were less than 0.10. The correlation between observed and predicted abundance increased for more abundant species and species with broader niches (Fig. 3). The variation in correlation values was better explained by abundance (R2 = 0.67) than niche breadth (R2 = 0.50). Abundance and niche breadth were positively correlated (rs = 0.56).

3 2 1 0 0 1 2 3 4 Predicted abundance

4 3 2 1 0 0 1 2 3 4 5 6 Predicted abundance

Upland Sandpiper 7 Observed abundance

752   LES D. MURRAY

6 5 4 3 2 1 0 0 1 2 3 4 Predicted abundance

FIGURE 2. Observed abundances of seven species of grassland bird in the Prairie Hardwood Transition Bird Conservation Region of the upper midwestern United States fitted against predicted abundances with least-squares regression (solid line) and 95% confidence limits (dotted lines). The dashed line is the line of one-to-one correspondence. Abundances were centered around zero and standardized. Standardized abundances represent the number of standard deviations from the mean of the respective dataset. Species are arranged in order of decreasing abundance. All species except the Henslow’s Sparrow and Sedge Wren show a general positive relationship between predicted and observed abundance, but all species show cases of over- and underprediction.

11/28/08 6:01:06 PM



0.6 �

0.5 0.4 0.3





� �

0.2 0.1 0.0



R 2 = 0.67

0.0 0.1 0.2 0.3 Log10(abundance + 1)

Spearman’s rank correlation

Spearman’s rank correlation

ACCURACY OF GRASSLAND BIRD ABUNDANCE MODELS   753



0.6 �

0.5 0.4



0.3



� �

0.2 0.1 0.0



R 2 = 0.50

0.2 0.4 Niche breadth

0.6

FIGURE 3. Spearman’s rank correlations between observed abundances and predicted abundances from models of seven grassland bird abundances in the Prairie Hardwood Transition Bird Conservation Region of the upper midwestern United States plotted against logtransformed observed abundances and niche breadth. Niche breadth is defined as the proportion of habitats in which each species was documented or presumed to breed (Best et al. 1995). R2 values for the fitted models also are presented. The models more accurately predicted abundances of more common species and generalist species than of rare, specialist species.

DISCUSSION Rank correlations suggested that model predictions were positively related to survey counts for all species except the Henslow’s Sparrow. However, the marginally positive correlations between predicted and observed abundances suggest that predictive models of grassland bird abundance will provide managers with little information on the abundance of grassland birds in 800 ha areas. In addition, percent grassland, a measure of habitat quantity, was slightly more correlated with observed abundance than predicted abundances generated by the models for two species and only slightly less correlated for the other five species. Thus, the simple measure of percent grassland performed equally well at predicting relative species abundance in 800 ha areas as the spatially explicit models, leading to the conclusion that the models have limited use for guiding management and conservation decisions in the region at 800 ha resolution. Independent data from areas surveyed in three years showed better correlation between observed and predicted abundances than data from areas surveyed in only a single year. Link et al. (1994) also found that repeated surveys were more useful than surveying additional sites when the strength of the correlation was as important as the direction of the relationship. Therefore, future efforts to assess accuracy of predictive models of bird abundance should incorporate surveys conducted over more than one year. As has been found in other studies (Garrison and Lupo 2002, Karl et al. 2002, Kadmon et al. 2003), predicted abundances were more accurate for more abundant species in our study. The relationship between abundance and accuracy might be a result of habitat being more saturated by abundant species than by rarer species. Unsaturated habitat could bring about prediction

15_MS8610.indd 753

errors in areas where habitat similar to that in used areas is available but not currently occupied (Fielding and Bell 1997), which could decrease predictive accuracy for rarer species. Our finding of increased predictive accuracy for more generalist species is contrary to the relationship found by others (Hepinstall et al. 2002, Brotons et al. 2004). These studies used several woody vegetation classes to model occurrence of bird species associated with woody vegetation. Thus, occurrences of specialists associated with particular woody vegetation types were more accurately predicted because they occurred in fewer vegetation types. In contrast, the lack of ability to distinguish among grassland types in the National Land Cover Dataset (Thogmartin, Gallant et al. 2004) did not allow for specialist grassland species to be associated with particular grassland types in the models we tested. Species abundance explained more variation in model accuracy than niche breadth. However, it is difficult to determine the contribution of each of these factors to model accuracy because abundance and niche breadth were correlated. Still, our results suggest that predicting abundances of rare habitat specialists is difficult given the available data. Thogmartin et al. (2006) identified two potential shortcomings of their modeling process that would affect accuracy of predicted abundances. First, although environmental variables were measured at multiple scales in an attempt to bound relevant biological processes, the scales used may not have been biologically relevant (Wiens 2002). Abundances of grassland birds in other studies were better explained by habitat variables measured in 150 ha areas than at larger extents (Murray et al. 2008), and habitat composition and structure within grasslands are known to be important factors in determining habitat use by grassland birds (Sample 1989, Herkert et al. 1993, Madden et al. 2000). In contrast, Cunningham and Johnson (2006) showed that grassland bird abundances were related to landscape features at larger scales (~1000–3000 ha) similar to those used by Thogmartin et al. (2006). Second, errors in the classification of remotely sensed data likely influenced model accuracy. In the upper Midwest, pixels classified as grassland-herbaceous in the 1992 National Land Cover Dataset were assigned correctly only 33% of the time, and the correct classification rate for pixels identified as pasturehay was only 16% (Environmental Protection Agency 2006). Thogmartin et al. (2006) attempted to control for this classification error by combining classes (Thogmartin, Gallant et al. 2004), but errors in the predictive models might still be a result of the large error rates in categorizing grassland habitats. In addition, the abundance data and land-cover data from different time periods used to develop the models might have caused errors in predicted abundances. Abundances used to create the predictive models were measured over a 20-year period and related to land-cover data from a single year. The availability of habitat along a survey route could have changed over these two decades, and high or low abundances during

11/28/08 6:01:08 PM

754   LES D. MURRAY

et al .

this time period could be falsely associated with habitat that did not exist at the time of high or low abundances. Trends in species abundance over time also could create errors in model predictions if changes in population size over time result in differential use of habitat (e.g., densitydependent habitat selection). Errors associated with differential habitat selection over time would be especially important if habitat selection changed between the time that data used in the models were collected and when the model results were evaluated. There is no evidence of substantial errors from differential habitat use over time in our study, as accuracy of predictions from our independent data was similar to that of predictions from a subset of the original data (Thogmartin et al. 2006). Inaccuracies in our models also were likely associated with the sampling design used to collect the independent data. First, the three survey stops might not have been representative of the entire 800 ha area if survey points were randomly located in areas with habitat not representative of the surrounding landscape. Second, predicted abundances were associated with counts made at 50 stops, but independent data were collected at only three stops. Thus, low observed abundances in areas of high predicted abundances could have been a result of the low probability of detecting a species at only three stops. In addition, large numbers of zero counts would lead to high variation in expected counts for a route with only three stops. Third, three years of data collection would not yield results as stable as the 20-year mean used in the models and could have biased our accuracy assessment. Despite the low probability of detecting birds along a route with only three survey points, relatively high abundances of most species were detected in areas with low predicted abundances. These underpredictions could have resulted from the steps used to map predicted abundances. The mapping process required calculating environmental variables within grids of the appropriate scales and using these values as predictors in the statistical models. Thus, the resulting maps did not represent the environmental variables centered on each 1 ha pixel. In addition, pixels classified as habitat not used by grassland birds were assigned abundances of zero during the mapping process, even though the surrounding landscape might have contained suitable habitat, which would have caused under prediction. In conclusion, several potential biases and modeling complications may have resulted in predicted abundances that for most species were only marginally positively correlated with observed abundances at 800 ha extents in the Prairie Hardwood Transition. In addition, percent grassland in an area performed nearly equally well at predicting relative abundances of grassland birds as the spatially predictive models of bird distributions. Consequently, use of the model predictions to guide management decisions is limited, and provides little additional insight compared to simple measures of habitat quantity. In addition, the model predictions were less correlated with observed

15_MS8610.indd 754

abundances for rarer and specialist species, showing that improved approaches to predict abundances of these species are required. Future efforts to model the distributions of grassland birds would benefit from a better understanding of habitat selection by and the effects of climate on these species, particularly the scales at which these variables are of biological significance. More accurate classification of grassland habitat from remotely sensed data and the ability to differentiate among grasslands based on vegetation structure and management activities also would benefit future modeling efforts. In lieu of advances in remotely sensed data, there is the potential to use existing data, such as the National Resources Inventory (Natural Resources Conservation Service 2008), to describe grassland management and structure in future models. ACKNOWLEDGMENTS We thank S. Craven, R. Fletcher, D. Johnson, D. Mladenoff, S. Temple, J. Zhu, and two anonymous reviewers for helpful comments on earlier versions of our manuscript and feedback on study design and analyses. We also thank M. Guzy, D. Johnson, D. Sample, J. Sauer, and S. Vos for their contributions to study design and analyses. We are grateful to all the volunteers that made this project possible. Funding was provided by the U.S. Geological Survey Upper Midwest Environmental Science Center and the U.S. Geological Survey Wisconsin Cooperative Wildlife Research Unit. We thank the Department of Forest and Wildlife Ecology, University of Wisconsin, Madison, for assistance with publication expenses. Mention of trade names or commercial products does not imply endorsement by the U.S. Government.

LITERATURE CITED Best, L. B., K. E. Freemark, J. J. Dinsmore, and M. Camp. 1995. A review and synthesis of bird habitat use in agricultural landscapes of Iowa. American Midland Naturalist 134:1–29. Brändle, M., and R. Brandl. 2001. Distribution, abundance and niche breadth of birds: scale matters. Global Ecology and Biogeography 10:173–177. Brotons, L., W. Thuiller, M. B. Araújo, and A. H. Hirzel. 2004. Presence-absence versus presence-only modeling methods for predicting bird habitat suitability. Ecography 27:437–448. Burnham, K. P., and D. R. A nderson. 2002. Model selection and multimodel inference: a practical information-theoretic approach. 2nd ed. Springer-Verlag, New York. Cunningham, M. A., and D. H. Johnson. 2006. Proximate and landscape factors influence grassland bird distributions. Ecological Applications 16:1062–1075. Environmental Protection Agency [online]. 2006. 1992 National Land Cover Data Set accuracy. (24 May 2006). Fielding, A. H., and J. F. Bell. 1997. A review of methods for the assessment of preciction errors in conservation presence/absence models. Environmental Conservation 24:38–49. Frawley, B. J., and L. B. Best. 1991. Effects of mowing on breeding bird abundance and species composition in alfalfa fields. Wildlife Society Bulletin 19:135–142. Garrison, B. A., and T. Lupo. 2002. Accuracy of bird range maps based on habitat maps and habitat relationship models, p. 367– 375. In J. M. Scott, P. J. Heglund, and M. L. Morrison [eds.],

11/28/08 6:01:09 PM

ACCURACY OF GRASSLAND BIRD ABUNDANCE MODELS   755

Predicting species occurrences: issues of accuracy and scale. Island Press, Washington, DC. Gaston, K. J., T. M. Blackburn, and J. H. Lawton. 1997. Interspecific abundance–range size relationships: an appraisal of mechanisms. Journal of Animal Ecology 66:579–601. Hepinstall, J. A., W. B. K rohn, and S. A. Sader. 2002. Effects of niche width on the performance and agreement of avian habitat models, p. 593–606. In J. M. Scott, P. J. Heglund, and M. L. Morrison [eds.], Predicting species occurrences: issues of accuracy and scale. Island Press, Washington, DC. Herkert, J. R. 1995. An analysis of midwestern breeding bird population trends: 1966–1993. American Midland Naturalist 134:41–50. Herkert, J. R., R. E. Szafoni, V. M. Kleen, and J. E. Schwegman. 1993. Habitat establishment, enhancement, and management for forest and grassland birds in Illinois. Illinois Department of Conservation Division of Natural Heritage Technical Publication No. 1. Johnson, D. H. 2001. Validating and evaluating models, p. 105–119. In T. M. Shenk and A. B. Franklin [eds.], Modeling in natural resource management: development, interpretation, and application. Island Press, Washington, DC. Johnson, F. L. 1986. Oak-hickory savannahs and transition zones: preservation status and management problems, p. 345–347. In D. L. Kulhavy and R. N. Conner [eds.], Wilderness and natural areas in the eastern United States: a management challenge. Stephen F. Austin State University, Nacogdoches, TX. Johnson, R. G., and S. A. Temple. 1990. Nest predation and brood parasitism of tallgrass prairie birds. Journal of Wildlife Management 54:106–111. K admon, R., O. Farber, and A. Danin. 2003. A systematic analysis of factors affecting the performance of climatic envelope models. Ecological Applications 13:853–867. K arl, J. W., L. K. Svancara, P. J. Heglund, N. M. Wright, and J. M. Scott. 2002. Species commonness and the accuracy of habitatrelationship models, p. 573–580. In J. M. Scott, P. J. Heglund, and M. L. Morrison [eds.], Predicting species occurrences: issues of accuracy and scale. Island Press, Washington, DC. K nopf, F. L. 1994. Avian assemblages on altered grasslands. Studies in Avian Biology 15:247–257. K nutson, M. G., G. Butcher, J. Fitzgerald, and J. Shieldcastle. 2001. Partners in Flight bird conservation plan for the upper Great Lakes plain (Physiographic Area 16). U.S. Geological Survey, Upper Midwest Environmental Sciences Center, and Partners in Flight, La Crosse, WI. Link, W. A., R. J. Barker, and J. R. Sauer. 1994. Within-site variability in surveys of wildlife populations. Ecology 75:1097–1108. M adden, E. M., R. K. Murphy, A. J. H ansen, and L. Murray. 2000. Models for guiding management of prairie bird habitat in northwestern North Dakota. American Midland Naturalist 144:377–392. M arsden, S. J., and M. Whiffin. 2003. The relationship between population density, habitat position and habitat breadth within a Neotropical forest bird community. Ecography 26:385–392. Mayer, D. G., and D. G. Butler. 1993. Statistical validation. Ecological Modelling 68:21–32. McCoy, T. D., M. R. Ryan, L. W. Burger Jr., and E. W. Kurzejeski. 2001. Grassland bird conservation: CP1 vs. CP2 plantings in

15_MS8610.indd 755

Conservation Reserve Program fields in Missouri. American Midland Naturalist 145:1–17. Murray, L. D. 2006. Predicting bird abundance and occurrence in midwestern landscapes. Ph.D. dissertation, University of Wisconsin, Madison, WI. Murray, L. D., C. A. R ibic, and W. E. Thogmartin. 2008. Relationship of obligate grassland birds to landscape structure in Wisconsin. Journal of Wildlife Management 72:463–467. Natural R esources Conservation Service [online]. 2008. National Resources Inventory. (28 October 2008). Nuzzo, V. A. 1986. Extent and status of Midwest oak savanna: presettlement and 1985. Natural Areas Journal 6:6–36. Patterson, M. P., and L. B. Best. 1996. Bird abundance and nesting success in Iowa CRP fields: the importance of vegetation structure and composition. American Midland Naturalist 135:153–167. Peterjohn, B. G., and J. R. Sauer. 1999. Population status of North American grassland birds from the North American Breeding Bird Survey, 1966–1996. Studies in Avian Biology 19:27–44. R Development Core Team [online]. 2006. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. (28 October 2008). Sample, D. W. 1989. Grassland birds in southern Wisconsin: habitat preference, population trends, and response to land use changes. M.Sc. thesis, University of Wisconsin, Madison, WI. Sample, D. W., and M. J. Mossman. 1997. Managing habitat for grassland birds: a guide for Wisconsin. Wisconsin Department of Natural Resources Publication SS-925-97. Temple, S. A., B. M. Fevold, L. K. Paine, D. J. Undersander, and D. W. Sample. 1999. Nesting birds and grazing cattle: accommodating both on Midwestern pastures. Studies in Avian Biology 19:196–202. Thogmartin, W. E., E. A. L. Gallant, M. G. K nutson, T. J. Fox, and M. J. Suárez. 2004. A cautionary tale of the National Land Cover Dataset 1992. Wildlife Society Bulletin 32:960–968. Thogmartin, W. E., M. G. K nutson, and J. R. Sauer. 2006. Predicting regional abundance of rare grassland birds with a hierarchical spatial count model. Condor 108:25–46. Thogmartin, W. E., J. R. Sauer, and M. G. K nutson. 2004. A hierarchical spatial count model with application to Cerulean Warblers. Ecological Applications. 14:1766–1779. Transeau, E. 1935. The prairie peninsula. Ecology 16:423–427. U.S. North American Bird Conservation Committee. 2000. North American Bird Conservation Initiative: bird conservation region descriptions. U.S. Fish and Wildlife Service, Arlington, VA. Vogelmann, J. E., S. M. Howard, L. Yang, C. R. Larson, B. K. Wylie, and N. Van Driel. 2001. Completion of the 1990s National Land Cover Data Set for the conterminous United States from Landsat Thematic Mapper data and ancillary data sources. Photogrammetric Engineering and Remote Sensing 67:650–662. Wiens, J. A. 2002. Predicting species occurrences: progress, problems, and prospects, p. 739–749. In J. M. Scott, P. J. Heglund, and M. L. Morrison [eds.], Predicting species occurrences: issues of accuracy and scale. Island Press, Washington, DC.

11/28/08 6:01:10 PM