Fish Sci (2011) 77:467–478 DOI 10.1007/s12562-011-0355-6
ORIGINAL ARTICLE
Fisheries
An adaptive sampling method based on optimized sampling design for fishery-independent surveys with comparisons with conventional designs Yong Liu • Yong Chen • Jiahua Cheng Jianjian Lu
•
Received: 11 January 2011 / Accepted: 4 April 2011 / Published online: 1 May 2011 Ó The Japanese Society of Fisheries Science 2011
Abstract The adaptive cluster sampling method is widely applied in terrestrial systems; however, it is not suitable for fisheries surveys because of the high cost of unlimited sampling in practice. An adaptive approach is often used in fisheries surveys to allocate sampling effort, usually following a stratified random design. Development of an adaptive sampling method based on optimized sampling design (this design has been suggested by previous study for fishery-independent surveys) has been not yet carried out. An adaptive sampling method based on optimized sampling design using the criterion of minimization of the mean of the shortest distance (MMSD) in the first phase was constructed in this study and compared with five other sampling designs: simple random, stratified random, adaptive based on stratified sampling, systematic, and optimum design based on the MMSD criterion. This design performed neither the best nor the worst among the six sampling designs considered in this study, but its advantages were obvious when the sampling effort saved using this design was considered in the comparison. This method tends to be more flexible and find fish aggregations more precisely. It is based on a more objective sampling design in the first phase compared with other Y. Liu (&) J. Lu The State Key Laboratory of Coastal and Estuarine Research, East China Normal University, Shanghai 200062, China e-mail:
[email protected] Y. Liu J. Cheng Key and Open Laboratory of Marine and Estuarine Fisheries Certificated by the Ministry of Agriculture, East China Sea Fisheries Institute, Chinese Academy of Fishery Sciences, Shanghai 200090, China Y. Chen School of Marine Sciences, University of Maine, Orono, ME 04469, USA
adaptive sampling designs based on stratified sampling designs. We suggest that this design be considered in developing fishery-independent survey programs. Keywords Adaptive sampling design Fishery-independent survey Optimized sampling design Sampling design comparison
Introduction Adaptive sampling designs can cope with the reality that few organisms are distributed absolutely evenly and randomly over a large spatial scale on the Earth and that aggregations or patches of organisms of interest always occur in the real world. It is logical that we would like to be adaptive in the sampling process to allocate more effort in areas in which high density of organisms of interest is observed in the survey [1]. A number of adaptive methods have been developed in the past for adding samples during a survey to increase sampling precision [1–4]. A well-known adaptive sampling method, often referred to as the adaptive cluster sampling method, has been widely applied to terrestrial systems in fields such as ecology, geology, and epidemiology [1]. Adaptive cluster sampling usually follows the following procedure: (1) an initial probability for sampling is determined prior to the survey, (2) during sampling, when an observed value of a selected unit satisfies a predefined condition of interest, additional units are added to the sample from the neighborhood of that unit, and (3) if any of these additional units satisfy the predefined condition, more units may be added [1, 5]. There are usually no limits for the number of sample units selected in such an adaptive sampling design since the additional sample units often do not increase the costs
123
468
substantially for surveys on land. However, for surveys in marine systems, the cost of each additional sample is high and needs to be considered. Thus, few marine fishery surveys can afford the high cost of an unlimited number of sample sites, which may explain why few adaptive cluster sampling methods have been applied in fisheries surveys, although they have been tested and assessed by simulation [6, 7]. Although many methods have been developed to optimize sample sizes, the process tends to be different from that for a typical adaptive cluster sampling method [1, 4, 5, 8, 9]. For fisheries surveys, an alternative approach is to use an adaptive allocation scheme within a stratified design [10– 13]. With this method, all of the strata are assigned a firstphase sample size. A second phase of random samples is added to the strata only if the appropriate conditions are met by the observations in the first phase. Francis [10] proposed a two-phase survey strategy for a trawl survey in which allocation of trawl stations in phase 2 is based on survey catches obtained in phase 1, which can reduce the skewness in the biomass estimate as well as reducing the expected error [13]. Jolly and Hampton [11] proposed another twophase approach for acoustic surveys, in which the transect allocation was adjusted during the survey for differences between the expected and observed distributions, which also can increase the precision of the estimation compared with other sampling designs [11, 12]. Although these two adaptive allocation sampling approaches can be used for different surveys, they are both based on stratified designs. To our knowledge, almost all the sampling designs with adaptive effort allocation are based on stratified sampling design in the first phase. However, according to Liu et al. [14], stratified design may not be the best for a fisheryindependent survey under some special conditions. Our results suggest that sampling design based on the criterion of minimization of the mean of the shortest distance (MMSD) [15] was the best [14]. Thus, two questions can be raised: (1) whether an adaptive sampling design based on the MMSD criterion in the first phase is better for fishery-independent surveys for fish abundance estimation, and (2) how the adaptive sampling method can be added to the survey design based on the MMSD criterion. To answer these two questions, we present a new adaptive allocation method for individual sampling design, and compare the precision of parameters and abundance estimation with five other sampling designs: simple random, stratified random, stratified design based adaptive, systematic, and optimum design based on the MMSD criterion.
Materials and methods This section includes the following five parts: (1) the material for calculation of the spatial characteristic
123
Fish Sci (2011) 77:467–478
parameters, (2) the methods for the six sampling designs in this study, (3) the methods of geostatistics used in this study, (4) the method of simulation, and (5) the method of analyzing results. Study area and data collection The study area covers from 27°000 N to 30°000 N latitude and from 120°000 E to 127°000 E longitude in the East China Sea (Fig. 1). The data, used to derive the original spatial autocorrelation parameters of Setipinna taty distribution, were obtained from a fishery-independent bottom trawl survey during 2000–2007. Details of the study area and the survey can be found in Liu et al. [14]. Sampling designs considered in the study This study is an extension of Liu et al. [14], and two sampling designs included in this study, i.e., systematic sampling design (referred to as design IV in this study) and optimized sampling design (referred to as design V in this study), were described in Liu et al. [14]. In addition to these two designs, we also considered another four designs in this study. Simple random sampling design (design I) We include simple random sampling in this study because it is the most basic design. Eighty-two sites were created randomly in the defined study area (Fig. 1). The simple random sampling design is referred to as design I in this study. Stratified random sampling design (design II) This design has good support from classic statistics theory [16], and is applied widely in fishery-independent surveys [12, 17]. In Liu et al. [14], four strata were defined over the survey area according to depth: less than 40, 40–60, 60–80 m, and deeper than 80 m. Because analysis of historical data suggests that both stratum area and distance from the coast were important in determining the spatial distribution of Setipinna taty [18, 19], the number of sampling stations in each stratum was determined by both of these variables. To prepare a suitable quantity of strata for the adaptive sampling design based on the stratified sampling in the first phase, different separations of the former strata were made by dividing a stratum into several substrata with the same areas. With the above separation, the number of strata was changed from 4 to 11. The determination of the number of sampling sites in each stratum was also based on the previous four-stratum design, averaging the sites of each former stratum into several substrata. The total number of sites in this design is
Fish Sci (2011) 77:467–478
469 120
Des.I: Simp. Rand. (Size:82)
122
124
126
Des.II: Stra. Rand. (Size:82)
128 Des.III: Adapt. || Stra.Rand. (Size:50) 36
34
32
Latitude
30
Des.IV: Syst. (Size:82)
Des.V: Opti.-MMSD (Size:82)
Des.VI: Adapt. || Opti.-MMSD (Size:50)
36
34
32
30
120
122
124
126
128
120
122
124
126
128
Longitude Fig. 1 Spatial distribution of sampling stations from the four sampling designs and the two first-phase sampling designs for adaptive sampling designs (with layers in dotted line in designs II and III). Design I, simple random sampling design; Design II, stratified random sampling design; Design III, the first-phase sampling design
of the adaptive sampling design based on stratified random sampling; Design IV, systematic sampling design; Design V, MMSD optimized sampling design; Design VI, the first-phase sampling design of the adaptive sampling design based on MMSD optimized sampling
82 (Fig. 1). The stratified random survey design is referred to as design II in this study.
Gi ¼ A2i Vi =ðni ðni þ 1ÞÞ;
Adaptive sampling design for stratified sample based survey (design III) This adaptive sampling design, described in Francis [10], is included in this study to compare the efficiencies of the old and new adaptive sampling designs. In the first phase, 50 sites were allocated into the 11 strata described in the previous sampling design (Fig. 1), with the number of sites in each stratum in proportion to that of the previous design with 82 sites. Slight adjustment was applied after the proportional site allocation to ensure that a minimum of 3 sites was allocated to each stratum [10]. The method for adding and allocating sites in the second phase was the same as the method described in Francis [10]. The equation we used for measuring relative gain (i.e., reduction in variance) G is:
ð1Þ
where Ai is the area of stratum i, and ni and Vi are the number of stations and the variance of the catch rate (catch weight/distance trawled), respectively, for the phase 1 survey in the stratum. The process of adding one site is based on the estimated relative gain (reduction in variance) G for all stratum, and the site is added to the stratum with the highest value of G in each allocation [10]. This process needs to be repeated for 32 times in this sampling design. This adaptive sampling design is referred to as design III in this study. Systematic sampling design (design IV) In this design, the sampling stations are distributed evenly with equal intervals between each pair of stations throughout the survey area (Fig. 1). The systematic sampling design is referred to as design IV in this study.
123
Fish Sci (2011) 77:467–478
In the adaptive allocation sampling, an initial sampling design with sample size of 50 optimized by the MMSD criterion was obtained. During the survey based on the initial (first phase) sampling design, if the observed values of some sites satisfy a predefined condition, a number of additional sites are sampled around this site. In this study, there is just one process of adding sites for the initial sampling design, and there is no further process for the added sites that satisfy the condition. The predefined condition was calculated based on the survey data collected during the period 2000–2007 (Fig. 2). The 80% and 90% quantiles of the abundance indices for all years were calculated and averaged to provide the thresholds against which the observed values of sites were evaluated to determine the necessity of adding additional sites in neighboring areas. The averages for the 80% and 90% quantiles resulted in thresholds of 305 and 859 individuals per hour, respectively (indicated by the dotted line in Fig. 2). To determine the number of sites to be added for each threshold, the average number of sites between the 80% and 90% quantiles and above the 90% quantile were calculated, being 5 and 4, respectively. To obtain a final sample size with the same or similar number of sites as the other sampling design (82 sites) based on a first phase of sampling with 50 sites, the numbers of sites added for the two thresholds were set to 3 and 5, respectively. In this way the expected average number of sites for the final sampling is equal to 50 ? 5 9 3 ? 4 9 5 = 85, which is close to
123
0.01 0.00
1622 at 90%
846 at 80%
138 at 80%
0.01 0.00
206 at 90% 163 at 80%
0.01 0.00
170 at 80%
500 305
985 at 90%
2006
0.01 0.00
1088 at 90%
2005
316 at 80%
2007
0.01 0.00
Mean:
2001
816 at 90%
170 at 80%
2004
0.01 0.00
2002
824 at 90%
236 at 80%
0
Adaptive allocation sampling design for individual site based survey (design VI)
1159 at 90%
494 at 80%
0.01 0.00
Density
The MMSD criterion was used as the objective function for this sampling design. The MMSD criterion requires that all sampling points be spread regularly over the sampling region, within which the expectation of the distances between an arbitrarily chosen point and its nearest sampling point is minimal [15]. A constrained spatial simulated annealing (CSSA) method was used as an effective optimization approach to achieve the final sampling design. Simulated annealing (SA), a generic probabilistic approach for finding an approximation to the global optimum of a given objective function, is referred to as spatial SA (SSA) when it is adopted for spatial sampling. The SSA optimization method, incorporating an effective way of finding a better sampling scheme by treating sampling points as continuous variables rather than chosen from a discrete grid, is called CSSA [15]. Details of the criterion description and the process of optimization can be found in Liu et al. [14]. The optimized sampling design is referred to as design V in this study.
0.01 0.00
2000
Optimized sampling design (design V)
2003
470
816 at 90%
1000 859
1500
2000
CPUE/ind. per hour Fig. 2 The predicted thresholds for design VI based on the frequency of abundance capture per hour for Setipinna taty in survey from 2000 to 2007 with the values at the quantiles of 80% and 90%
82. However, the actual number of sites for this adaptive sampling is not fixed and likely to vary around 85, depending on the observed values of sites sampled in the survey. In the simulation study, five sites were added around the sites with abundance indices above the threshold of the 90% quantile (high reference point), and three were added for those between the 80% and 90% quantile thresholds (low reference point). The process of adding sites was as follows: when a site with value higher than a reference point was found, the nearest-neighbor sites were identified. For a site with value more than the high reference point, five nearest-neighbor sites were selected. If the site was inside the polygon constructed by these five neighboring sites (case 1 in Fig. 3), triangles were constructed from each pair of adjacent nearest-neighbor sites plus the observed site (the grey point; Fig. 3) and five new sites were added at the centers of gravity of these triangles (circles with a dot in the center; Fig. 3). If the observed site was outside of the polygon (case 2 in Fig. 3), the observed site was located to one side of all the neighboring sites, for example, when a site was located at an edge of the survey area. In this case, one extra nearest-neighbor site, compared with the above situation, was added. In this way, five triangles could still be formed, and five new sites added at their centers of
Fish Sci (2011) 77:467–478
471
34
4 3 1
6
32
5
Y NðDb; r2 Rð/Þ þ s2 IÞ;
2
28
30
Longitude
36
parameter. In practice, there is always another type of independent variance in observed data, which is usually referred to as the nugget effect s2. In geostatistical terminology, s2 ? r2 is the total sill, r2 is the partial sill, s2 is the nugget effect, and 3/ is the practical range [21]. Estimation of these geostatistical parameters was by the following methods. The geostatistical multivariate Gaussian model can be written as [22]
120
122
124
126
128
Latitude
ð2Þ
where D is an n 9 p matrix of covariates, b is the corresponding vector of regression parameters, and R is the correlation function, which depends on the correlation range parameter u. The product of the two parameters D and b describes a linear specification for the spatial trend, l(x). The log-likelihood function is Lðb; s2 ; r2 ; /Þ ¼ 0:5fn logð2pÞ þ logfjðr2 Rð/Þ þ s2 IÞjg
Fig. 3 Illustration of the process of adding sites in the second phase for the adaptive sampling based on optimization sampling in the first phase
þ ðy DbÞT ðr2 Rð/Þ þ s2 IÞ1 ðy DbÞg:
gravity. In a final step, it was necessary to check whether there were points among the centers of gravity that were too close to the selected site or selected neighbors. This was done by comparison against a limit set at 1/4 of the average distance from each point to its nearest-neighbour points in the first-phase sampling design. If a distance was found to be too close according to this definition, the resulting center of gravity was deleted (case 3 in Fig. 3, one empty circle very close to the selected grey point). For the low reference point, the method of adding three new sites was similar to the procedure described above for the high reference point (cases 4–6 in Fig. 3). This adaptive sampling design is referred to as design VI in this study.
The model parameters can be estimated through maximization of the likelihood function. In the simulation study, two kinds of geostatistical simulation methods were used: unconditional and conditional simulation. Simulating a realization (unconditional simulation) is a key aspect of the simulation. Y can be represented by Y = l ? S ? sT, where l = E[Y], T ¼ ðT1 ; . . .; Tn Þ is a set of mutually independent random variables following the standard Gaussian distribution N(0,1), and S ¼ ðS1 ; . . .; Sn Þ follow a zero-mean multivarP iate Gaussian distribution, S MVNð0; Þ. The realization of S can be obtained through a linear transformation: P S = AZ, where A is a matrix such that AA0 ¼ , and Z ¼ ðZ1 ; . . .; Zn Þ follow N(0,1) [22]. Conditional simulation is the simulation of a spatial process S(x) at location xi : i ¼ 1; . . .; N, conditional on observed values S(x) at location xj : j ¼ 1; . . .; n ðj 6¼ iÞ, which is used to assess the abundance in this study.
Geostatistical model framework Since original observations usually cannot satisfy the Gaussian stochastic process in practice, pretransformation of the data is necessary. A log-Gaussian geostatistical model was used in this paper, which requires log transformation of the original data, a typical Box–Cox Gaussian transformation [20]. The stationary Gaussian process was applied in this study, which is defined by two statistics: the mean of every location x, which is a constant, l(x) = l, and the covariance, which only depends on the distance between x and x0 , i.e., c(x, x0 ) = c(h), where h is the Euclidean distance between locations x and x0 . The variance of the stationary process is a constant, r2 = c(0), and the correlation function is defined as q(h) = c(h)/r2, which is described by an exponential function in this study, written as qðhÞ ¼ expðh=/Þ, where / is the correlation range
ð3Þ
Simulation procedure The simulation procedure included three steps. The first step was to geostatistically simulate the realization (unconditional simulation) based on the spatial autocorrelation parameters described above, which was considered as the ‘‘real population.’’ The second step was to draw samples from the ‘‘real population’’ according to the sampling designs. The third step was to assess abundance based on the above drawn samples using the geostatistical method (conditional simulation). The above two simulations were run for 200 and 100 times, respectively [14].
123
472
Fish Sci (2011) 77:467–478
Analysis of results
Table 1 Exponential covariance function parameters estimated for Setipinna taty for survey years 2000–2007
Two measures, the same as those used in Liu et al. [14], were used for comparison of the simulation results. The first measure, relative standard error (RSE), evaluates the accuracy and precision of estimated parameters for covariance functions. The RSE is calculated as sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi n X RSE ¼ ðViestimated V true Þ2 =n=V true ; ð4Þ
Year
b
r2
/
s2
2000
3.0592
7.2142
2.0170
0.0000
2001
2.7871
3.7746
1.3539
1.6692
2002
3.1624
5.5360
0.5943
0.0000
2003
4.1283
4.8187
0.7080
0.0000
2004
2.8841
5.7039
1.2152
0.0128
2005
2.9405
6.3422
0.8233
0.0000
2006 2007
2.8729 2.4368
3.5129 3.7789
0.3110 0.9685
0.0000 0.0000
i¼1
where Viestimated is the value of parameters estimated from the ith unconditional simulation (for a given sampling design in a given year), and V true is the parameter value derived from the original data (for a given year). The second measure, relative absolute error (RAE), evaluates the accuracy of stock abundance estimates. The RAE was calculated as RAE ¼ jV estimated V true j=V true ;
ð5Þ
where V estimated is the average of 100 conditional estimates of stock size based on the sampled data from unconditional simulation, and V true is the assumed ‘‘true’’ value of stock size derived from the unconditional simulation. To evaluate whether, on average, the estimator is lower or higher than the true value, a directional measure, relative error (RE), was used. The equation for RE is similar to that for RAE, but without using absolute difference, i.e., RE ¼ ðV estimated V true Þ=V true :
ð6Þ
similar in nature, we considered them as the ‘‘random group’’ in the next analysis. Design IV (systematic design) led to good and even distribution in the center of the survey area (Fig. 1), but not a good distribution at some edges of the survey area with a few blank areas of different sizes. Under the same optimal criterion function, both design V and the first phase of design VI led to good and even coverage of the whole survey area including the edges. Because there were fewer sites in the first phase of design VI compared with design V, the average distances of points were larger than those of design V. The distributions of designs IV–VI were relatively more even than those of designs I–III. Thus, we considered these three designs as the ‘‘even group.’’ Comparing the RSEs of estimated covariance function parameters
Results Exponential covariance function parameters for the survey Using survey data for the 8 years, parameters quantifying the characteristics of the spatial distribution of Setipinna taty in the north of East China Sea were estimated to simulate the fish distribution in the study (Table 1). The mean of r2 (5.0852) was the largest, and the mean of / (0.9989) was the smallest; the variation of r2 (1.3452) was the largest, and the variation of b (0.4913) was the smallest. Four sampling designs and two first-phase sampling designs for adaptive sampling Designs I and II and the first phase of design III shared common characteristics; i.e., uneven and randomized sites were distributed over the study area, with some areas being not well covered. However, design II had relatively better coverage than design I. Because these three designs were
123
For parameter b estimation, the smallest value of RSE (implying the highest accuracy and precision) was derived for design V, followed by designs IV and VI, and design III had the largest RSE. RSEs of the even group seemed to be less than those of the random group. The smallest values of RSE were derived for designs I and III, respectively, for parameters r2 and /, both belonging to the random group. The largest RSE values were found for designs IV and VI, respectively, for parameters r2 and /, both belonging to the even group (Table 2). Designs for the even group seemed to be good for estimating parameter b, and designs in the random group seemed to be good for estimating parameters r2 and /. Within the even group, design VI had the highest RSE for parameter b but the lowest RSE values for parameters r2 and /. Comparing the RAEs of estimated stock sizes Design IV yielded the estimates with the smallest RAE value, implying that it had the highest accuracy (Table 2),
Fish Sci (2011) 77:467–478
473
Table 2 Mean and standard deviation of relative standard error (RSE) for the three parameters of covariance function, relative absolute error (RAE) for stock size, and relative error (RE) for both simulations Measure
Object
Item
Design I
b
II
III
IV
V
VI
Mean
0.2750
0.2752
0.2785
0.2683
0.2650
SD
0.1114
0.1099
0.1070
0.1099
0.1087
0.1061
r2
Mean SD
0.3292 0.0944
0.3318 0.0821
0.3313 0.0739
0.3456 0.0654
0.3404 0.0777
0.3399 0.0638
/
Mean
0.4863
0.5063
0.4662
0.6637
0.6412
0.6294
SD
0.1018
0.0932
0.0762
0.1880
0.1966
0.2935
RAE
Abundance
Mean
0.2833
0.2908
0.2452
0.2339
0.2458
0.2504
RE
b
Mean
r2
Mean
RSE
SD
0.2729
0.0660
0.0455
0.0544
0.0557
0.0658
0.0676
-0.0045
-0.0043
20.0011
-0.0044
-0.0046
-0.0013
SD
0.2942
0.2937
-0.1087
-0.1175
SD
0.296 20.1086
0.2872 -0.1319
0.2839 -0.125
0.2902 -0.1279
0.3249
0.3224
0.3216
0.3402
0.3386
0.3309
-0.0415
-0.0422
-0.0471
0.0294
0.0082
-0.0119
/
Mean SD
0.4949
0.5133
0.4699
0.6863
0.6665
0.6846
Abundance
Mean
0.0776
0.0565
0.0571
0.0488
0.0498
0.0607
SD
0.3838
0.394
0.3287
0.3186
0.3393
0.3465
Sample size
82
82
82
82
82
69a
The smallest mean of RSE, RAE, and absolute RE for each parameter among five designs is in bold, and the largest is italicized and underlined. Design I, simple random sampling design; Design II, stratified random sampling design; Design III, the first-phase sampling design of the adaptive sampling design based on stratified random sampling; Design IV, systematic sampling design; Design V, MMSD optimized sampling design; Design VI, the first-phase sampling design of the adaptive sampling design based on MMSD optimized sampling a
Averaged sample size
followed by designs III, V, and VI. The estimates for design II were the worst. Designs in the random group, except for design III, tended to have higher RAEs compared with designs in the even group, indicating that designs in the even group were better for estimating stock sizes. Comparing REs of estimated stock sizes and covariance function parameters Almost all the average relative errors of the three parameters were less than zero for all the designs (Table 2), implying that the estimates of parameters tended to be smaller than the true values. However, all the average relative errors for stock sizes were more than zero, implying that stock sizes tended to be overestimated. An interesting phenomenon was that all the standard deviations of REs were proportional to the average RSEs of the three parameters and to the RAEs of the stock size for all the designs. Total sample size distribution in design VI for different years Although the total sample size designed for the adaptive sampling based optimized sampling was expected to be 85,
because of the high variation of the realized value of drawn samples in the simulation, there were differences in the total sample size for all the years (Fig. 4). The average total sample size ranged from 58 (in 2007) to 83 (in 2003), and the standard deviation (SD) of the total sample size by year ranged from 6.27 (in 2006) to 22.12 (in 2000). The highest total sample size appeared in 2000, which also had the highest SD.
Discussion Selection of measures for performance comparison Accuracy and precision are two measures used to describe experimental results. An accurate measurement is one in which the results of the experiment are in agreement with the ‘‘true’’ value. A precise measurement is one where the spread of results is ‘‘small’’ [14, 23, 24]. In this study, the RSE was used to summarize and compare both the accuracy and precision of parameter estimation for the covariance function [25, 26], the RAE was used to compare the accuracy of stock size estimation [27], and the RE was used to evaluate the direction of the bias in the estimation of parameters or abundance. However, it was interesting to
123
474
Fish Sci (2011) 77:467–478
Percent of Total
60 80 100 120 140 160
60 80 100 120 140 160
2000
2001
2002
2003
Range: 50 - 154 Mean: 73 SD: 22.12
Range: 50 - 122 Mean: 66 SD: 13.12
Range: 50 - 114 Mean: 71 SD: 11.98
Range: 53 - 130 Mean: 83 SD: 14.48
50 40 30 20 10 0
2004
2005
2006
2007
Range: 50 - 118 Mean: 66 SD: 14.7
Range: 50 - 109 Mean: 71 SD: 12.58
Range: 50 - 82 Mean: 60 SD: 6.27
Range: 50 - 90 Mean: 58 SD: 7.95
50 40 30 20 10 0 60 80 100 120 140 160
60 80 100 120 140 160
Total sample size Fig. 4 Frequency of total sample number of adaptive sampling based on the MMSD optimized sampling in the simulation for the 8 years from 2000 to 2007
find that the standard SDs of the REs for both parameters and abundance estimations were consistent with the RSE and RAE. Because the RSE and RAE were shown to be good overall measures of the performance of estimators [25, 27] and consistent with other measures in this study, we use these two measures as the focus of comparison in this discussion. Performance comparison of different designs Based on their similarity, the six designs considered in this study could be divided into two groups: the random group and even group. These two groups showed different efficiencies for estimation of parameters and abundances. For parameter estimation, the even group seemed to be better than the random group for estimation of b; however, the random group seemed to be better than the even group for the estimation of r2 and / (Table 2). For abundance estimation, the random group (except for design III) seemed to be worse than the even group. In Liu et al. [14], we found that the accuracy of abundance estimation was mainly determined by the key covariance parameter b. In this study, this was still true for the group, but not always correct for a single design; for example, b estimation using design III was the worst, but its abundance estimation precision was the 2nd best. Thus, we concluded that the quality of estimating the parameter b and other parameters was all important for the final abundance estimate. However, the detailed relationship between the quality of parameter estimation and quality of abundance assessment needs more study in the future.
123
Design VI had characteristics of both evenness (optimized sampling sites) and randomness (adaptive adding of sites) in the distribution of sites, and also had characteristics of both the even group and random group in the estimation. Of the even group, for parameter and abundance design VI had the highest RSE and RAE, similar to the characteristics of random group, whose RSE and RAE tended to be higher than that of even group. This might be the result of it being more random than the other two designs in the even group. For the other two parameters, r2 and /, more randomness caused it to have the lowest RSE among the designs in the even group, similar to the features of the random group, whose RSE tended to be less than that of the even group. Many previous studies have proved such a tendency of increasing accuracy with evenness of site distribution [14, 16, 28–30], which coincided with the differences in the estimation precision between the two groups for parameter b, but not for parameters r2 and /. The random group tended to perform better for estimation of parameters r2 and /. Parameter r2 is the variance in the stationary process and / is the correlation range over which there is a relationship between the variance and pair distance [21, 22]. Both of these parameters have direct or indirect relationship with the variation of the spatial distribution. Distinct samples have equal chance of being selected by the random sampling method; however, the selection of one sample could determine or affect the other samples when using an even (or systematic) sampling method [16]. Therefore, the random group could capture more spatial variations than the even group. This
Fish Sci (2011) 77:467–478
475
Comparison of the two adaptive sampling designs
may explain why the random group tended to be better at estimating the two variance-related parameters.
Two adaptive sampling designs were included in this study. One was based on stratified sampling design (design III), and the other one was based on optimized sampling design (design VI). Based on the simulation results, the performance of design III for estimation of parameters was found to be unstable, with the best estimation for parameter /, the worst estimation for parameter b, and a relatively good (2nd best) abundance estimate. For the yearly performance (Table 3), the adaptive sampling design based on stratified sampling was good, improving the performance by 5–29% in different years (compared with design II). It could be inferred that the strata in design III were appropriate for the fishery [16], and the adaptive sampling design based on stratified sampling was a good choice to improve the efficiency of sampling for survey of this population [10, 13]. The performance of design VI seemed to be ranked in the middle (with neither the best nor worst estimations), but the average sample size of design VI was 16% (13/82) less than the other designs, making it attractive because of relatively low cost.
Determination of the total sample size distribution for design VI The large differences in the total sample size distributions among the 8 years were obvious (Fig. 4). By comparing them with the parameters for the simulation (Table 1) and the threshold-setting process for the adaptive sampling (Fig. 2), some implication about the determination of the average total sample size and the standard deviation (Fig. 4) could be found. The average total sample size was related to the difference between the thresholds and the corresponding quantiles for each year. The largest average total sample size for 2003 implied that the 80% and 90% quantiles for 2003 were higher than the thresholds at the highest levels among the 8 years. The relatively smaller average total sample sizes for 2006 and 2007 suggested that the 80% and 90% quantiles for 2006 and 2007 were lower than the thresholds at the highest levels among the 8 years. Similarly, the standard deviation of the total sample size is related to the parameter r2 (Table 1) on which the simulation was based. The highest standard deviation of the total sample size for 2000 corresponded to the largest value of the parameter r2 in 2000; the smallest standard deviation of the total sample size for 2006 and 2007 corresponded to the smallest values of parameter r2 in 2006 and 2007. Both of these relationships were not completely proportional; however, the tendencies were obvious, which could give us some indications to control the total sample size and obtain stable sampling using the adaptive sampling design developed in this study.
Four categories of the performance of design VI The results discussed above were the average over the 8 years, but the performance of different designs might vary from year to year. Because our goal was to find a costeffective design with good performance and small sample size, we compared the performance between design V and design VI using the RAE and sample size for each year (Table 3). Their performance for the 8 years can be summarized into the following four categories: When the
Table 3 Comparison of relative absolute error (RAE) for stock size between nonadaptive and adaptive sampling designs for each simulation year Item
Design
Year 2000
Relative absolute error
Sample size
2002
2003
2004
2005
2006
2007
II
0.2695
0.3386
0.3497
0.2815
0.2688
0.3300
0.2751
III
0.1926
0.2902
0.3325
0.2296
0.2167
0.2893
0.2381
0.1730
Relative diff
0.29
0.14
0.05
0.18
0.19
0.12
0.13
0.19
Sample size Relative absolute error
2001
82 V
0.1887
VI
0.1676
Relative diff.
0.11
82 0.3499 0.3690 -0.05
82 0.2829
82 0.2232
0.2825
0.2146
0.00
0.04
82 0.1955 0.2115 -0.08
82 0.2961 0.2608 0.12
82 0.2755 0.3083 -0.12
0.2136
82 0.1544 0.1887 -0.22
V
82
82
82
82
82
82
82
82
VI
73
67
71
83
66
71
60
58
Relative diff.
0.11
0.19
0.13
-0.02
0.19
0.14
0.27
0.30
There are two pairs in this comparison: design II versus design III, and design V versus design VI; the value of ‘‘Relative diff.’’ was calculated by the following equation: [V(adpt.) - V(non-adpt.)]/V(non-adpt.)
123
476
thresholds were less than the true values, which resulted in relatively large sample size, the performance of design VI tended to be normal, without any improvement: the sample size was similar to or larger than the sample sizes for the other designs, and the precision was also similar to that for the basic design or just showed a small improvement (Table 3, case of 2003). However, when the thresholds were more than the true values, which resulted in relatively small sample size, the performance of design VI tended not to be good, with reduced precision and strong decrease of sample size (Table 3, cases of 2006 and 2007). When the thresholds were similar to the true value, the performance of design VI tended to be similar to or slightly worse than that for the basic design, with a small decrease in sample size (Table 3, cases of 2001, 2002, and 2004). In addition to the above three scenarios, when the thresholds were neither close to nor distant from the true values (Fig. 2), and the spatial distribution characteristic parameter r2 tended to be large (the values of r2 in 2000 and 2005 were the two largest, Table 1), performance was improved with a reduced sample size (Table 3, 2000 and 2005 cases). Based on the above analysis, we found that the thresholds were important in determining the performance of the design and requirement of the final total sample size. If the spatial distribution of targeted organisms varied greatly (i.e., with high spatial variation), the performance of design VI could be better. Advantages of design VI Of the six sampling designs considered in this study (i.e., simple random design, stratified random design, adaptive sampling design based on the stratified random design, systematic design, MMSD optimized design, and adaptive sampling design based on the optimized design), the adaptive sampling design developed in this study did not have outstanding performance, and it performed neither the best nor the worst. However, if we consider the fact that the average sample size of design VI (adaptive sampling design based on the optimized design) was only 69 sites (Tables 2), which was less than the other sampling designs by 16% (13/82), the advantage of this adaptive sampling design might be more obvious. This means that, with lower sampling effort, the adaptive sampling design could yield relatively good estimates of key population parameters (Table 2). It is well known that increasing the sample size can increase the precision of estimation [14, 30, 31]. If the precision of estimation is assumed to be proportional to the sample size for the adaptive sampling in this study, the precision could be improved by 19% (13/69) when the average sample size was raised to 82 as in the other designs. If the precision with the adaptive sampling design were raised by only 10% (much less than 19%), it would
123
Fish Sci (2011) 77:467–478
yield the best estimations of both parameters (b and r2) and abundance. Adaptive sampling designs applied in fisheries are almost entirely based on first-phase stratified sampling designs [10, 11, 13]. However, the adaptive sampling design developed in this study is based on an optimized sampling design which was demonstrated to be the best for fish stock size estimation in Liu et al. [14]. Compared with other adaptive sampling designs based on stratified sampling design, our adaptive sampling design based on optimized sampling design has the following advantages: Firstly this method is more flexible than the others. In the adaptive sampling designs based on stratified sampling design, the critical values determining whether we need to add second-phase samples are calculated through a number of samples in special strata; i.e., when all the samples in the special strata are not obtained completely, we cannot determine whether additional samples need to be added. However, the adaptive sampling method developed in this study can determine this based on a single site sampling result, which is more flexible than the methods described in other studies [10–13] and more convenient and practical for surveys in practice. Secondly, this method can describe fish clustering patterns more precisely. The number of strata in stratified sampling is usually determined based on environmental variables shown to be important in influencing fish distribution, such as depth and geographic area [10, 32]. However, the areas where fish aggregate are uncertain [10] and not always in agreement with the variables defining strata. Fish aggregation may spread over several strata, in which case the center of aggregation cannot be identified correctly and the added samples cannot be located exactly in the high-density area. The adaptive sampling method developed in this study, however, can solve the above problems by adding samples just based on single-site sampling, through which the high-density area can be identified more directly and precisely. Thirdly, the first-phase sampling design on which this adaptive sampling method depends is more objective than other designs [14]. Stratified random sampling is often used in fisheryindependent surveys. Its design may, however, include some subjectivity, such as the choices of variables used to determine strata, the number of strata, and the criteria used to allocate sampling effort among strata [16]. The optimized sampling design proposed in the first phase of this adaptive sampling tends to be more objective, and the results are more stable. Further development of design VI Although the advantages of the adaptive sampling method developed in this study are obvious, there are still several issues that need to be solved in the process of using this
Fish Sci (2011) 77:467–478
approach in fishery-independent surveys. The determination of the criterion value that, when exceeded, invokes additional sampling is very important for the adaptive sampling method [33]. In this study, this criterion value was calculated by averaging the quantiles of fish abundance from 8 years in order to cover all the simulations over this period. However, the quantile values can be seen to change greatly over these 8 years (Fig. 2). In practice, before a survey starts, an estimation of the abundance level should be made based on population dynamics in previous years [10, 34]. Based on the precise abundance estimation, a suitable criterion value (quantile value) can be obtained, and the adaptive sampling method can more accurately capture the fish aggregation characteristics so that more precise estimation of stock size can be achieved. Moreover, if the predicted abundance is precise and the criterion value is suitable, the total sample number based on the adaptive sampling method would be less and the performance of this design would be more efficient (Table 3; Fig. 4). In simulation we found that the total number of samples for the adaptive sampling fluctuated greatly, from 50 to 154 (Fig. 4). Because of the expense of additional samples in fisheries research, it is very important to control the sample size [10, 34]. In the simulation study, the adaptive sampling design required fewer samples to achieve good precision for stock size estimation. However, the sample number in some special cases in the simulation could exceed 154 (Fig. 4), more than 50% more than the other sample sizes. This is unacceptable in real surveys. So, control of the total sample size during a survey is another important problem that needs to be addressed in practice. The number of sites in the first phase was arbitrarily set at 50, about 60% of the total, which is slightly less than the 75% suggested by Francis for strata-based sampling design [10]. An evaluation of the impact of the number of sampling sites in the first phase is beyond the focus and scope of this study. We will evaluate how the choice of sample size in the first phase may influence the performance of this adaptive sampling design in a separate study. In this study, the efficiency of the adaptive sampling method was tested and compared using only one level of sample size. How this may vary with smaller or larger sample size needs to be evaluated. Future studies need to address how these designs may perform under different scenarios, considering more factors (e.g., different levels of sample size). Acknowledgments Thanks are due to all scientific staff and crew for their assistance in collecting data during the surveys. The work was conducted under the auspices of the Special Research Fund for the National Non-profit Institutes (East China Sea Fisheries Research Institute, no. 2009M01), the Ministry of Science and Technology Public Project (2006/2007), and the Chinese Ministry of Agriculture Assessment of Marine Fisheries Resources Programme. Part of the
477 work reported in this study was done when the senior author (Y.L.) was a visiting scholar at the University of Maine. Y.C.’s contribution to this study is supported by the Maine Sea Grant College program.
References 1. Thompson SK (1990) Adaptive cluster sampling. J Am Stat Assoc 85:1050–1059 2. Thompson S (1991) Stratified adaptive cluster sampling. Biometrika 78:389–397 3. Thompson SK, Seber GA, Okogbaa OG (1996) Adaptive sampling. Wiley, New York 4. Brown JA, Manly BJ (1998) Restricted adaptive cluster sampling. Environ Ecol Stat 5:49–63 5. Smith DR, Conroy MJ, Brakhage DH (1995) Efficiency of adaptive cluster sampling for estimating density of wintering waterfowl. Biometrics 51:777–788 6. Conners ME, Schwager SJ (2002) The use of adaptive cluster sampling for hydroacoustic surveys. ICES J Mar Sci 59:1314 7. Mier K, Picquelle S (2008) Estimating abundance of spatially aggregated populations: comparing adaptive sampling with other survey designs. Can J Fish Aquat Sci 65:176–197 8. Brown JA (2003) Designing an efficient adaptive cluster sample. Environ Ecol Stat 10:95–105 9. Su Z, Quinn TJ (2003) Estimator bias and efficiency for adaptive cluster sampling with order statistics and a stopping rule. Environ Ecol Stat 10:17–41 10. Francis R (1984) An adaptive strategy for stratified random trawl surveys. N Z J Mar Fresh 18:59–71 11. Jolly GM, Hampton I (1990) A stratified random transect design for acoustic surveys of fish stocks. Can J Fish Aquat Sci 47:1282–1291 12. Manly BF, Akroyd JA, Walshe KA (2002) Two-phase stratified random surveys on multiple populations at multiple locations. N Z J Mar Fresh 36:581–591 13. Stephen S, Mark L (2006) Improving the precision of designbased scallop drag surveys using adaptive allocation methods. Can J Fish Aquat Sci 63:1639–1646 14. Liu Y, Chen Y, Cheng J (2009) A comparative study of optimization methods and conventional methods for sampling design in fishery-independent surveys. ICES J Mar Sci 66:1873–1882 15. van Groenigen JW, Stein A (1998) Constrained optimization of spatial sampling using continuous simulated annealing. J Environ Qual 27:1078–1086 16. Cochran WG (1977) Sampling techniques, 3rd edn. Wiley, New York 17. Miller TJ, Skalski JR, Ianelli JN (2007) Optimizing a stratified sampling design when faced with multiple objectives. ICES J Mar Sci 64:97–109 18. Liu Y, Cheng J, Li S (2004) A study on the distribution of Setipinna taty in the East China Sea. Mar Fish 26:255–260 19. Liu Y, Cheng J, Chen X (2006) Studies on the seasonal distribution of Setipinna taty in the East China Sea. Mar Fish Res 27:1–6 20. Box GE, Cox DR (1964) An analysis of transformations. J R Stat Soc 26:211–252 21. Isaaks EH, Srivastava RM (1989) An introduction to applied geostatistics. Oxford University Press, New York 22. Diggle PJ, Ribeiro PJ Jr (2007) Model-based geostatistics. Springer, New York 23. Taylor JR (1997) An introduction to error analysis: the study of uncertainties in physical measurements. University Science Books, USA
123
478 24. Hughes I, Hase T (2010) Measurements and their uncertainties: a practical guide to modern error analysis. Oxford University Press, New York 25. Chen Y (1996) A Monte Carlo study on impacts of the size of subsample catch on estimation of fish stock parameters. Fish Res 26:207–223 26. Andrew NL, Chen Y (1997) Optimal sampling for estimating the size structure and mean size of abalone caught in a New South Wales fishery. Fish B-Noaa 95:403–413 27. Paloheimo JE, Chen Y (1996) Estimating fish mortalities and cohort sizes. Can J Fish Aquat Sci 53:1572–1579 28. Ripley BD (1981) Spatial statistics. Wiley, New York 29. Simmonds EJ, Fryer RJ (1996) Which are better, random or systematic acoustic surveys? A simulation using North Sea herring as an example. ICES J Mar Sci 53:39–50
123
Fish Sci (2011) 77:467–478 30. Jardim E, Ribeiro JP (2007) Geostatistical assessment of sampling designs for Portuguese bottom trawl surveys. Fish Res 85:239–247 31. Simbahan GC, Dobermann A (2006) Sampling optimization based on secondary information and its utilization in soil carbon mapping. Geoderma 133:345–362 32. Chen Y, Sherman S, Wilson C, Sowles J, Kanaiwa M (2006) A comparison of two fishery-independent survey programs used to define the population structure of American Lobster (Homarus Americanus) in the Gulf of Maine. Fish B-Noaa 104:247–255 33. Hanselman DH, Quinn TJ, Lunsford C, Heifetz J, Clausen D (2003) Applications in adaptive cluster sampling of Gulf of Alaska rockfish. Fish B-Noaa 101:501–513 34. Wang YG, Ye Y, Milton DA (2009) Efficient designs for sampling and subsampling in fisheries research based on ranked sets. ICES J Mar Sci 66:928–934
