Active Contour and Hill Climbing for Tree Crown Detection and Delineation Yinghai Ke, Wenhua Zhang, and Lindi J. Quackenbush
Abstract This paper presents a new approach for individual tree crown detection and delineation that is applicable under various imaging conditions. The approach extracts crown area using a region-based active contour model and then detects tree tops within the crown area by considering both spectral and shape characteristics of the crown. The detected tree tops allow subsequent clustering of crown pixels using a new hill-climbing algorithm. We tested the approach on a Norway spruce stand using three types of high spatial resolution imagery: an Emerge natural color vertical aerial image, an off-nadir QuickBird panchromatic image, and a natural color digital orthoimage. In comparison to the published region growing algorithm, our approach improved tree crown detection by over 10 percent for all three types of imagery, and provided accurate tree crown diameter estimation, which has utility in tree volume estimation, species composition, and forest health analysis.
Introduction Successful forest management has vital ecological, social, and economic significance, and it is critical to provide precise, accurate, timely, and complete forest information. Individual tree parameters such as location, diameter at breast height (DBH), tree height, crown size and species provide important forest information (Kangas and Maltamo, 2006). Compared to conventional ground-based forest inventory and visual interpretation of aerial photography, the increasing availability and affordability of sub-meter aerial and satellite image sources provides opportunities for cost-effective and timely forest inventories. A variety of image processing techniques were developed for automated detection and delineation of individual tree crowns. Such techniques make possible the automated estimation of crown size and canopy closure, and facilitate species level classification and forest health monitoring (Leckie et al., 2005). Furthermore, such techniques enhance the derivation of forest stand characteristics and gap analysis (Gougeon and Leckie, 2003; Leckie et al., 2003).
Yinghai Ke is with the Pacific Northwest National Laboratory, 902 Battelle Boulevard, Richland, WA 99352, and formerly with the Department of Environmental Resources and Forest Engineering, State University of New York College of Environmental Science and Forestry, Syracuse, NY 13210 (
[email protected]). Wenhua Zhang and Lindi J. Quackenbush are with the Department of Environmental Resources and Forest Engineering, State University of New York College of Environmental Science and Forestry, 1 Forestry Dr., 402 Baker Lab, Syracuse, NY 13210. PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Current tree crown detection and delineation methods utilize the reflectance pattern displayed by a forest in high spatial resolution (ground sample distance (GSD) between 30 cm and 1 m) imagery. For trees with conical structure, bright peaks in the image correspond to tree tops because of the higher level of solar illumination. Reflectance decreases toward the crown boundaries; darker pixels surrounding the bright crown correspond to shading from neighboring tree crowns or are caused by bidirectional reflectance effects. On vertical, i.e., nadir viewing, images, such tree crowns exhibit a circular shape and the tree top is associated with the pixels in the center of tree crown. Based on this reflectance pattern, current algorithms can be largely grouped into three classes: (a) detect local maxima as tree tops and find crown edges using edge detection techniques (e.g., Walsworth and King, 1999; Pouliot et al., 2002; Wang et al., 2004; Erikson, 2004), (b) detect local minima and follow tree crown boundaries (e.g., Gougeon, 1995; Leckie et al., 2005), and (c) detect both local maxima as tree top and local minima to help define crown region (e.g., Culvenor, 2002; Pouliot et al., 2005). In addition to the reflectance pattern, some methods utilize the geometric characteristics of tree crowns. For example, the algorithm developed by Wang et al. (2004) assumes that tree tops are located around the center of the tree crown, and the algorithm developed by Gougeon (1995) assumes that crown boundaries are convex. Most of the methods previously published were applied to vertical aerial imagery over coniferous stands, with few applications on satellite imagery reported. Ke and Quackenbush (2010, in press) conducted research to compare the watershed segmentation (Wang et al., 2004), region growing (Culvenor, 2002) and valley-following (Gougeon, 1995) algorithms, which represent the above three categories of tree crown detection and delineation methods. That study reported that while region growing yielded the highest producer’s and user’s accuracy, only 70 percent of the tree crowns were correctly delineated. All three algorithms assessed resulted in lower accuracy when applied to off-nadir satellite imagery than to vertical aerial imagery. Scale was found to be a critical factor influencing crown detection and delineation in many studies (Brandtberg, 1998; Culvenor, 2000; Pouliot et al., 2002; Pouliot and King, 2005). In this context, scale considers the relationship between the image spatial resolution and the size of tree crowns. Very high-resolution imagery (e.g., 5 to 15 cm GSD)
Photogrammetric Engineering & Remote Sensing Vol. 76, No. 10, October 2010, pp. 1169–1181. 0099-1112/10/7610–1169/$3.00/0 © 2010 American Society for Photogrammetry and Remote Sensing O c t o b e r 2 0 1 0 1169
may provide too much detail, with branches and shadows causing such variation that individual tree crowns are divided. However, as resolution decreases, the distinction between crown area and background also decreases, especially for small tree crowns, thus crown edges are less detectable. Pouliot et al. (2002) reported that the optimum ratio between crown diameter and the ground pixel size for their algorithm was 15:1. When the ratio is large, optimal scales need to be determined to ensure detection of small trees and avoid excess detection of branches (Brandtberg, 1998; Culvenor, 2000; Pouliot and King, 2005). Among the studies reviewed, 50 to 100 cm GSD imagery was most frequently used to identify trees in mature forest (e.g., Wang et al., 2004; Leckie et al., 2004; Bunting and Lucas, 2006; Pollock, 1999; Culvenor, 2002; Wulder et al., 2000). The objective of this study was to develop a methodology capable of providing accurate tree detection and delineation under various imaging conditions. The new approach used the active contour model originally presented by Kass et al. (1987) and presented a hill-climbing algorithm to delineate tree crowns imaged under different sensor geometries. The algorithm was tested using high spatial resolution imagery from different sources and compared to results generated using the region growing algorithm presented by Culvenor (2002).
Data Collection Study Area and Imagery The study area is located in the Heiberg Memorial Forest, approximately 33 km south of Syracuse in upstate New York (42.75°N, 76.08°W). Heiberg forest is a 1,600 ha property owned and managed by the State University of New York College of Environmental Science and Forestry (SUNY-ESF). Our study site covered three adjacent Norway spruce compartments that were established in 1931. Trees were planted at 2 m 2 m spacing when they were 3-year-old saplings. Three plots were selected from this site based on the thinning activities within the compartments: Plot 1 was thinned between 1979 and 1980; Plot 2 was thinned inside the forest stand in 1980, while there was no thinning along the road that passes through this plot; and Plot 3 was thinned in 1985. Three remotely sensed images acquired from different sources were used in this study. Digital orthoimagery (DOI) was acquired from the New York State Digital Orthoimagery Program (http://www.nysgis.state.ny.us). This was an 8-bit natural color digital image acquired using the Intergraph Digital Mapping Camera (DMC) sensor in April 2006, with 0.61 m GSD and 2.4 m horizontal accuracy. The second image was a panchromatic satellite image acquired by the QuickBird sensor in August 2004 with an 11° average look angle. The original 11-bit QuickBird image was resampled by the nearest-neighbor method to 0.6 m pixel size. A third image was acquired by the Emerge digital airborne sensor (first generation DSS sensor) in October 2001. The imagery was collected at 12-bits and offloaded as an 8-bit true color vertical image with 0.6 m pixel size. All three images were georeferenced to UTM Zone 18N coordinates with WGS84 datum. Sub-images that covered the three study plots were extracted from the full images (Figure 1). Reference Data Visual Interpretation Due to the differences in imaging conditions and acquisition time among the three images, reference data was generated for each image separately. Manual delineation of tree crowns in each image was performed by three interpreters with a 1170 O c t o b e r 2 0 1 0
forest engineering background. Each person counted individual trees on a computer screen displaying the image. The relative differences between the resultant counts within a single image were 10 percent. With the average count number used as guidance, one of the interpreters outlined the boundary of each tree crown as polygon layers in ArcGIS® 9.2. Crown diameters were calculated based on the assumption of circular crown shape for both the Emerge image and the digital orthoimagery. Because of the crescentshape of the crowns in the QuickBird imagery, crown diameters could not be similarly derived, therefore, only crown area was considered. Gong et al. (2002) presented an optimization method for estimating coniferous crown parameters from off-nadir aerial images. However, the algorithm required stereo imagery and was applied to open forest so was not considered for this research. Specific characteristics of the tree crowns in each plot as identified by the manual interpretation are listed in Table 1. Field Data Collection Ground data was collected within Plot 3 in July 2008 to support the analysis of the digital orthoimagery. Property managers verified that there was no significant change in forest conditions between April 2006 and July 2008, so the comparison was considered reasonable. However, the time delay after acquisition of the QuickBird and Emerge imagery was too great to utilize the ground reference, hence analysis using the field data focused on the digital orthoimagery. The field study established a network of four 30 m 30 m field plots to form a 60 m 60 m work area within Plot 3. For each tree, the stem location, DBH, height, and crown diameter were measured and recorded. A total of 220 trees were measured with mean crown diameter of 3.64 m (Table 1). The stem position and tree measurements were used to create a point layer in ArcGIS® 9.2. A few trees were found to have died during the last three to four years, which could explain the lower tree count manually delineated from the orthoimage than from the other images.
Methodology A flowchart of our algorithm is presented in Figure 2. The algorithm input is a grey-level image. For both the orthoimagery and the Emerge image, the green band was selected since it showed the greatest apparent distinction between tree crowns and the shaded area between trees and provided the best results during preliminary analysis. The algorithm is composed of three stages. First, the image is segmented using a region-based active contour model to separate tree crown area from background and define initial tree crown boundaries. Subsequently, the tree top detection algorithm is applied in the crown area to locate individual tree crowns. We consider both spectral and shape information of the individual tree crown image, and utilized expert knowledge to refine tree top detection results. Finally, individual tree crown boundaries are outlined using a hill-climbing algorithm based on the tree tops defined in the second step. The following sections address each stage in detail. Image Segmentation: Active Contour Model Active contour models have been successfully utilized for image segmentation in computer vision since initially proposed by Kass et al. (1987). Active contour models have proved advantageous over traditional edge detectors because they can achieve sub-pixel accuracy of object boundaries (Xu et al., 2000) and the resultant boundaries are regular (Tsai et al., 2003). The idea is to evolve a contour (or curve) by a local force such that it moves toward the boundary of the object in the image. The stopping criteria of the PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
(a)
(b)
(c)
(d)
Figure 1. Sample images of Norway spruce plots: (a) Digital orthoimagery (green band), (b) QuickBird panchromatic image, (c) Emerge image (green band), and (d) Enlarged view of Emerge image on Plot 3.
TABLE 1. Digital orthoimagery (April 2006) Plot No Reference Count 1 2 3
597 320 305
SUMMARY
OF
REFERENCE TREE CROWN CHARACTERISTICS
QuickBird panchromatic image (August 2004)
Tree crown diameter (m) Mean
SD
3.64 3.89 3.51
0.72 0.81 0.65
Reference Count 611 333 326
Tree crown area (m2) Mean
SD
6.85 7.57 6.56
3.08 3.40 2.81
evolution is formulated as the minimization of an energy function that considers image gradient and geometric properties of the contour (edge-based models) or considers both object and background region intensities (region-based models). Region-based active contour models do not depend on the image gradient and thus are advantageous over edgebased active models in segmenting objects with weak boundaries (Li et al., 2007). PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
60 60 m field data from Plot 3 (July 2008)
Emerge image (October 2001)
Reference count 619 342 317
Tree crown diameter (m) Mean
SD
2.95 2.96 2.92
0.66 0.78 0.69
Reference count N/A N/A 220
Tree crown diameter (m) Mean
SD
N/A N/A 3.64
N/A N/A 0.89
In our research, we adopted the region-based active contour model presented by Li et al. (2007). The justifications of utilization of this model are: (a) it performed better than a gradient-based edge detector in separating tree crowns and background where grey-level differences are not distinct, and (b) the method accounted for intensity inhomogeneity within the object by including local image properties in the energy formulation, thus the method is O c t o b e r 2 0 1 0 1171
Figure 2. Algorithm framework: n is area threshold; n 30 for orthoimage, n 25 for QuickBird and Emerge images.
less likely to split larger crowns even with larger intensity variation. Equation 1 shows the energy function utilized in this approach: Ex l1
3
K(x y) ƒ I(y) f1(x) ƒ 2dy
(1)
in(C)
l2
3
K(x y) ƒ I(y) f2(x) ƒ 2dy
out(C)
where I is the intensity image, C represents the contour in the image, K(u) is a localized weighting function that decreases when u increases, l1 and l2 are positive constants which represent the weights for internal energy and external energy, and f1(x) and f2(x) are two numbers that fit image intensities near the point x. The contour C evolves based on level set theory and the final contour 1172 O c t o b e r 2 0 1 0
is represented by the zero values of the level set function that minimizes the energy function. The level set function values are equal to zero on the contour, positive outside the contour and negative inside the contour. Crown objects can be extracted by locating pixels with negative value. The active contour model reasonably defined crown area boundaries (Figure 3). In circumstances where neighboring trees touched each other, the model did not isolate individual tree crowns; rather, it identified clumps of trees. Thus, the crown objects extracted from this step were categorized into individual crowns and crown clusters, which could be separated by area. The crown area threshold n was determined by carefully examining those isolated and easy-to-identify individual tree crowns whose area was within the range of the reference crown area (Table 1). Using this method, crown objects with area less than 30 pixels in the orthoimagery were considered individual crowns, larger PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
(a)
(b)
Figure 3. (a) Crown (or crown cluster) boundaries delineated by the active contour model in the Emerge image; Plot 3 outlined by white rectangle, and (b) Enlarged view of Plot 3.
than 30 pixels were called potential crown clusters since they could be a larger tree crown or a cluster of trees. Similarly, the threshold was selected as 25 pixels for Emerge and QuickBird imagery. Spectral-Shape-Knowledge-based Tree Top Detection The second phase of the process, detection of tree tops, used multiple steps to consider reflectance patterns, shape information, and expert knowledge. Spectral-based Local Maxima Detection In the first step in tree top detection, the image was scanned using a 3 3 moving window; the center pixel of the window was considered a local maxima if it had the highest intensity value within the window (Wulder et al., 2000). The size of the window ensured detection of small tree crowns; however, large crowns may appear to have multiple local maxima. The false treetops were removed in the following refinement steps. Shape-based Tree Top Refinement Template matching is applied by searching for a match to a model of the full extent of the object image, also called a template, within the original image region (Gonzalez and Woods, 2008), thus it has the ability of capturing the shape of the object. Quackenbush et al. (2000) developed a set of templates by manually selecting typical trees from aerial imagery and then used the corresponding intensity values PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
within windows that covered each tree as templates. The locations where the correlation between templates and image reached the maximum were considered tree crowns. The templates used in that study were square templates and included background pixels surrounding circular crowns. However, the correlation calculated when these background pixels were included in the templates did not perform well when gaps between neighboring crowns were inconsistent. In this research, we utilized individual crowns as templates and calculated correlation using only pixels located within the crown. Ten crown templates were manually selected from two sources: individual crowns identified during image segmentation to represent smaller crowns, and potential crown clusters that were considered by interpreters to represent single, larger crowns. These crown templates were selected to cover a range of crown areas. The calculated Pearson’s correlation coefficient for each movement of the template was assigned to the geometric center of the template. This ensured that the maximum correlation was located on the geometric center of the crown when it has similar spectral and shape characteristics with the template crown. Each template was passed across the image to produce correlation layers. We created a maximum correlation layer by assigning the maximum correlation value from the ten layers for each pixel. A 3 3 window was applied on the maximum correlation layer to locate the shape-based local maxima within the crown objects. The spectral-based local maxima detected in the previous step may include cases where multiple treetops were located within a single identified crown. Thus, we refined the tree top detection by including only those spectral-based local maxima that were located in proximity of shape-based local maxima as potential tree tops. We obtained potential tree tops by applying a 3 3 proximity operator, which defines the maximum distance allowed between the two types of local maxima. Knowledge-based Tree Top Refinement The final step in refining tree top locations incorporated knowledge regarding the distance between trees in the forest, which is especially useful for plantations. In our study area, trees were initially planted at 2 2 m spacing, thus we assumed that the spacing between tree tops could not be less than 2 m, i.e., approximately three pixels in the image. Under this restriction, we considered tree tops that were too close to others and had lower intensity value as spurious tree tops. Therefore, the final tree tops not only represent points that have highest illuminations and resemble the crown shapes, but also have the most probable location. The final tree tops were labeled and used as input for the crown delineation step. Crown Delineation: Hill-climbing Algorithm The crown areas defined by the active contour model consisted of individual tree crowns as well as crown clusters. With individual tree tops defined within each crown cluster, the goal of this stage was to use a hill-climbing algorithm to isolate individual tree crowns within the clusters. Hillclimbing is a mathematical optimization algorithm which aims at locating the maximum value of an objective function by moving toward a state which improves the current one (Huang and Shibasaki, 1995). The algorithm has been utilized to find tree top and define crown area using lidar (Persson et al., 2002) and very high resolution aerial imagery (Pouliot et al., 2005). The major differences between our algorithm and the traditional hill-climbing algorithm are: (a) our maximum point (tree top) is known, and (b) our focus is to find a route from each pixel within the crown object that O c t o b e r 2 0 1 0 1173
in Figure 4b). This situation occurs when intensity variation causes multiple peaks within a crown but only one tree top was defined. Under this circumstance, the moving point will follow the third rule: • Rule 3: The moving point is allowed to move downward if the difference between the intensity of the current and the next pixel is less than a defined threshold, and it then continues to climb to the peak.
The threshold is selected based on the magnitude of the intensity variation in the image. For the Emerge and orthoimagery, it was set as 5; for the QuickBird image where the variation was larger, the threshold was set as 10. Each pixel was clustered and assigned the same label as the tree top.
(a)
Comparison Method: Region Growing Based on a prior comparative study (Ke and Quackenbush, 2010, in press), we used the region growing algorithm presented by Culvenor (2002) as a comparison method. Region growing is an image segmentation approach used to separate regions and recognize objects in an image. Culvenor’s study used local maxima which were determined by local peak reflectance in four directions (N-S, E-W, NE-SW, NW-SE) as seed pixels. From the seed pixels, neighboring pixels were examined sequentially and added to the growing region until the pixel intensity was lower than a threshold defined by the product of the average of local maxima and a ratio factor, or until the pixel reached the predefined local minima networks. Our study strictly followed the algorithm described in Culvenor (2002). In our implementation, the ratio factor was selected as 0.5 for each of the three images because it gave the best results.
Results and Discussion
(b)
Figure 4. (a) 3D view of crown objects with two tree tops, and (b) 3D view of crown object with single tree top
“climbs” to a tree top so that the pixel can be assigned to that tree top. Figure 4a illustrates the three-dimensional view of image intensity within a crown object where two tree tops were detected; the vertical axis represents intensity values. For each pixel in the crown clump, we need to identify the tree it belongs to; thus, this approach assigns pixels to one of the defined tree tops within the clump. The algorithm can be understood by considering a moving point that can only move one step (or one pixel distance) at a time. The basic principle of the clustering is to let the point climb the hill until the peak (one of the identified tree tops) is reached and an assignment is made. Specifically, the rules of hill-climbing are: • Rule 1: The moving point can only move upward; • Rule 2: If the moving point could reach multiple peaks (e.g., point falls at the valley between hills), it must follow the shortest route to the peak (tree top).
With these two rules, there may be scenarios where the moving point could not move anywhere (such as P1 labeled 1174 O c t o b e r 2 0 1 0
Visual Evaluation The results of applying our algorithm to the three image types under study are illustrated in Plate 1. In general, the majority of Norway spruce trees were extracted from the background and delineated separately on all three images. The algorithm was able to capture the crescent shape of the tree crowns in the QuickBird imagery. The three images also demonstrate the growth of tree crowns: tree crowns delineated in the orthoimagery acquired in 2006 were larger than those in the Emerge imagery acquired in 2001. This change in forest conditions between image acquisition dates did not impact the goals of the project, i.e., to examine whether the algorithm could provide accurate crown delineation. We did not compare the separate images, rather they were assessed independently. Tree Count Evaluation The comparison of tree count estimations from the two algorithms showed that while both methods produced less than 15 percent tree count error, our algorithm produced smaller errors than the region growing algorithm for both the orthoimagery and QuickBird image (Table 2). Region growing overestimated the tree counts by around 10 percent in the QuickBird imagery, while our algorithm produced only 1 percent error. The greater overestimation from region growing was caused by large within-crown intensity variation, thus multiple local maxima were detected within a single crown. However, with consideration of shape information and the distance between tree tops, tree counts estimated from our algorithm were more accurate. Tree counts estimated from both algorithms were similar and close to the reference counts for the Emerge image (error less than PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
(a)
(b)
(c)
Plate 1. Individual tree delineation over Norway spruce stand Plot 3 on (a) digital Orthoimagery green band (April 2006), (b) QuickBird (August 2004), and (c) Emerge imagery green band (October 2001). Red lines represent the boundaries of individual tree crowns.
TABLE 2.
TREE COUNT ESTIMATION AND ERROR (%): NEGATIVE ERROR INDICATES UNDERESTIMATION COMPARED TO REFERENCE DATA; POSITIVE ERROR INDICATES OVERESTIMATION
Digital orthoimagery (April 2006)
QuickBird panchromatic image (August 2004)
Emerge image (October 2001)
Plot No Reference Count 1
597
2
320
3
305
HC
RG
592 (0.8) 349 (9.1) 327 (7.2)
659 (14) 362 (13) 333 (9.2)
Reference Count 611 333 326
HC
RG
604 (1.1) 348 (1.0) 326 (0)
661 (8.2) 367 (10) 354 (8.6)
Reference Count 619 342 317
HC
RG
618 (0.2) 339 (0.8) 315 (0.6)
613 (1.0) 341 (0.3) 317 (0)
HC: Hill-climbing algorithm; RG: Region growing algorithm.
1 percent). However, both algorithms generally overestimated tree counts in the digital orthoimagery, possibly because the larger tree crowns seen on the digital orthoimagery were more likely to be split as multiple tree tops. Individual Tree Detection and Delineation Assessment The plot level accuracy reported in Table 2 reflects the aggregated accuracy of tree detection, but can be misleading due to the potential cancellation of commission and omission errors. Additionally, tree counts do not reflect how well delineated crowns match reference crowns. Further analysis of the results is needed to evaluate the detection and delineation results at the individual tree level from both the reference crown and delineated crown perspectives. The reference crown perspective considers how well each PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
reference crown was delineated; while the delineated crown perspective reflects how well each delineated crown represented a reference crown. This analysis is analogous to the approach used by Leckie et al. (2004) and provides assessment at the object-level. Figure 5 demonstrates the typical detection and delineation scenarios when the delineated and reference crowns are overlaid. Each component within Figure 5 displays a specific ratio of delineated to reference crowns, which can vary depending on the perspective taken. The reference crown perspective displays the number of occurrences where reference crowns were missed (e.g., Figure 5(i), were delineated as one crown (e.g., Figure 5(iii) and Figure 5(v)), or were delineated as more than one crown (e.g., Figure 5(iv)). The delineated crown perspective considers the O c t o b e r 2 0 1 0 1175
Figure 5. Examples of typical detection scenarios from (a) reference, and (b) delineated crown perspectives with reference crowns shown as circular outline and delineated crowns as polygons: (i) Simple omission error; (ii) Simple commission error; (iii) Exact match; (iv) Commission through over-segmentation; and (v) Omission through undersegmentation. In all cases, ratio shows [delineated crown : reference crown].
TABLE 3.
DETECTION RESULTS
FROM
HILL-CLIMBING ALGORITHM
FOR
THREE IMAGES
Delineated Crowns : Reference Crowns Plot No.
Reference Crown Perspective 0:1
1:1
2:1
3:1
1 2 3 Total
15 4 0 19
534 264 285 1083
47 48 20 115
1 4 0 5
1 2 3 Total
28 14 22 64
533 268 278 1079
48 50 26 124
QuickBird panchromatic image 2 611 21 1 333 20 0 326 21 3 1270 62
1 2 3 Total
5 0 0 5
567 281 292 1140
47 56 25 128
0 5 0 5
Total
1:0
1:2
1 : ( 3)
Total
Exact 1:1
44 37 17 98
2 1 0 3
592 349 327 1268
458 217 261 936
(August 2004) 536 42 286 41 280 25 1102 108
5 1 0 6
604 348 326 1278
445 211 226 882
3 3 1 7
618 339 315 1272
490 216 251 957
1:1
Digital Orthoimagery (April 2006) 597 19 527 320 16 295 305 19 291 1222 54 1113
Emerge image (October 2001) 619 15 548 342 0 273 317 5 279 1278 20 1100
number of occurrences where delineated crowns did not overlap any reference crown (e.g., Figure 5(ii)), or where a delineated crown represented one reference crown (e.g., Figure 5(iii) and Figure 5(iv)), or more than one reference crowns (e.g., Figure 5(v)). Table 3 and Table 4 summarize the frequencies of various errors for the hill-climbing and region growing algorithms, respectively, from both the reference crown and delineated crown perspectives for each plot in each of the images. As an example, for the hill-climbing results shown in Table 3, in the digital orthoimagery of Plot 1 with 597 reference crowns, there were 534 reference crowns that matched only one delineated crown (including the scenarios illustrated in Figure 5(iii) and 5(v)). Among 592 delineated crowns, there were 527 crowns where each one represented only one reference crown (as illustrated in Figure 5(iii) and 5(iv)), and there were 44 delineated crowns where two reference crowns were located on a 1176 O c t o b e r 2 0 1 0
Both Perspectives
Delineated Crown Perspective
52 63 30 145
single delineated crown. Table 3 shows that there were cases when one reference crown was covered by multiple delineated crowns (commission error), which still results in 1:1 correspondence from the delineated crown perspective. Thus, the number of 1:1 correspondence from the delineated crown perspective is greater than the number of 1:1 correspondence in the reference crown perspective. The right hand column in Table 3 provides the total number of instances where there was a 1:1 correspondence from both perspectives, for example, 458 individual trees were delineated correctly in Plot 1 with the hill-climbing method in the digital orthoimage. Comparison between the various error types showed that the hill-climbing errors were dominated by 1:2 omission errors and 2:1 commission errors. For example, in Plot 1 there were only 15 cases where reference crowns were missed by the algorithm, but 47 cases where one reference crown was split into two crowns. This latter problem was PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
TABLE 4.
DETECTION RESULTS
FROM
REGION GROWING ALGORITHM
FOR
THREE IMAGES
Delineated Crowns : Reference Crowns Plot No.
Reference Crown Perspective 0:1
1:1
2:1
3:1
1 2 3 Total
39 20 17 76
491 261 259 1011
62 37 29 128
5 2 0 7
1 2 3 Total
42 30 20 92
447 216 227 890
102 72 68 242
QuickBird panchromatic image 20 611 34 15 333 20 11 326 9 46 1270 63
1 2 3 Total
10 10 8 28
540 279 269 1088
65 46 30 141
4 7 10 21
Total
1:0
1:2
1 : ( 3)
Total
Exact 1:1
32 21 19 72
2 2 0 4
659 362 333 1354
353 176 200 729
(August 2004) 549 71 292 51 299 41 1140 163
7 4 5 16
661 367 354 1382
362 161 174 697
2001) 448 238 248 934
18 9 7 34
613 341 317 1271
378 179 223 780
1:1
Digital Orthoimagery (April 2006) 597 75 550 320 46 293 305 35 279 1222 156 1122
Emerge image (October 619 59 342 30 317 29 1278 0 118
mainly caused by large tree crowns with irregular shape where two tree tops were detected within what should have been one crown. The other major type of error was where two reference crowns were delineated as one, which occurred when small crowns with touching branches produced only one detectable tree top. Comparison of the hill-climbing and region growing algorithms, presented in Table 3 and Table 4, showed that the hill-climbing algorithm produced a larger number of instances of exact 1:1 correspondence between reference and delineated crowns, and also produced a smaller number of both omission errors (e.g., 0:1 correspondence or 1:2 correspondence) and commission errors (e.g., 2:1 correspondence). This evaluation of the tree crown detection results indicated that the spectral-shape-based tree top detection incorporated with knowledge-based refinement was more accurate in estimating crown location than the tree top detection used by region growing algorithm. Tree crown detection and delineation accuracy was further quantified using a method adapted from Larmar et al. (2005) who modified user’s accuracy and producer’s accuracy from pixel-based classification to fit object-based analysis. In our study, producer’s accuracy was defined as the portion of reference crowns with a 1:1 exact match to a delineated crown, and user’s accuracy was defined as the portion of delineated crowns that had a 1:1 exact match to a reference crown. Our algorithm obtained 75 to 85 percent producer’s and user’s accuracy in the orthoimage and Emerge image; accuracy obtained in QuickBird image was around 70 percent (Figure 6). For all three images, the hillclimbing algorithm produced better detection results than the region growing method ( 10 percent higher accuracy). Comparison between the three plots demonstrates that accuracies in Plot 2 were slightly lower than the other two plots for both algorithms. This was largely due to reduced thinning along the road in Plot 2, which led to more variation in crown size and less spacing between trees. Tree Crown Diameter (Area) Estimation Delineation accuracies were evaluated by comparing the diameter (for Emerge image and orthoimage) or area (for PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Both Perspectives
Delineated Crown Perspective
88 64 33 185
QuickBird image) of the delineated tree crowns with that of the reference crowns for the exact 1:1 matched trees. The accuracies were summarized using mean error, mean absolute error (MAE), and root mean square error (RMSE) as a percentage of mean reference crown diameter (area). The percent RMSE was calculated for the Emerge and orthoimage using the following equation from Pouliot et al. (2002):
RMSE%
Ca
(deli refi)2/n ref
(2)
where n is the number of correctly delineated trees, deli is the estimated diameter for the ith correctly delineated crown, refi is the reference diameter for the corresponding reference crown, and ref is the mean reference crown diameter. This equation was adapted slightly for the QuickBird image to consider the RMSE as a percentage of the mean reference crown area. Comparisons showed both algorithms were accurate in estimating crown diameter for the Emerge image and orthoimage, with mean error (0.03 to 0.42 m) less than the size of one pixel (0.6 m), while our algorithm yielded larger mean error (0.24 to 0.42 m) for all three images (Table 5). RMSE (%) errors were similar for both methods. Similarly for crown area estimation in the QuickBird image, both algorithms produced low mean error (0.25 to 1.05 m2), all under three pixels; our algorithm produced lower accuracies in terms of mean error and MAE, with similar RMSE as region growing (Table 5). The larger mean error of our algorithm could be explained by the differences between the two algorithms in defining individual crown area. The region growing algorithm expands crown area by grouping pixels until a pre-defined gap network is reached, thus there is at least a one-pixel-wide gap between neighboring crowns. Therefore, the delineated crown dimension may have a closer match when compared to the manually delineated reference crowns since gaps were also clearly defined during manual interpretation. Unlike region growing, our algorithm partitions the crown cluster into individual tree crowns, thus every pixel within the crown area belongs to a certain crown, and there are no gaps between neighboring crowns. O c t o b e r 2 0 1 0 1177
(a)
(b)
(c)
Figure 6. Accuracy comparison of hill-climbing algorithm and region growing algorithm on (a) digital orthoimagery, (b) QuickBird panchromatic image, and (c) Emerge image.
It is expected that our algorithm will perform well in dense forest conditions where the gap between neighboring trees is often less than the size of a pixel. Assessment Using Ground Reference Based on the fact that there was minimal change in forest conditions between 2006 and 2008, we evaluated the performance of our new algorithm and the region growing algorithm on the digital orthoimagery, which was acquired in 2006, by comparing the results with ground reference measurements within Plot 3 (Table 6), which were collected in 2008. 1178 O c t o b e r 2 0 1 0
The ground-based assessment considered situations where one reference stem was located within one delineated crown. Of the 220 trees observed in the field, there were 178 occasions where crowns delineated using the hill-climbing algorithm contained only one reference stem, and 162 reference stems similarly located within crowns from region growing. Comparison between these reference crown diameters and the corresponding delineated crown diameters revealed nine outliers for hill-climbing and fifteen outliers for region growing. These outliers were mainly caused when a large tree was split into several pieces such that the reference diameter actually corresponded to the sum of adjacent delineated crowns. After removing the outliers, 169 (77 percent) of the ground reference trees were correctly detected by the hill-climbing algorithm, and 147 (67 percent) by region growing. Both the hill-climbing and region growing algorithms obtained low mean crown diameter errors, 0.06 m and 0.08 m, respectively, for the correctly detected trees; however, our algorithm produced lower MAE and RMSE than the region growing algorithm (Table 6). Region growing tended to underestimate the crown dimension since it defines the gap between neighboring tree crowns as a one(or more)pixel-wide local minima network; in dense forest, the gap is often less than the size of a pixel. The linear relationship between the delineated and field-observed crown diameter was found to be stronger when using the hill-climbing algorithm, with higher correlation coefficient (0.82) than region growing (0.73) (Figure 7). The least-squares regression models suggests that it is possible to predict the ground-measured crown diameter using delineated crowns derived by hill-climbing algorithm, with R2 of 0.69; while R2 value for region growing algorithm was only 0.53 (Figure 7). Algorithm Evaluation The algorithm presented in this paper used three basic steps: segmentation to separate crowns from background, tree top detection, and crown delineation. The algorithm was designed to overcome limitations of existing methods at each step. Figure 8 compares the active contour model and conventional thresholding for crown area/background separation on the Emerge image. A limitation of thresholding is the need to predefine the threshold parameter, and the impact of this value on delineation quality. Unlike the active contour model, the threshold parameter is implemented globally and does not consider the local spectral variance of the crowns. The initial crown boundaries defined by the active contour model (Figure 8a) were smooth and better captured crown shape compared to thresholding (Figure 8b). The tree top detection presented in our algorithm combines spectral and shape information and utilizes expert knowledge of the minimum distance between stems. Spectral-based local maxima defined local mountainous tree crown peaks; shape-based local maxima found crown geometric centers that most resembled templates: circular on the Emerge image and orthoimagery, and crescent shaped on the QuickBird image. Table 7 compares the tree count results from our algorithm to the spectral- and shape-based components both separately and combined without the expert knowledge. The use of spectral or spatial maxima filters alone produced over-estimation of tree counts, while combining the two methods reduced false tree tops. The addition of expert knowledge further refined the results, and produced the most accurate tree top detection. While the distance between tree stems is especially useful for managed plantations, in natural forests, or plantations where natural regeneration has occurred, the distance between tree stems is typically less PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
TABLE 5.
SUMMARY STATISTICS
Digital orthoimagery Crown diameter estimation
OF
CROWN DIAMETER DELINEATION ERRORS
QuickBird panchromatic Crown diameter estimation
Emerge image Crown diameter estimation
Plot No Mean error (m)
MAE (m)
1 2 3
0.29 0.27 0.37
0.43 0.43 0.44
1 2 3
0.05 0.03 0.20
0.39 0.41 0.41
RMSE (%)
Mean error (m2)
MAE (m2)
Hill-climbing algorithm 0.42 2.30 1.03 2.41 1.05 2.05 Region growing 14 0.25 1.72 13 0.70 2.33 16 0.49 1.89 14 14 15
TABLE 6.
SUMMARY
OF
Hill-climbing Region growing
Mean error (m)
MAE (m)
RMSE (%)
38 42 39
0.32 0.42 0.24
0.47 0.55 0.43
20 21 17
40 42 40
0.14 0.29 0.23
0.34 0.43 0.38
15 18 17
GROUND REFERENCE ASSESSMENT
Tree crown detection accuracy Algorithm
RMSE (%)
Tree crown delineation accuracy
Detected trees
One-stem-to-one-crown correspondence
Correctly detected
Mean error (m)
MAE (m)
RMSE (%)
r
224 235
178 162
169 147
0.06 0.08
0.36 0.49
8.0 11
0.82 0.73
r: correlation coefficient between reference crown diameter and delineated crown diameter
defined. However, knowledge of forest conditions such as tree density collected during field visits can still support expert rules to improve tree crown detection. In addition to minimum stem spacing, our algorithm requires user input including the crown area threshold n to separate individual crowns from potential crown clusters, the number of templates, and the selection of templates. For a forest with more variable crown sizes, the individual crown area threshold should be low to ensure only a single crown was selected. The number and type of templates should represent various crown sizes and shapes. Having established tree tops, crown boundaries are outlined using a hill-climbing algorithm. Our algorithm was an adaption of the hill-climbing idea used by Persson et al. (2002) and Pouliot et al. (2005). In their method, tree tops were not pre-defined; instead, each pixel in the image was forced to climb until it reached a position where all neighboring pixels had lower values. This position defined a tree top, and all pixels that climbed to this tree top were grouped into a crown. The potential problem of their method was that spurious tree tops may be found (such as P1 in Figure 4b). The tree tops defined in their method were identical to the tree tops defined by 3 3 local maxima filter (Pouliot et al., 2005), which produced over-estimation of tree count (Table 7). The improved algorithm presented in our study excluded spurious tree tops, and assigned pixels within the spurious tree tops to the real tree top.
Conclusions This study developed a tree crown detection and delineation method based on the active contour model and a hill-climbing algorithm. The region-based active contour approach provided the initial boundaries of crown objects. Individual tree top detection incorporated both spectral and shape information of individual tree crowns in the tree PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
top detection, and the expert knowledge of the forest stand was utilized to refine tree top detection. The hill-climbing algorithm classified every pixel within crown objects to a tree top. By analyzing the tree count estimation and the match between reference crowns and delineated crowns, we found the new algorithm was successful in tree detection and crown delineation from both vertical aerial images such as the Emerge and digital orthoimagery and the off-nadir QuickBird satellite image. Comparison with the existing region growing algorithm showed that our algorithm yielded more than 10 percent higher accuracy in tree detection and delineation. Our algorithm also provided accurate crown diameter (0.24 to 0.42 m for orthoimagery and Emerge image) or area (0.42 to 1.05 m2 for QuickBird image) estimations. The results from the different images suggest that the algorithm is applicable for working in various types of image acquisition conditions. Assessment of the algorithms using ground-based reference data showed that our algorithm was more accurate, and the delineated tree crowns were comparable with the field-measured tree crowns. A close linear relationship between delineated crown diameter and field-measured crown diameter was found, indicating that our results could be used to predict field-measured crowns (R2 0.69). Analysis also showed advantages of the new algorithm at each of the processing steps. We found the off-nadir image still pose challenges in that the crescent-shape-crown dimensions could not be directly associated with crowns in the field. Our future research will involve utilizing the geometric and radiometric characteristics of off-nadir imagery to derive accurate crown size estimation and to provide tree volume estimation, and test the algorithm in different forest conditions. Future research will also consider the utility of using the delineated tree crowns to improve evaluation of species composition and forest health at the stand level. O c t o b e r 2 0 1 0 1179
(a)
(a)
(b)
Figure 8. Image segmentation on Emerge image Plot 3 using (a) active contour model, and (b) thresholding (threshold 85 determined by visual examination).
References
(b)
Figure 7. Scatterplots of delineated crown diameter versus ground reference crown diameter for correctly detected trees using (a) hill-climbing algorithm, and (b) region growing algorithm.
TABLE 7.
Plot No. 1 2 3
1180 O c t o b e r 2 0 1 0
Brandtberg, T., 1998. Automated delineation of individual tree crowns in high spatial resolution aerial images by multiple-scale analysis, Machine Vision and Applications, 11:64–73. Bunting, P., and R.M. Lucas, 2006. The delineation of tree crowns in Australian mixed species forests using hyperspectral Compact Airborne Spectrographic Imager (CASI) data, Remote Sensing of Environment, 101:230–248. Culvenor, D.S., 2000. Development of a Tree Delineation Algorithm for Application to High Spatial Resolution Digital Imagery of Australian Native Forest, Ph.D. dissertation, University of Melbourne, Melbourne, Australia. Culvenor, D.S., 2002. TIDA: An algorithm for the delineation of tree crowns in high spatial resolution remotely sensed imagery, Computers & Geosciences, 28(1):33–44. Erikson, M., 2004. Species classification of individually segmented tree crowns in high-resolution aerial images using radiometric and morphologic image measures, Remote Sensing of Environment, 91(3):469–477.
COMPARISON BETWEEN TREE TOP DETECTION METHODS
ON
EMERGE IMAGE
Reference tree count
Spectral-based local maxima filter
Shape-based local maxima
Spectral/shape-based local maxima w/o expert knowledge
Our tree top detection
619 342 317
735 459 386
694 423 355
657 370 341
618 339 315
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Gong, P., Y. Sheng, and G.S. Biging, 2002. 3D model-based tree measurement from high-resolution aerial imagery, Photogrammetric Engineering & Remote Sensing, 68(11):1203–1212. Gonzalez, R.C., and R.E. Woods, 2008. Digital Image Processing, Third edition, Prentice Hall, Upper Saddle River, New Jersey, 954 p. Gougeon, F.A., 1995. A crown-following approach to the automatic delineation of individual tree crowns in high-spatial resolution aerial images, Canadian Journal Remote Sensing, 21(3):274–284. Gougeon, F.A., and D.G. Leckie, 2003. Forest information extraction from high spatial resolution images using an individual tree crown approach, Information Report BC-X-396, Natural Resources Canada, Canadian Forest Service (F.A. Gougeon and D.G. Leckie, editors), Victoria, B.C., Pacific Forestry Centre, pp. 1–17. Huang, S.B., and R. Shibasaki, 1995. Development of genetic algorithm hill-climbing method for spatio-temporal interpolation, Proceedings of The 6th symposium on Functional Image Information System IIS, April, Tokyo, Japan, pp. 81–86. Kangas, A., and M. Maltamo, 2006. Forest Inventory: Methodology and Applications (Managing Forest Ecosystem), Springer, The Netherlands, 362 p. Kass, M., A. Witkin, and D. Terzopoulos, 1987. Snakes: Active contour models, International Journal of Computer Vision, 1(4):321–331. Ke, Y., and L.J. Quackenbush, 2010. A comparison of three methods for automatic tree crown detection and delineation methods from high spatial resolution imagery, International Journal of Remote Sensing, in press. Larmar, W.R., J.B. McGraw, and T.A. Warner, 2005. Multitemporal censusing of a population of eastern hemlock (Tsuga Canadensis L.) from remotely sensed imagery using an automated segmentation and reconciliation procedure, Remote Sensing of Environment, 94(1):133–143. Leckie, D.G., F.A. Gougeon, N. Walsworth, and D. Paradine, 2003. Stand delineation and composition estimation using semiautomated individual tree crown analysis, Remote Sensing of Environment, 85(3):355–369. Leckie, D.G., C. Jay, F.A. Gougeon, R. Sturrock, and D. Paradine, 2004. Detection and assessments of trees with Phellinus weirii (Laminated root rot) using high resolution multispectral imagery, International Journal of Remote Sensing, 25(4):793–818. Leckie, D.G., F.A. Gougeon, S. Tinis, T. Nelson, C.N. Burnet, and D. Paradine, 2005. Automated tree recognition in old growth conifer stands with high resolution digital Imagery, Remote Sensing of Environment, 94(3):311–326. Li, C., C.Y. Kao, J.C. Gore, and Z. Ding, 2007. Implicit active contours driven by local binary fitting energy, Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition, 17–22 June, Minneapolis, Minnesota, pp. 1–7.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Persson, Å., J. Holmgren, and U. Söderman, 2002. Detecting and measuring individual trees using an Airborne Laser Scanner, Photogrammetric Engineering & Remote Sensing, 68(9):925–932. Pollock, R.J., 1999. Individual tree recognition based on a synthetic tree crown image model, Proceedings of the International Forum on Automated Interpretation of High Spatial Resolution Digital Imagery for Forestry (D.A. Hill and D.G. Leckie, editors), 10–12 February 1998, Victoria, British Columbia, Canadian Forest Service, Pacific Forestry Centre, pp. 25–34. Pouliot D.A., and D.J. King, 2005. Approaches for optimal automated individual tree crown detection in regenerating coniferous forests, Canadian Journal of Remote Sensing, 31:255–267. Pouliot, D.A., D.J. King, F.W. Bell, and D.G. Pitt, 2002. Automated tree crown detection and delineation in high-resolution digital camera imagery of coniferous forest regeneration, Remote Sensing of Environment, 82(2):322–334. Pouliot, D.A., D.J. King, and D.G. Pitt, 2005, Development and evaluation of an automated tree Detection - Delineation algorithm for monitoring regenerating coniferous forests, Canadian Journal of Forest Research, 35:2332–2345. Quackenbush, L.J., P.F. Hopkins, and G.J. Kinn, 2000. Using template correlation to identify individual trees in high resolution imagery, Proceedings of the 2000 ASPRS Annual Conference, 22–26 May, Washington, D.C., American Society for Photogrammetry and Remote Sensing, Bethesda, Maryland, unpaginated CD-ROM. Tsai, A., A.J. Yezzi, C. Tempany, D. Tucker, A. Fan, W.E.L. Grimson, and A.S. Willsky, 2003. A shape-based approach to the segmentation of medical imagery using level sets, IEEE Transactions on Medical Imaging, 22(2):137–154. Walsworth, N.A., and D.J. King, 1999. Comparison of two tree apex delineation techniques, Proceedings of the International Forum on Automated Interpretation of High Spatial Resolution Digital Imagery for Forestry (D.A. Hill and D.G. Leckie, editors), 10–12 February 1998, Victoria, British Columbia, Canada, pp. 93–104. Wang, L., P. Gong, and G.S. Biging, 2004. Individual tree-crown delineation and treetop detection in high-spatial resolution aerial imagery, Photogrammetric Engineering & Remote Sensing, 70(3):351–357. Wulder, M., K.O. Niemann, and D.G. Goodenough, 2000. Local maximum filtering for the extraction of tree locations and basal area from high spatial resolution imagery, Remote Sensing of Environment, 73(1):103–114. Xu, C., D. Pham, and J. Prince, 2000. Image segmentation using deformable models, Handbook of Medical Imaging, Volume 2: Medical Image Processing and Analysis (J. Fitzpatrick and M. Sonka, editors), SPIE Press, London, United Kingdom, pp. 129–174. (Received 17 June 2009; accepted 25 September 2009; final version 04 February 2010)
O c t o b e r 2 0 1 0 1181
