Multivariate Image Texture by Multivariate ... - Semantic Scholar

Download PDF
1 downloads 0 Views 1MB Size Report
Comment
color texture (Jain and Healey, 1998). The between-band texture was found to provide ... Kerry and Oliver, 2003). Multivariate Variogram for Multispectral Image.
147-157_06-124.qxd

1/13/09

4:10 PM

Page 147

Multivariate Image Texture by Multivariate Variogram for Multispectral Image Classification Peijun Li, Tao Cheng, and Jiancong Guo

Abstract Traditional image texture measure usually allows a texture description of a single band of the spectrum, characterizing the spatial variability of gray-level values within the singleband image. A problem with the approach while applied to multispectral images is that it only uses the texture information from selected bands. In this paper, we propose a new multivariate texture measure based on the multivariate variogram. The multivariate texture is computed from all bands of a multispectral image, which characterizes the multivariate spatial autocorrelation among those bands. In order to evaluate the performance of the proposed texture measure, the derived multivariate texture image is combined with the spectral data in image classification. The result is compared to classifications using spectral data alone and plus traditional texture images. A machine learning classifier based on Support Vector Machines (SVMs) is used for image classification. The experimental results demonstrate that the inclusion of multivariate texture information in multispectral image classification significantly improves the overall accuracy, with 5 to 13.5 percent of improvement, compared to the classification with spectral information alone. The results also show that when incorporated in image classification as an additional band, the multivariate texture results in high overall accuracy, which is comparable with or higher than the best results from the existing single-band and two-band texture measures, such as the variogram, cross variogram and Gray-Level Co-occurrence Matrix (GLCM) based texture. Overall, the multivariate texture provides the useful spatial information for land-cover classification, which is different from the traditional single band texture. Moreover, it avoids the band selection procedure which is prerequisite to traditional texture computation and would help to achieve high accuracy in the most classification tasks.

Introduction It is often found that classes of land-cover may be discriminated from multispectral imagery on the basis of their spectral signature, but also of their texture (Jensen, 1982; Franklin and Peddle, 1990; Gong et al., 1992; Franklin et al., 2001; Coburn and Roberts, 2004). Image texture, as one of the important

Peijun Li and Jiancong Guo are with the Institute of Remote Sensing and GIS, Peking University, Beijing 100871, P R China ([email protected]). Tao Cheng is with the Department of Earth and Atmospheric Sciences, University of Alberta, Edmonton, Alberta, Canada T6G 2E3. PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

spatial information types from the image, has been widely used in remote sensing image classification, image segmentation, and other fields of information processing, which significantly improves the overall accuracy in most cases (e.g., Jensen, 1982; Cohen, 1990; Franklin and Peddle, 1990; Gong et al., 1992; Ryherd and Woodcock, 1996; Franklin et al., 2001; Coburn and Roberts, 2004). The traditional image texture measure, such as the classical Gray-Level Co-occurrence Matrix (GLCM) method (Haralick et al., 1973), usually allows a texture description of a single spectral band, which only characterizes the spatial variability (or spatial autocorrelation) of the spectral feature (e.g., Digital Number) within the band. Thus, texture is usually extracted individually from a single band. However, texture features from the different bands of a multispectral image are generally different and have different discriminating capability of land-cover types. Thus, while the image texture is used in the multispectral image classification, an individual band usually has to be first decided for texture computation in order to obtain a high accuracy (Marceau et al., 1990; Arbarca-Hernandez and ChicaOlmo, 1999; Berberoglu et al., 2000; Chica-Olmo and ArbarcaHernandez, 2000; Coburn and Roberts, 2004). This could be accomplished either by selecting one directly from the original multispectral bands (Marceau et al., 1990; Coburn and Roberts, 2004), or by first conducting the Principal Component Analysis (PCA) on the original multispectral image and then selecting one or two PCs (e.g., Arbarca-Hernandez and Chica-Olmo, 1999; Berberoglu et al., 2000; Chica-Olmo and Arbarca-Hernandez, 2000). However, a problem with the approach is that it only uses the texture information from the selected bands or components, not accounting for the spatial autocorrelation among the bands. Recently, some two-band (or between-band) texture measures were proposed to express the between-band spatial correlation, such as cross variogram, pseudo cross variogram (Chica-Olmo and Arbarca-Hernandez, 2000), and opponent color texture (Jain and Healey, 1998). The between-band texture was found to provide very useful spatial information for discrimination between different land-cover types (ChicaOlmo and Arbarca-Hernandez, 2000). Most optical remote sensing images consist of multiple bands (e.g., multispectral image), which record the information from different parts of the electromagnetic spectrum.

Photogrammetric Engineering & Remote Sensing Vol. 75, No. 2, February 2009, pp. 147–157. 0099-1112/09/7502–0147/$3.00/0 © 2009 American Society for Photogrammetry and Remote Sensing February 2009

147

147-157_06-124.qxd

1/13/09

4:10 PM

Page 148

Multispectral bands can be considered as the multivariate variables which have spatial correlations to some extent. Spatial autocorrelation of multivariate data has successfully been used for the analysis of ecological community, soil properties, and geological objects (Webster, 1973; Mackas, 1984; Sokal, 1986; Oden and Sokal, 1986; Harff and Davis, 1990). In those cases, the spatial variations of a set of variables were summarized by a structural function. However, spatial autocorrelation among the bands of a multispectral image is seldom exploited in remote sensing. Although some multivariate texture (multichannel texture) measures were proposed for color image processing (e.g., Shafarenko et al., 1997; Tseng and Lai, 1999), there is little research on multivariate texture for remote sensing applications (Lira and Rodriguez, 2006). The objective of this paper is to characterize and compute the multivariate spatial autocorrelation among all bands of a multispectral image by the multivariate variogram (Bourgault and Marcotte, 1991). The multivariate spatial autocorrelation is then proposed as a multivariate texture measure, and combined with spectral information for image classification, in order to improve the overall accuracy.

Methods Geostatistics are a set of techniques for spatial data analysis and have been introduced in remote sensing since late 1980s (Curran and Atkinson, 1998). The variogram, which is the basic function in geostatistics, was widely used in image texture characterization and subsequent image classification (see the review paper by Atkinson and Lewis, 2000). Most previous studies on geostatistical image texture extracted image texture by the variogram from a single band. However, little effort has been directed to extract texture across different bands (Chica-Olmo and AbarcaHernandez, 2000). Bourgault and Marcotte (1991) first formally presented the multivariate variogram, which summarizes the spatial variations of the multiple variables (e.g., multispectral bands) and highlights the multivariate spatial autocorrelation of the multiple variables. Some authors have reported successful applications of the multivariate variogram to the multivariate analysis of the soil properties (Lark, 1998; Kerry and Oliver, 2003). Multivariate Variogram for Multispectral Image The Digital Number (DN) of a remote sensing image is geostatistically considered as a regionalized variable, characterized by both random and spatial correlation aspects (Ramstein and Raffy, 1989; Chica-Olmo and Abarca-Hernandez, 2000). The variogram for a single band image (univariate variogram) is expressed as follows (Matheron, 1971): g( h) ⫽

1 E5DN(x) ⫺ DN(x ⫹ h)62, 2

(1)

where g(h) represents half of the mathematical expectation of the quadratic increments of pixel pair values at a distance h, i.e., semivariance; g(h) is a vectorial function depending on the modulus and the angle of the distance vector h between the pixels x and x ⫹ h. The experimental univariate variogram for a single band can be expressed as: gexp ( h) ⫽

1 N( h) g (DN(x) ⫺ DN(x ⫹ h))2, 2N( h) i⫽1

(2)

where N(h) is the number of distant pair h. This expression was widely used to extract texture from remote sensing imagery for image classification (e.g., Miranda et al., 1998; 148

February 2009

Arbarca-Hernandez and Chica-Olmo, 1999; Berberoglu et al., 2000; Chica-Olmo and Arbarca-Hernandez, 2000). By extending Equation 2 to the multivariate space, the multivariate variogram was proposed to characterize the multivariate spatial autocorrelation among multiple variables (Bourgault and Marcotte, 1991). For a multispectral image which can be modeled as multivariate data, each pixel can be viewed as a vector of p bands (variables): DN(x) ⫽ [dn1 (x),dn2 (x), Á ,dni (x), Á ,dnp (x)]1*p

(3)

where DN(x) is a row vector of p values at the pixel x, and dni(x) is the pixel value of the ith band. The multivariate variogram is defined as follows (Bourgault and Marcotte, 1991): gm ( h) ⫽

1 E[( DN(x) ⫺ DN(x ⫹ h))M( DN(x) ⫺ DN(x ⫹ h))T ] (4) 2

where gm(h) represents the multivariate variogram at distance h, h is the distance vector, DN(x) and DN(x ⫹ h) are pixel vectors at positions x and x ⫹ h separated by the lag h, (.)T is the transpose of the matrix, M is a symmetric positive definite matrix of p ⫻ p used as a metric defining the relations between the bands (Bourgault and Marcotte, 1991). Examples of such metrics are: the identity matrix (Euclidean), the inverse of the variance-covariance matrix (Mahalanobis), etc. The experimental multivariate variogram is estimated by averaging the multivariate distance squared: gm_exp ( h) ⫽

1 N( h) g [( DN(xi ) ⫺ DN(xi ⫹ h)) 2N( h) i⫽1 M( DN(xi ) ⫺ DN(xi ⫹ h))T ].

(5)

where N(h) is the number of distant pairs h, DN(xi) and DN(xi ⫹ h) are the pixel vectors at xi and xi ⫹ h, (.)T is the transpose of the matrix. By analogy with the traditional univariate variogram (Equation 2), which describes the spatial variability within a band and has been widely used in the extraction of texture from a single band, the multivariate variogram characterizes the multivariate spatial variability among all the bands of a multispectral image and can be also used to compute the multivariate texture from the multispectral image. Multivariate Texture Computation As mentioned above, Equation 5 can be used to compute multivariate texture from a multispectral image. Therein, different matrices, M, can be used. In the present study, two commonly used matrices, the identity matrix (i.e., Euclidean distance was used as distance measure) and the inverse of the variance-covariance matrix (Mahalanobis distance) were used in Equation 5. In the case of Euclidean distance, the experimental multivariate variogram is simply the sum of the univariate experimental variograms (Bourgault and Marcotte, 1991). An image texture by the multivariate variogram in Equation 5 was computed within a neighborhood using a moving window for a specified lag distance h. The multivariate texture value computed from all multispectral bands was assigned to the central pixel of the moving window. Thus, there are two parameters to be considered when computing multivariate texture using the multivariate variogram: window size and lag distance (including size and direction). The selection of an appropriate window size is one of the most important steps in the calculation of the multivariate texture, since too large or too small window size can not reflect the real texture properties PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

147-157_06-124.qxd

1/13/09

4:10 PM

Page 149

(Mather et al., 1998). This is usually done by trial and error. The window size for texture computation that maximizes the classification accuracy is usually considered as the appropriate one in most texture analysis studies. However, Coburn and Roberts (2004) found that, for images that contain complex spatial structures, there is no optimal window size that adequately characterizes all the texture information present in the image. They showed that the inclusion of texture features of more than one single window size in the image classification provided higher classification accuracies, compared to the addition of the texture extracted using a single window size. The lag distance h also affects the multivariate variogram texture values. The lag distance can be measured in individual directions, or for all directions (omnidirectional) within a moving window. If the spatial distribution of the DNs of the image reveals obvious anisotropy, the directional variogram should be taken into account. Otherwise, it is preferable to choose only the omnidirectional variogram. Omnidirectional variogram texture is usually produced by averaging the texture features from all directions (e.g., NS, EW, NE-SW, NW-SE). In most cases, a lag distance of one pixel was used for geostatistical texture computation (Arbarca-Hernandez and Chica-Olmo, 1999; Berberoglu et al., 2000; Chica-Olmo and Abarca-Hernandez, 2000), since it is considered as the one that best describes the radiometric differences in the immediate neighborhood of the central pixel (Chica-Olmo and Abarca-Hernandez, 2000). Nevertheless, other h values can also provide useful information for land-cover discrimination (Lark, 1996). In this paper, the omnidirectional multivariate variogram was used. Two cases were considered. In the first case, the texture with a single window size and a single lag distance (i.e., single scale) was used in the combined spectral and texture classification. Several different window sizes with a lag h ⫽ 1 were individually used for multivariate texture extraction and compared in terms of the overall accuracy, and the one with the highest classification accuracy was selected. In the second case, the texture images with multiple window sizes and multiple lag distances (i.e., multiple scales) were combined in image classification. Specifically, in order to use all spatial information in different window sizes, all lag distances less than half the window size were used to compute the texture values. In this case, for a specified window size, several multivariate texture values using different lag distances can be obtained for a pixel. For example, for a moving window of 7 pixels ⫻ 7 pixels, lag distances of 1, 2, and 3 pixels were used to compute the multivariate variogram texture. As a result, three texture images were produced for this window size. Image Classification by SVM Texture features extracted by using the multivariate variogram were incorporated to image classification as the additional bands. Since when texture features at multiple scales were included in combined spectral and texture classification, the obtained data would be high-dimensional. An appropriate classifier which can deal with high-dimensional data was required. Thus, the Support Vector Machines (SVMs), a recently developed statistical learning method, was used as the classifier in this study. The SVM classifier can effectively handle high-dimensional data with a limited number of training samples. In many cases reported, the SVMs have been found to perform with higher classification accuracy than the traditional supervised classifiers, such as Maximum Likelihood classifier (e.g., Gualtieri and Cromp, 1998; Huang et al., 2002; Melgani and Bruzzone, 2004). The SVMs classifier is based on the statistical learning theory (Vapnik, 2000) and tries to find an optimal hyperplane PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

as a decision function in the high dimensional space (Boser et al., 1992, Cristianini and Shawe-Taylor, 2000). In the case where the classes are linearly separable, the SVM selects from the infinite number of linear decision boundaries the one that minimizes the generalization error. If the classes are not linearly separable, the SVM tries to find the hyperplane that maximizes the margin while, at the same time, minimizing a quantity proportional to the number of misclassification errors. The SVM can also be extended to allow for nonlinear decision surfaces. In this case, the input data are projected into a high-dimensional feature space using kernel functions (Boser et al., 1992; Vapnik, 2000) and formulating a linear classification problem in that feature space. A number of kernels are discussed in the literature, but it is not clear how to choose one which gives the best generalization for a particular problem (Pal and Mather, 2005). In the present study, the Radial Basic Function (RBF), which is one of the commonly used kernels, is used as the kernel function. SVMs were initially designed for binary (two-class) problems. When dealing with multiple classes, an appropriate multi-class method is needed. A number of methods are suggested in the literature to create multi-class classifiers using two-class methods (Hsu and Lin, 2002). In this study, a “one against one” approach (Knerr et al., 1990) is used. Comparison with Existing Texture Measures In order to further evaluate and validate the proposed texture measure, the existing texture measures were also compared: univariate variogram, cross variogram, and classical GLCM texture. The univariate variogram texture was computed for a single band using Equation 2. The cross variogram quantifies the joint spatial variability (cross correlation) between two bands at pixel locations xi and xi ⫹ h, which is defined as half of the average product of the h-increments relative to the spectral bands j and k (Journel and Huijbregts, 1978; Chica-Olmo and Abarca-Hernandez, 2000): gjk( h) ⫽

1 N( h) g (dnj(xj) ⫺ dnj(xi ⫹ h)) 2N(h) i⫽1 (dnk(xi) ⫺ dnk(xi ⫹ h)).

(6)

The cross variogram texture was computed for each band pair of the multispectral image. The Grey-Level Co-occurrence Matrix (GLCM) is a spatial dependence matrix of relative frequencies in which two neighboring pixels that are separated by a given distance and a given angle, occur within a moving window (Haralick et al., 1973). Eight texture measures derived from co-occurrence matrices were calculated in the study: mean, variance, homogeneity, contrast, dissimilarity, entropy, second moment, and correlation. The texture measure with the highest overall accuracy was selected for comparison.

Results The performance of the proposed multivariate texture was evaluated through the image classification which is a common way to evaluate a texture measure in remote sensing. The multivariate texture produced by the proposed method was incorporated into image classification as an additional band and evaluated in terms of the overall classification accuracy. The classification using spectral information alone was carried out as the benchmark classification result. For comparison, texture features by the traditional variogram, cross-variogram, and GLCM were also incorporated into the classification process as additional bands, respectively. February 2009

149

147-157_06-124.qxd

1/13/09

4:10 PM

Page 150

The image classification was conducted by the SVM classifier with a RBF kernel. In the present study, an independent validation data set was used to determine the optimal C and g values for the classification. Specifically, a SVM classifier was first trained using the training samples and then evaluated for the optimal C and g using the independent validation set. The SVM classifier with the optimal C and g values was finally used to classify the whole image data. Two real remote sensing datasets were used in the study to evaluate and validate the proposed method. The training, validation and test samples were independently selected from the image.

3 4 5 6 7 8

Example 1 In the first experiment, a sub-scene of 1,500 pixel ⫻ 1,200 pixel of a SPOT5 multispectral image covering the northwest suburban area of Beijing (Figure 1) was selected for the test. The image was acquired in October 2004. Eight land-cover types were recognized in the area, and the samples used were summarized in Table 1. In order to choose an appropriate window size for final texture computation and subsequent image classification, several window sizes, 3 ⫻ 3, 5 ⫻ 5, 7 ⫻ 7 and 9 ⫻ 9 pixels, with lag distance of one pixel were respectively used to extract image texture and compared in terms of the classification accuracy. It was found that the window size of 7 pixels ⫻ 7 pixels was the one with the highest overall accuracy. Thus the window size of 7 pixels ⫻ 7 pixels was

considered as the appropriate window size for texture computation. Figure 2 shows the texture images produced by traditional variogram (Figure 2b) from SPOT-band 3 (Near Infrared (NIR) band) and by multivariate variogram from all bands (Figure 2c). From the figure, it can be seen that although two texture images show some similarities, multivariate texture is more sensitive to the different spatial patterns, and has less edge effect. In order to evaluate multi-scale texture for image classification, the multivariate texture features with several window sizes (from 3 ⫻ 3 to 7 ⫻ 7 pixels) and all lag distances less than half of the window size were also used in image classification. The classification results by SVM using spectral information alone and plus the different texture features are

Figure 1.

150

February 2009

SPOT5 NIR

TABLE 1. SAMPLES

1 2

FOR SPOT5 IMAGE

CLASSIFICATION

Class

Training

Validation

Test

Bare land Artificial woodland Forest Crop field Rural Shadow Residential Water

2019 4441

1156 1783

5888 10353

2595 1052 1238 369 1778 749

996 504 860 160 567 314

6705 4663 5088 773 3166 2084

band image of the study area (northwest Beijing suburban area).

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

147-157_06-124.qxd

1/13/09

4:11 PM

Page 151

summarized in Table 2. From Table 2, the joint use of spectral and multivariate texture information leads to a significant increase in overall accuracy. The accuracy of classification using spectral information alone is 83.67 percent. By adding the multivariate texture, the classification accuracy increases by 5.19 percent for Euclidean distance and 4.08 percent for Mahalanobis distance, compared to the spectral classification. The difference in Kappa coefficient is even more significant, with the improvements of 6.35 percent and 5.02 percent, respectively. By comparing the results for the spectral classification and the classification adding multivariate texture (Table 3), both producer’s accuracies and user’s accuracies of most classes considerably increase, except the classes Shadow and Water with almost the same accuracies in both classifications. Plate 1 shows the classification results by spectral information alone and plus multivariate texture. It can be seen that when multivariate texture was included in image classification, the land-cover classes become more homogeneous. Moreover, the confusion between some classes, such as between classes Bareland and Rural, Residential and Rural, Artificial Woodland and Forest are significantly reduced, thus producing a higher overall accuracy. This validates the performance of the proposed method. The traditional GLCM texture, variogram texture and cross-variogram texture were extracted from the image and included in combined classification for comparison. For these texture measures, the appropriate band or band pair (for cross-variogram texture) must be first decided. We calculated texture images with single window (single scale) and multiple window sizes (multiple scales) using these measures individually from all bands and band pairs, and compared them in terms of overall accuracy in combined spectral and texture classification. It was found that the NIR band and a combination of Green and NIR bands of the SPOT5 image resulted in the highest accuracies in the combined spectral and texture classification, among all bands and band pairs, respectively. The highest classification accuracies by including the variogram and the cross-variogram at single scale are 88.69 percent and

TABLE 2. CLASSIFICATION ACCURACIES USING SPOT5 MULTISPECTRAL DATA AND TEXTURE FEATURES Band combinations Spectral Spectral Spectral Spectral Spectral Spectral Spectral Spectral Spectral Spectral

Figure 2. Portions of (a) SPOT5 NIR band image, (b) texture from band 3 by univariate variogram, and (c) texture from all four bands by multivariate variogram.

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

⫹ ⫹ ⫹ ⫹ ⫹ ⫹ ⫹ ⫹ ⫹

MV-7_1 (Euclidean) MV-7_1 (Mahalanobis) NRV-7_1 CoV-7_1 GLCM(homogeneity)-7 MV-3,5,7 (Euclidean) MV-3,5,7 (Mahalanobis) NRV-3,5,7 GLCM(homogeneity)-3,5,7

OA

(%)

83.67 88.86 87.75 88.69 88.31 88.90 89.45 88.29 89.19 88.83

Kappa 0.8017 0.8652 0.8519 0.8631 0.8542 0.8660 0.8725 0.8584 0.8692 0.8650

OA: Overall Accuracy, Spectral: four SPOT5 bands, MV-7_1: Multivariate texture with window size of 7 ⫻ 7 pixels and lag of 1 pixel, MV3,5,7: Multivariate textures with window sizes of 3 ⫻ 3 pixels to 7 ⫻ 7 pixels and all lags less than half window sizes, NRV-7_1: Traditional univariate variogram texture of NIR band with window size of 7 ⫻ 7 pixels and lag of 1 pixel, NRV-3,5,7: Univariate variogram textures with window sizes of 3 ⫻ 3 pixels to 7 ⫻ 7 pixels and all lags less than half window sizes, CoV-7_1: cross variogram texture with window size of 7 ⫻ 7 pixels and lag of 1 pixel, GLCM(Homogeneity)-7: Homogeneity texture with a window size of 7 ⫻ 7 pixels, GLCM(Homogeneity)-3,5,7, Homogeneity textures with multiple window size from 3 ⫻ 3 pixels to 7 ⫻ 7 pixels, Euclidean: identity matrix, Mahalanobis: inverse of variance-covariance matrix.

February 2009

151

147-157_06-124.qxd

1/13/09

4:11 PM

Page 152

TABLE 3. CLASSIFICATION RESULTS USING

Bare land

SPOT5 SPECTRAL INFORMATION ALONE AND PLUS MULTIVARIATE TEXTURE (EUCLIDEAN DISTANCE) WINDOW SIZE 7 PIXELS ⫻ 7 PIXELS (ALL IN PERCENT)

Artificial woodland

Forest

89.47 95.30 74.26 98.19 72.29 94.39 OA, 83.67; Kappa, 80.17 Spectral and PA 95.79 95.59 80.7 texture UA 99.37 77.9 93.57 classification OA, 88.86; Kappa, 86.52 OA: Overall Accuracy; PA: Producer’s Accuracy; UA: User’s Accuracy Spectral classification

PA

UA

BY

Crop field

Rural

Shadow

Residential

Water

68.45 98.12

76.61 80.34

100 99.87

76.47 68.20

95.92 99.80

74.91 98.09

85.18 90.16

100 99.74

90.18 83.8

96.21 99.9

Plate 1. (a) portions of SPOT5 false color composite image (Band 4, 3, 2 as R, G, B), (b) classification results by spectral information, and (c) classification results by spectral information and multivariate texture. Color assignments: 1, Bare land; 2, Artificial woodland; 3, Forest; 4, Crop field; 5, Rural; 6, Shadow; 7, Residential; 8, Water.

88.31 percent, respectively, all slightly lower than or comparable with that of including multivariate texture at single scale (Table 2). Eight texture measures from GLCM were also computed from the NIR band. The classification results showed that the inclusion of different GLCM texture measures produced the overall accuracies from 86.99 percent to 88.90 percent, Kappa coefficients from 0.843 to 0.8660. Among all GLCM texture measures, the measure Homogeneity achieved the highest overall accuracy of 88.90 percent and corresponding Kappa Coefficient of 0.8660, which is comparable with the multivariate texture (Table 2). 152

February 2009

The results also showed that the multi-scale texture has the performance similar to the single-scale texture in the combined classification in this example. By adding the multivariate texture features with multiple scales, the classification accuracy slightly improved, compared with the classification by adding the multivariate texture with a single scale (Table 2). Table 4 showed the overall accuracies for the classification incorporating variogram texture with multiple scales from different bands. It can be seen from the table that variogram texture from different bands produced different overall accuracies in the combined classification. The variogram texture from the NIR band PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

147-157_06-124.qxd

1/13/09

4:11 PM

Page 153

TABLE 4. ACCURACIES FOR CLASSIFICATION ADDING TRADITIONAL VARIOGRAM TEXTURE FROM INDIVIDUAL BANDS OF SPOT5 IMAGE USING MULTIPLE WINDOWS Band combinations Spectral Spectral Spectral Spectral

⫹ ⫹ ⫹ ⫹

GV-3,5,7 RV-3,5,7 NRV-3,5,7 SWV-3,5,7

OA

(%)

87.72 88.02 89.19 87.58

GV:

Kappa 0.8514 0.8550 0.8692 0.8496

variogram texture from Green band; RV: variogram texture from Red band; NRV: variogram texture from Near Infrared band; SWV: variogram texture from Shortwave Infrared band. The multiple window sizes from 3 ⫻ 3 to 7 ⫻ 7 pixels and all lag distances less than half of window sizes were used for texture extraction.

produced the highest overall accuracy among all bands, which is slightly lower than that of multivariate texture. The results also showed that the inclusion of GLCM texture feature Homogeneity from the NIR band with multiple window sizes in the image classification did not improve the overall accuracy, compared with the inclusion of Homogeneity texture at single window size (Table 2). From Table 2 it can be seen that the identity matrix (Euclidean) and the inverse of variance-covariance matrix (Mahalanobis) used in multivariate texture extraction in Equation 5 have almost the same performance for image classification. Example 2 An Ikonos multispectral image with 4 m resolution was used in the second experiment. The image was acquired on 05 October 2002, which covers the Xiangshan area in northwest Beijing. An image subset of 750 pixels ⫻ 750 pixels was used in the study (Figure 3). Training, validation

Figure 3. Ikonos NIR band image of the study area (Xiangshan, Beijing).

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

February 2009

153

147-157_06-124.qxd

1/13/09

TABLE 5. SAMPLES

1 2 3 4 5 6 7 8

4:11 PM

Page 154

FOR IKONOS IMAGE

CLASSIFICATION

Class

Training

Validation

Test

Bare land Residential Bush nursery Woodland Crop field Rural Water Vinyl house

590 925 1083 832 378 1252 104 293

610 983 1226 931 416 1314 118 304

1757 2299 3067 1806 817 2402 320 896

and test samples used for this experiment were shown in Table 5. After the comparison of the classification results of including the multivariate texture using from window sizes from 3 ⫻ 3 to 9 ⫻ 9 pixels, all with a lag distance of one pixel, it was found that the window size of 5 pixels ⫻ 5 pixels for multivariate texture extraction have the highest accuracy and thus used in this experiment. The multivariate texture features with several window sizes (from 3 ⫻ 3 to 9 ⫻ 9 pixels) and all lag distances less than half of the specified window size were also combined with spectral information for image classification. Figure 4 shows the texture images from univariate variogram and multivariate variogram. As in the first example, from Figure 4, multivariate texture showed more sensitive to the different spatial patterns than univariate texture. It is shown in Table 6 that the combination of spectral and multivariate texture information leads to a significant increase in overall classification accuracy, compared to spectral classification. The accuracy of classification using spectral information alone is 78.22 percent. By adding the multivariate texture with the window size 5 pixels ⫻ 5 pixels and the lag distance of one pixel, the overall classification accuracies is 84.68 percent (Euclidean) and 84.47 percent (Mahalanobis), with the improvements of 6.46 percent and 6.25 percent, respectively, compared to the spectral classification. By comparison of classification results (Table 7), it was found that the accuracies of classes Rural, Bareland, and Residential significantly increased, while other classes have the similar results to the spectral classification. Plate 2 showed the spectral classification and the classification including multivariate texture. From the Plate, when multivariate texture is included, the classes become more homogeneous. However, there are some misclassifications for linear features. This is probably caused by the edge effect of texture (Ferro and Warner, 2002). The texture features by the traditional variogram from individual bands were also incorporated into image classification for comparison. The highest accuracy of classification by adding the traditional variogram texture from the NIR band is 84.45 percent, with almost the same accuracy as the classification incorporating multivariate texture (84.68 percent) (Table 6). The classification results showed that cross variogram texture from Ikonos band 1 (Blue) and band 3 (Red) produced the highest overall accuracy (83.49 percent) (Table 6), which is slightly lower than that of multivariate texture (Table 6). Eight GLCM texture measures were respectively computed from the NIR band using a window size of 5 pixels ⫻ 5 pixels. The inclusion of different GLCM texture measures produces overall accuracies from 82.15 percent to 85.63 percent, Kappa coefficients from 0.7874 to 0.8288. The measure Homogeneity was also found to produce the highest overall accuracy of 85.63 percent and a kappa coefficient of 0.8288, which is comparable with multivariate texture (Table 6). 154

February 2009

Figure 4. Portions of (a) Ikonos NIR band image, (b) texture image from band 4 by univariate variogram, and (c) texture from all 4 bands by multivariate variogram.

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

147-157_06-124.qxd

1/13/09

4:11 PM

Page 155

TABLE 6. CLASSIFICATION ACCURACIES USING IKONOS MULTISPECTRAL DATA AND TEXTURE FEATURES Band combinations

OA

Spectral Spectral Spectral Spectral Spectral Spectral Spectral Spectral Spectral Spectral

78.22 84.68 84.47 84.45 83.49 85.63 91.84 89.96 91.89 88.04

⫹ ⫹ ⫹ ⫹ ⫹ ⫹ ⫹ ⫹ ⫹

MV-5_1 (Euclidean) MV-5_1 (Mahalanobis) NRV-5_1 CoV-5_1 GLCM (Homogeneity)-5 MV-3,5,7,9 (Euclidean) MV-3,5,7,9 (Mahalanobis) NRV-3,5,7,9 GLCM (Homogeneity)-3,5,7,9

(%)

Kappa 0.74 0.82 0.8149 0.8149 0.8033 0.8288 0.9028 0.8804 0.9035 0.8575

OA:

Overall Accuracy; Spectral: four Ikonos bands; MV-5_1: Multivariate texture with the window size of 5 by 5 pixels and lag of 1 pixel; MV-3,5,7,9: Multivariate textures with window sizes of 3 ⫻ 3 pixels to 9 ⫻ 9 pixels and all lags less than half window sizes; NRV-5_1: Traditional univariate variogram texture of NIR band with window size of 5 ⫻ 5 pixels and lag of one pixel; NRV-3,5,7,9: Univariate textures with window sizes of 3 ⫻ 3 pixels to 9 ⫻ 9 pixels and all lags less than half window sizes; CoV-5_1: Cross variogram texture with window size of 5 ⫻ 5 pixels and lag of one pixel; GLCM(Homogeneity)-5: Homogeneity texture with a window size of 5 ⫻ 5 pixels; GLCM(Homogeneity)-3,5,7,9: Homogeneity textures with multiple window size from 3 ⫻ 3 pixels to 9 ⫻ 9 pixels; Euclidean: identity matrix; Mahalanobis: inverse of variancecovariance matrix.

It is worth to note that multi-scale texture features exhibit very good performance in image classification, compared to the single-scale texture features. By adding multivariate texture features with multiple window sizes and multiple lag distances (i.e., multiple scales), classification accuracy significantly improved, with 6.16 percent (Euclidean) and 5.49 percent (Mahalanobis) higher than the classification by adding the multivariate texture with a single window size and a single lag distance (i.e., single scale), and with 13.62 percent and 11.74 percent higher than the spectral classification (Table 6). The addition of variogram textures of multiple scales from different bands in classification obtained different accuracies, ranging from 89.76 percent (Kappa Coefficient 0.8782) to 91.89 percent (Kappa Coefficient 0.9035) (Table 8). The addition of variogram textures of multiple scales from the NIR band produced the highest overall accuracy (91.89 percent), which is the same as that of inclusion of multivariate texture (Table 6 and Table 8). The inclusion of GLCM textures with multiple window sizes from the NIR band was also found to produce higher accuracy than the inclusion of the texture of single window size. However, the inclusion of GLCM texture with multiple window sizes achieved lower overall accuracies than that of inclusion of multivariate

texture images with multiple window sizes and multiple lag distances, 3.8 percent (Euclidean) and 1.92 percent (Mahalanobis), respectively. From Table 6, it can be seen that although different matrices M used in multivariate texture computation in Equation 5, i.e., the identity matrix (Euclidean) and the inverse of variance-covariance matrix (Mahalanobis), they have almost the same performance for image classification. The two examples drew the same conclusion on the matirce M. Thus, the selection of M matrices does not affect the performance of the multivariate texture.

Summary and Conclusions The standard approach to texture analysis is to process a single spectral band with a moving window of the fixed size. This kind of image texture only reflects the univariate spatial variability of a single band image. In this study a multivariate texture measure based on the multivariate variogram was proposed, which is computed from all the bands of a multispectral image and characterizes the multivariate spatial autocorrelation among those bands. Thus, the multivariate texture, which is different from the traditional single-band texture, provides useful spatial information that can be used for discriminating between land-cover types. The proposed multivariate texture was evaluated in the combined spectral and texture classification using two real remote sensing data. The results showed that the inclusion of multivariate texture in the combined classification considerably improved the overall accuracy, by 5 to 13.5 percent, over the spectral classification. Compared with traditional texture measures such as variogram, GLCM texture, the multivariate texture achieved the overall accuracy higher than or at least the same as the best results produced by these traditional texture measures in both the single scale and multiple cases. Generally, while applied in image classification, multivariate texture produces a stable high overall accuracy in the most cases. Moreover, since multivariate texture is derived from all bands of a multispectral image, by a single function (i.e., multivariate variogram), it avoids the band selection when used for image classification, which is required for the traditional texture measures. This is another advantage of the multivariate texture over the traditional texture measures. This validates the effectiveness of the proposed method. Thus, the proposed multivariate texture is very applicable to land-cover/land use mapping and other applications using multispectral data.

Acknowledgments This research was supported by NSFC (Grant Number 40372130), and partially supported by Peking University

TABLE 7. CLASSIFICATION USING IKONOS SPECTRAL INFORMATION ALONE AND PLUS MULTIVARIATE TEXTURE (EUCLIDEAN DISTANCE) 5 PIXELS ⫻ 5 PIXELS (ALL IN PERCENT) Bareland

Residential

Bush nursery

81.59 55.15 90.41 85.03 67.27 86.96 OA, 78.22; Kappa, 74.05 Spectral and PA 87.99 60.94 92.53 texture UA 96.14 79.65 92.78 classification OA, 84.68; Kappa, 81.75 OA: Overall Accuracy; PA: Producer’s Accuracy; UA: User’s Accuracy

Spectral classification

PA

UA

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

BY

WINDOW

SIZE

Woodland

Crop field

Rural

Water

Vinyl house

88.59 88.40

89.60 84.92

71.57 60.06

85.63 89.25

72.54 85.64

89.42 87.39

88.00 91.13

89.97 68.93

78.13 96.53

87.72 86.66

February 2009

155

147-157_06-124.qxd

1/13/09

4:11 PM

Page 156

TABLE 8. ACCURACIES FOR CLASSIFICATION ADDING TRADITIONAL VARIOGRAM TEXTURE FROM INDIVIDUAL BANDS OF IKONOS IMAGE USING MULTIPLE WINDOWS Band combinations Spectral Spectral Spectral Spectral

⫹ ⫹ ⫹ ⫹

OA

BV-3,5,7,9 GV-3,5,7,9 RV-3,5,7,9 NRV-3,5,7,9

(%)

89.76 89.79 90.73 91.89

Kappa 0.8782 0.8784 0.8897 0.9035

BV:

variogram texture from Blue band; GV: variogram texture from Green band, RV: variogram texture from Red band; NRV: variogram texture from Near Infrared band. The multiple window sizes from 3 ⫻ 3 to 9 ⫻ 9 pixels and all lag distances less than half of window sizes were used for texture extraction.

Engineering Institute. We would like to thank two anonymous reviewers for their constructive suggestions, which improved the manuscript.

References

Plate 2. portions of (a) Ikonos false color composite image (Band 3, 4, 2 as R, G, B), (b) classification results by spectral information, and (c) classification results by spectral information and multivariate texture. Color assignments: 1, Bare land; 2, Residential; 3, Bush Nursery; 4, Woodland; 5, Crop field; 6, Rural; 7, Water; 8, Vinyl house.

156

February 2009

Arbarca-Hernandez, F., and M. Chica-Olmo, 1999. Evaluation of geostatistical measures of radiometric spatial variability for lithological discrimination in Landsat TM images, Photogrammetric Engineering & Remote Sensing, 65(6):705–711. Atkinson, P.M., and P. Lewis, 2000. Geostatistical classification for remote sensing: An introduction, Computers & Geosciences, 26(4):361–371. Berberoglu, S., C.D. Lloyd, P.M. Atkinson, and P.J. Curran, 2000. The integration of spectral and textural information using neural networks for land-cover mapping in the Mediterranean, Computers & Geosciences, 26(4):385–396. Boser, B., I. Guyon, and V.N. Vapnik, 1992. A training algorithm for optimal margin classifiers, Proceedings of the 5th Annual ACM Workshop of Computational Learning Theory (D. Maussler, editor), 27–29 July, Pittsburgh, Pennsylvania, Morgan Kaufman, San Mateo, California, pp. 144–153. Bourgault, G., and D. Marcotte, 1991. Multivariate variogram and its application to the linear model of coregionalization, Mathematical Geology, 23(7):899–928. Chica-Olmo, M., and F. Arbarca-Hernandez, 2000. Computing geostatistical image texture for remotely sensed data classification, Computers & Geosciences, 26(4):373–383. Coburn, C.A. and A.C.B. Roberts, 2004. A multiscale texture analysis procedure for improved forest stand classification, International Journal of Remote Sensing, 25(20):4287–4308. Cohen, W.B., T.A. Spies, and G.A. Bradshaw, 1990. Semivariograms of digital imagery for analysis of conifer canopy structure, Remote Sensing of Environment, 34:167–178. Cristianini, N., and J. Shawe-Taylor, 2000. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods, Cambridge University Press, Cambridge, 204 p. Curran, P.J., and P.M. Atkinson, 1998. Geostatistics and remote sensing, Progress in Physical Geography, 22(1):61–78. Ferro, C.J.S., and T.A. Warner, 2002. Scale and texture in digital image classification, Photogrammetric Engineering & Remote Sensing, 68(1):51–63. Foody, G.M., and A. Mathur, 2004. A relative evaluation of multiclass image classification by support vector machines, IEEE Transactions on Geoscience and Remote Sensing, 42(6):1335–1343. Franklin, S., and D. Peddle, 1990. Classification of SPOT HRV imagery and texture features, International Journal of Remote Sensing, 11:551–556. Franklin, S.E., M.A. Wulder, and G.R. Gerylo, 2001. Texture analysis of IKONOS panchromatic data for Douglas-fir forest age class separability in British Columbia, International Journal of Remote Sensing, 22:2627–2632 Gong, P., D.J. Marceau, and P.J. Howarth, 1992. A comparison of spatial feature extraction algorithms for land use classification PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

147-157_06-124.qxd

1/13/09

4:11 PM

Page 157

with SPOT HRV data, Remote Sensing of Environment, 40:137–151. Gualtieri, J.A., and R.F. Cromp, 1998. Support vector machines for hyperspectral remote sensing classification, Proceedings of SPIE, the International Society for Optical Engineering, 3584:221–232. Harff, J., and J.C. Davis, 1990. Regionalization in geology by multivariate classification, Mathematical Geology, 22:573–588. Haralick, R.M., B. Shanmugam, and I. Dinstein, 1973. Texture features for image classification, IEEE Transactions on Systems, Man, and Cybernetics, 3(6):610–621. Hsu, C.-W., and C.-J. Lin, 2002. A comparison of methods for multiclass Support Vector Machines, IEEE Transaction on Neural Networks, 13:415–425. Huang, C., L.S. Davis, and J.R.G. Townshend, 2002. An assessment of support vector machines for land cover classification, International Journal of Remote Sensing, 23(4):725–749. Jain, A., and G. Healey, 1998. A multiscale representation including opponent color features for texture recognition, IEEE Transactions on Image Processing, 7(1):124–128. Jensen, J.R., 1982. Detecting residential land-use development at the urban fringe, Photogrammetric Engineering & Remote Sensing, 48(6):629–643. Journel, A.G., and C.J. Huijbergts, 1978. Mining Geostatistics, Academic Press, New York, 600 p. Kerry, R., and M.A. Oliver, 2003. Variograms of ancillary data to aid sampling for soil surveys, Precision Agriculture, 4(3):261–278. Knerr, S., L. Personnaz, and G. Dreyfus, 1990. Single-layer learning revisited: A stepwise procedure for building and training neural network, Neurocomputing: Algorithms, Architectures and Applications (J. Fogelman, editor), NATO ASI, Springer-Verlag, Berlin, pp. 41–50. Lark, R.M., 1996. Geostatistical description of texture on an aerial photograph for discriminating classes of land cover, International Journal of Remote Sensing, 17:2115–2133. Lark, R.M., 1998. Forming spatially coherent regions by classification of multivariate data: An example from the analysis of maps of crop yield, International Journal of Geographical Information Science, 12(1):83–98. Lira, J., and A. Rodriguez, 2006. A divergence operator to quantify texture from multi-spectral satellite images, International Journal of Remote Sensing, 27(13):2683–2702. Mackas, D.L., 1984. Spatial autocorrelation of Plankton community composition in a continental shelf ecosystem, Limnology and Oceanography, 29:451–471. Marceau, D.J., P.J. Howarth, J.M. Dubois, and D.J. Gratton, 1990. Evaluation of The grey-level co-occurrence matrix method for land cover classification using SPOT imagery, IEEE Transactions on Geoscience and Remote Sensing, 28(4):513–519.

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

Mather, P.M., B. Tso, and M. Koch, 1998. An evaluation of Landsat TM spectral data and SAR-derived textural information for lithological discrimination in the Red Sea Hills, Sudan, International Journal of Remote Sensing, 19(4):587–604. Matheron, G., 1971. The theory of regionalized variables and its applications, Les Cahiers du Centre de Morphologie Mathematique de Fontainebleau, Ecole des Mines de Paris, Fascicule 5, Fontainebleau, 211 p. Melgani, F., and L. Bruzzone, 2004. Classification of hyperspectral remote sensing images with support vector machines, IEEE Transactions on Geoscience and Remote Sensing, 42(8):1778–1790. Miranda, F.P., L.E.N. Fonseca, and J.R. Carr, 1998. Semivariogram textural classification of JERS-1 (Fuyo-1) SAR data obtained over a flooded area of the Amazon rainforest, International Journal of Remote Sensing, 19(3):549–556. Oden, N.L., and R.R. Sokal, 1986. Directional autocorrelation: An extension of spatial correlograms to two dimensions, Systematic Zoology, 35:608–617. Pal, M., and P.M. Mather, 2005. Support vector machines for classification in remote sensing, International Journal of Remote Sensing, 26(5):1007–1011. Ramstein, G., and M. Raffy, 1989. Analysis of the structure of radiometric remotely-sensed images, International Journal of Remote Sensing, 10:1049–1073. Ryherd, S., and C.E. Woodcock, 1996. Combining spectral and texture data in the segmentation of remotely sensed images, Photogrammetric Engineering & Remote Sensing, 62(1):181–194. Shafarenko, L., M. Petrou, and J. Kittler, 1997. Automatic watershed segmentation of randomly textured color images, IEEE Transactions on Image Processing, 6:1530–1543. Sokal, R.R., 1986. Spatial data analysis and historical processes, Data Analysis and Informatics, IV: Proceedings of Fourth International Symposium on Data Analysis and Informatics, 1985, Versailles, France, North-Holland, Amsterdam, pp. 29–43. Tseng, D.C., and C.C. Lai, 1999. A genetic algorithm for MRF-based segmentation of multispectral textured images, Pattern Recognition Letters, 20:1499–1510. Vapnik, V.N., 1998. Statistical Learning Theory, John Wiley, New York, 736 p. Vapnik, V.N., 2000. The Nature of Statistical Learning Theory, Springer-Verlag, New York, 314 p. Webster, R., 1973. Automatic soil-boundary location from transect data, Mathematical Geology, 5:27–37.

(Received 04 May 2007; accepted 04 October 2007; revised 07 November 2007)

February 2009

157