Accuracy Assessment and Validation of a Land Use-Land Cover Database in Arezzo (Italy)∗. Pierpaolo Napolitano ISTAT Via Adolfo Ravà, 150 00142 Roma ITALY Tel. +39 06 5952.4436 Fax. +39 06 5412.725 e-mail:
[email protected] Giancarlo Carbonetti ISTAT Via Adolfo Ravà, 150 00142 Roma ITALY Tel. +39 06 5952.4436 Fax. +39 06 5412.725 e-mail:
[email protected] Javier Gallego JRC , I-21020 Ispra (Varese) ITALY Tel. +39 0332 785101 Fax. +39 0332-789936 e-mail:
[email protected] Keywords: land cover, land use, accuracy assessment, area estimation, calibration.
Abstract ISTAT promoted a pilot project for the realization of a land use / land cover database at 1:25.000 scale over a test area in the Province of Arezzo, 2000 km2 wide. The aim of the project is to set the basis for a possible development of a national land use database – land cover GIS. The nomenclature was derived from Corine Land Cover. In this paper methodology and criteria for the accuracy assessment are described and the project of a field validation is presented.
1. Introduction A land use-land cover database, with the geometric accuracy of 1:25.000 scale cartography, may respond to several information needs, required on both national and regional administrative level, for sustainable resource management. Unfortunately a considerable lack of information on the state of the territory and landscape is observed in Italy. ISTAT, the National Statistical Institute of Italy, has developed a database on an area of 200,000 hectares in the Arezzo province, in central Italy, to test the possible setting up of a database at country level. The pilot project, partially funded by the European Union (EU). The territory is divided into polygons classified on land use-land cover classes. The nomenclature has five levels; the three first levels coincide with the CORINE land-cover nomenclature (EC, 1993); a fourth level is added for urban areas and a forth and a fifth level are added for forests and semi-natural areas. We call this layer the thematic component. The project utilised digital black and white ortho-images from aerial photographs; they have a resolution of one meter on the ground, and a spectral resolution of 256 grey shades. They provide the same geometric accuracy as the cartography at 1:10.000 scale. The project also used three Landsat-TM quarter-scenes, taken at the following dates: 10/05/97, 26/10/97, 06/05/98. The geographic reference system is UTM ED50, according with the ISTAT territorial databases. The photo-interpretation and the drawing of polygons were carried out directly on screen. The minimum mapping unit was 1,56 hectares (1 hectare for urbanised areas). The polygon layer was integrated with a vector layer composed by linear and point features, referring to railways, highways, major roads and rivers and topographic names. Linear and ∗
Pierpaolo Napolitano is the author of the paragraphs n 1 and n 2; Giancarlo Carbonetti is the author of the paragraph n. 3; Javier Gallego
point features were digitized on screen taking into account the digital ortho-images geometry, after identification on 1:250.000 scale maps. We call this layer the topographic component.
2. Accuracy assessment criteria ISTAT tested the correct application of the procedures. Preliminary check-ups were executed on ortho-images and the geo-referencing procedure of Landsat images. Main control was carried out according to statistical tests. Geometric and thematic quality tests were carried out for each feature: polygons, arcs, and points. A completeness test was executed just for the linear and point features. We face two different types of risk: a) the producer's risk of refusal of a good product and b) the consumer’s risk to accept a low quality product. Two types of checks have been made on the thematic component. The population (over 13,000 polygons) was stratified into 35 strata from the third level nomenclature to extract a sample of 373 polygons, that were reviewed by a different photo-interpreter. Additionally a sample of almost 1800 points, randomly selected by strata, was reinterpreted. The number of sample units strata was proportional to the size of the strata. The statistical tests were defined according to the following parameters: -Null hypothesis: The disagreement between photo-interpretation and reviewing is 12.5%:; -5% probability of type I error (rejecting an acceptable product). -10% probability of type II error (accepting a bad product). For these parameters the sample size is 120 and the refusal limit is 10 (Montgomery 1999). The topographic component was object of a similar test of acceptance.
3. Results of the quality controls The elements of the topographic component were sampled from strata according to their type (roads, highway, railways, rivers, etc.). The results of quality controls are summarised in tables 1 (topographic component) and 2 (thematic component). 1 arc out of 120, and 2 text points out of 120 were out of the tolerance, which required: 1) the inclusion of the element in a 25 m buffer around the reinterpreted element, and 2) the same class of legend between the original element and the reinterpreted one. Two different controls were made on the thematic component (table 2): reinterpretation of a sample of polygons, the second on the reinterpretation of a sample of points. The study area was subdivided in three lots for the check on polygons. The elements of the thematic component were sampled from strata identified according to the second level of the legend. The number of the refused polygons is less then the limit fixed by the test. For the check by points, an area equal to the minimum mapping unit (1.56 ha and 1 ha for urban) was built around the point. For all second level categories of the legend with at least 120 sampling units, the test accepted the null hypothesis. In table 2 B) we can also see that the overall number of the refused points is less than 2% of all the checked points. The table 3 C) provides the error matrix calculated comparing the result obtained by the photo-interpretation (rows) and that obtained in the reinterpretation (columns). We can notice that the worst result, anyhow below the refusal limit of test acceptance, has been obtained for the class 2.4 (mixed agricultural areas). The same analysis has been done in reference to a minimum mapping unit of size equal to 0.5 hectares. In that case no points have been classified in the class 2.4 . Arcs Text-points
Checked elements 120 120
Correct 119 118
% correct 99,17 98,33
Wrong 1 2
% wrong 0,83 1,67
Table 1. Topographic component quality control (arcs and text-points)
Lot 1 Lot 2 Lot 3 Total
Number of Number of checked polygons correct polygons 120 115 133 125 120 113 373 353
Percentage of correct polygons 95,83 93,98 94,17 94,64
Number of wrong polygons 5 8 7 20
Percentage of wrong polygons 4,17 6,02 5,83 5,36
Table 2. Thematic component quality control: Polygons Number of checked points 600 599 597 1796
Lot 1 Lot 2 Lot 3 Total
Number of correct points 592 588 581 1761
Percentage of correct points 98,67 98,16 97,32 98,05
Number of wrong points 8 11 16 35
Percentage of wrong points 1,33 1,84 2,68 1,95
Table 3. Thematic component quality control: Points
11 12 13 14 21 22 23 24 31 32 33 41 42 51 52 Tot
11
12
13
14
21
22
23
24
31
32
33
41
42
51
52
Tot
51 0 0 0 0 0 0 0 1 0 0 0 0 0 0 52
1 30 0 0 0 0 0 0 0 0 0 0 0 0 0 31
0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 32
1 0 0 17 0 0 0 0 0 0 0 0 0 0 0 18
1 0 0 0 308 0 0 0 1 0 0 0 0 0 0 310
1 0 0 0 1 131 1 0 2 0 0 0 0 0 0 136
0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 24
2 0 0 0 4 1 0 116 0 2 1 0 0 0 0 126
0 0 0 0 1 1 0 0 898 2 0 0 0 0 0 902
0 0 0 0 4 1 0 0 2 130 0 0 0 0 0 137
0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 10
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 1 0 0 0 0 17 0 18
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
57 30 32 17 318 134 25 116 905 134 11 0 0 17 0 1796
Table 4. Error matrix of the legend II level
4. The project of field validation Measuring the accuracy of a land cover map requires a better observation of reality on a sample of points or areal units. Checking a photo-interpretation with another photointerpretation on the same data provides the required more reliable data for some land cover types, for which errors are due to insufficiently experienced staff or to a less careful work when a large area has to be photo-interpreted. In other cases there is a real limitation of the possibility to identify land cover types and the comparison gives a measure of subjectivity. In these cases reference data for accuracy assessment should come from a ground survey. A ground validation is being studied now, with these objectives: 1) to obtain a set of reference data suitable for map assessment; 2) to measure thematic accuracy with a standard error no greater than 7%; and 3) to test calibration methods for area estimates. A double phase sampling design has been proposed: in the first phase 8000 points are selected, randomly or with a systematic 500 m grid, and photo-interpreted. This first phase sample will be stratified into 3 or 4 strata taking into account the CORINE Land Cover class (second level), the distance from the polygon border and point interpretation difficulty. A
subsample of 800 points will be selected for field visit with higher rates for points for which errors are more likely: close to polygon borders and labelled as difficult by the photointerpreter. A minimum sample size for each CORINE Land Cover class should be guaranteed. The resulting confusion matrix will allow to compute thematic accuracy, with standard errors, and calibration estimators for land cover classes.
5. Land cover mapping and area estimation. Land cover maps are sometimes used for area estimation of a land cover class by simply adding the area of the polygons labelled as belonging to that class. This approach is rather naïf and can lead to a serious bias if the mapping scale is not sufficient or the thematic accuracy is not very high (Gallego et al, 2000). We can compare the total area obtained by this method from CORINE Land Cover and the ISTAT mapping exercise (table 5). ISTAT CORINE Artificial 84.9 57.1 Arable land 375.9 234.8 Permanent crops 141.0 99.7 Pastures 21.8 66.9 Heterogeneous agriculture 110.8 328.2 Forest 1117.6 1029.5 Natural vegetation 132.4 167.2 Open spaces and wetland 1.7 5.5 Water 5.3 2.3
Table 5: Total area per land cover class from CORINE and ISTAT land cover maps Part of the disagreement can be explained by the scale effect. Compared with CORINE Land Cover, the area labelled as “heterogeneous agriculture” is drastically reduced and probably attributed to “pure” classes. This would explain, at least partly, the increase of arable land, permanent crops and forest. Another part of the disagreement may be due to a different interpretation of the nomenclature or to photo-interpretation errors, but it is difficult to know which is the impact of each source of disagreement until a ground survey is conducted. References - European Commission, 1993, - Corine Land Cover-Technical guide – Luxembourg - Gallego, F.J. Carfagna, E., Peedell S., 2000,The use of CORINE Land Cover to improve area frame survey estimates. Research in Official Statistics, (in press), - Iacobini A., 1991, - Il controllo statistico della qualità - EUROMA, La Goliardica, Roma - ISTAT, 1998, - Capitolato tecnico per il progetto pilota del database sull'uso e la copertura del suolo in scala 1:25.000 su un’area test - A cura della Commissione ISTAT - Montgomery D. C.,1997, Introduction to statistical quality control III edition, Wiley, NewYork
