workshop report on spatial data usability - CiteSeerX

Proceedings AGILE 2002 Conference on GI Science, April, Mallorca, Spain. 2002

WORKSHOP REPORT ON SPATIAL DATA USABILITY 1

2

1

Monica Wachowicz , Catharina Riedemann , Wies Vullings , 3 1 Juan Suárez , and Joep Cromvoets 1

Wageningen UR, Centre for Geo-Information Wageningen, The Netherlands

2

University of Muenster, Institute for Geoinformatics Muenster, Germany

3

Forest Research, Northern Research Station Roslin, United Kingdom

Abstract In this paper we describe the Workshop on Spatial Data Usability held at the Centre for Geo-Information, Wageningen UR. The purpose of the workshop was to open a dialogue within the GI research community concerning the research issues on spatial data usability, and to develop a research agenda on how to identify, measure and evaluate different characteristics of data usability with the goal of improving it. The objective of this paper is to promote better understanding of spatial data usability and develop recommendations on how to incorporate it into GI research.

1.

Introduction

Almost 150 years ago a London doctor combined maps of cholera deaths and water pumps to discover the source of a deadly epidemic, and the case has since become an acclaimed use of spatial analysis taught to generations of geography students worldwide. Moving forward to the present day, data mining techniques are now radically changing the way supermarkets think about product placement within their stores, and telephone customers are moving away from their traditional “YellowPages” directories and turning instead to enhanced “YellowMap” products. While these are all very positive examples, on the other hand a recent UK government hearing into the establishment of an underground radioactive waste repository determined not to proceed with this major project after the results of groundwater hydrology modelling were rejected because they could not be validated. Each one of these cases tells us different stories about the degree of success of each data set when addressing information needs. The ability of the data to accommodate user-defined expectations translates their degree of usability in terms of ‘usefulness’. Our interest here lies in understanding exactly what distinguishes these cases from others. Is it the correct choice of data, models and algorithms for a given application? Is it simply a matter of data quality, is it the ‘interestingness’ or ‘unexpectedness’ of the data (as knowledge discoverers would say)? Is it the integration of data and adding of value that produces these extreme examples? Clearly, a better understanding of the parameters that define the usability of individual data sets seems to be necessary in order to improve their usability.


This paper summarises the findings of a recent Workshop on Spatial Data Usability organised by the Centre for Geo-Information, Wageningen UR. The purpose of the workshop was to open a dialogue within the GI research community concerning these initial questions on spatial data usability, and to develop a research agenda on how to identify, measure and evaluate different characteristics of data usability. The objective of this paper is to promote better understanding of spatial data usability and develop recommendations on how to enhance its value for GI research.

2.

‘Brainstorming Workshop’ in Wageningen

A group of scientists affiliated to AGILE decided to develop a multidisciplinary view of the initial questions on spatial data usability and formulate research issues. The organising committee consisted of the following members: Gary Hunter (University of Melbourne), Arnold Bregt (Wageningen UR), and Monica Wachowicz (Wageningen UR). The committee decided that the first workshop should have small, focussed sessions to facilitate the interchange of ideas. The workshop was held at the Centre for GeoInformation, Wageningen UR, on the 19th and the 20th of November 2001. A web page describing the workshop can be found at http://cgi.girs.wageningen-ur.nl/cgi/news/workshop/workshop-page.htm. The general objectives of the workshop were to: bring together scientists to examine the elements of spatial data usability for scientific applications; share experiences on what makes some spatial data sets more useful than others; and to help determine the direction of future research topics in spatial data usability. The workshop program was organised around a mixture of presentations and discussion groups. Presentations provided examples showing the different interpretations given to spatial data usability as well as research issues involved in the definition and analysis of spatial data usability. The discussion groups addressed the following questions: 1. What do we mean by usability? 2. Why is usability important? 3. What are the characteristics of spatial data usability? 4. What are the research problems to be solved in spatial data usability? 5. What should the research priorities be? Participation in the workshop consisted of submission of an abstract in order to start informal interaction and discussion among representatives of the GI community. The presentations covered different aspects involved in data usability concerning data producers, end-users, statisticians, computer scientists, geographers, modellers, etc. These presentations served as catalysts for further discussion held within smaller groups showing similar points of view. The list of participants can be found on the web site as well as the responses of each group to the five discussion questions. In the following sections of this paper, we give an overview of the workshop by describing the responses of each group to the five discussion questions and their key findings for spatial data usability research.


3.

What do we mean by usability?

A number of possible definitions of usability were discussed during the workshop, and the needs of spatial data usability are compared and contrasted with broader data related activities of providers and users of geoinformation. For example, one official definition of usability is given by the ISO 9241-11 standard on Display Screen (VDU) Regulations, Use of Ergonomics for Procurement and Design (http://www.chi-sa.org.za/usability.htm). In this definition, system usability comprises " the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use, where: -

Effectiveness measures the accuracy and completeness with which users achieve specified goals;

-

Efficiency measures the resources expended in relation to the accuracy and completeness with which users achieve goals;

-

Satisfaction measures the freedom from discomfort, and positive attitudes towards the use of the product."

Usability elements outline the features and characteristics of the product that influence the learnability, effectiveness, efficiency and satisfaction with which users can achieve specified goals in a particular environment. The context of use determines the types of users, tasks, equipment, and the physical and social environments in which a product is used. Therefore, a system consists of users (i.e. the people who interact with the products), equipment (hardware, software and materials), tasks (activities required to achieve a goal) and a physical and social environment, for the purpose of achieving particular goals. Usability has also been defined as "a set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by a stated or implied set of users" (ISO 9126 - http://www.issco.unige.ch/ewg95/node14.html). This definition differs from the definition from an ergonomic point of view, in which other characteristics such as efficiency and effectiveness are also seen as elements of usability. Nielsen, concerned with software usability, sketches a broader context he calls system acceptability. One aspect of it is usefulness, “the issue of whether the system can be used to achieve a desired goal” [Nielsen 1993, p. 24]. Following Grudin [Grudin 1992], he breaks it down into utility and usability, utility denoting “the question of whether the functionality of the system in principle can do what is needed”, and usability denoting “the question of how well users can use this functionality” [Nielsen 1993, p. 25]. These definitions distinguish contents and “packing” in such a way that quality would belong to utility meanwhile presentation would belong to usability. It was difficult to arrive at a consensus amongst the participants for a valid definition of data usability. In particular, it was hard to isolate a core set of fundamental techniques that clearly distinguish data usability from any single component discipline: in some way it was a uniquely powerful combination of individual techniques that characterises the field. In summary, data usability was identified as an umbrella term consisting of several elements aggregated into five main groups (Figure 1):


1. Marketing: Added Value, Benefits, Costs, Novelty, Services Provided, and Satisfaction 2. Quality: Authoritative, Guarantee Against Error, Integrity, Metadata, Reliability, Validity, and Utility 3. Software and Tools: Human Computer Interaction, Standardisation, Integration, Searchable, and Interface 4. Human Perception - Cognition: Authoritative, Decision Type, Interestingness, Novelty, Popularity, Satisfaction, Trust, User Skill Levels, Familiarity, Interpretation, Visualisation 5. Applications: Aggregation Levels, Type, Exclusiveness, Visualisation, Integration, Decision Type, Use with Models and Algorithms, Availability and Accessibility.

Software and tools

Quality

Human Perception - Cognition

Marketing

Applications

Figure 1. The cluster of usability elements into categories. Each individual category can be discussed in the light of specific user groups and/or applications. For example, in the marketing category, cost is a very common and often a very important element and plays a large part in usability. In the applications group, availability and accessibility also determine the use of data. Moreover, it is important to realise that elements such as accessibility and cost of spatial data can vary between different countries and cultures. Another example is the update frequency. In order to decide on the desired update frequency, users should be asked to define the tolerable difference between decisions made and the time frame for future decisions. During the workshop, some of the elements were allocated to more than one category according to alternative interpretations of each concept. For instance, ‘Visualisation’ covered aspects that could relate to ‘Software and Tools’ or ‘Human Perception and Cognition’. These overlaps s between groups made the teams think that the transition from one category to another was not clearly defined. Another conclusion was that the category with more number of overlaps was the ‘Human Perception and Cognition’. This pointed out a core interest in the way users estimate the fitness for purpose of a particular data set and the decisions to be made. However, this is another elusive concept since perceptions seem to be biased by users preconceptions. A key concept that could easily fit inside any of the five groups is ‘Utility’. This concept is the fundamental parameter that defines the transition


between groups. Membership of one or more groups related to the scope of the application. The ability of a data set to address the explicit issues in each group could be measured in terms of its utility for each group. Accordingly, a data set may be useful or not useful for any one of the five aspects that define usability with a transition in between. Membership of a group defines the usability of a data set for a range of applications and inversely its exclusion from others.

4.

Why is usability important?

A straightforward reason is because data are produced and used by people. Consequently, users play an important role in determining data usability. Spatial data can support better informed and faster decision making, but this potential is not always exploited adequately. Likewise, spatial data might not be usable enough to enable the quality and speed of decision, which are actually possible. Extensive preprocessing of spatial data, for example, slows down usage. Bad usability can prevent information from being used at all. In Germany, utility companies have built their own databases although public authorities collect and maintain cadastral data and would like to sell them. Data sets that are produced for one specific purpose usually serve this purpose well. The requirements are clear and the product can be adjusted to them. But as soon as such a tailored data set is used in other contexts, difficulties arise. Although in principle new applications might need the same kind of data, their specific tasks and differing hardware and software environments will demand adaptation (e.g. spatial extent, feature classes, data models, spatial reference system, and data format). Primary data, like topographic data and aerial photos, are originally designed to serve a variety of applications. They are mostly intended to serve as base data integrated with customer data. Authorities producing such data must realise that, like private enterprises, they must provide adapted products if they want to reach potential customers. This means the ability of the data sets to overlap all the five aspects used to define usability. Therefore, usability seems to be an important factor for multiple uses of data and consequently has an impact on profitability. This thought finally leads to the fact that spatial information is a good and that there is (or should be) a market where this good is traded. In the domains of hardware and software, usability has proven to be a key selling argument, at the latest when functionality of products started resembling each other. It is sensible to think that this experience can be transferred to the domain of data. Therefore, good usability should provide competitive advantages and provide returns to all who are involved in the value chain providing usable information. Bad usability, however, can prevent information use. Why produce data that are not used? This simple question calls for usability consideration. During the workshop, we agreed that if data were considered from a usability point of view, both the collection and the usage of data would be affected.

5.

What are the characteristics of spatial data usability?

If you want to measure the usability of a given product, the easiest way is to assume that usability is characterised by the user’s gain from using a data set to solve a specific problem. The effects of using the product are measured: obtaining information, making profit, saving time, and being satisfied. These are indirect


measures or characteristics, which are reliable and easy to record. They tell you how usable a product is, but they do not reveal why. If you look at usability from an engineering perspective, i. e. you want to design something usable, indirect measures help to validate the concept, but they are not sufficient. You must know about the intrinsic characteristics of the product and you must understand the rules determining the combination of characteristics and circumstances, which lead to a usable product. This knowledge provides design guidelines and enables predictions concerning product usability. The umbrella term “usability” comprises a collection of such characteristics. We constructed a list of elements, some of them are interrelated [See list of elements in Section 3]. They need grouping (clustering, generalisation, aggregation) and prioritisation in order to be manageable. The latter cannot be done without knowing the application, the type of decision process, and the user to be supported. In one application, price might be an important selection criterion (e.g. direct marketing), whereas in another costs are largely irrelevant (because the decision has an extremely high value) and data quality determines usability (e. g. deposit exploration). Computer specialists may find it acceptable to do some pre-processing, whereas a holidaymaker looking for a hotel could not. All characteristics we have listed relate to data usability in general. This raises the question whether there is anything special about the usability of spatial data. With some characteristics it is obvious that the spatial dimension requires extra consideration, e. g. spatial accuracy or integration of different geometric types. Moreover, we expect the cognition of spatial representation to be necessary with many or all usability aspects. But it is unclear how this affects methods for usability engineering. Differences between expert and non-expert users in relation to usability were also discussed. The use of GIS is no longer complied to experts, but non-experts often lack the know-how to deal with spatial data. It seems necessary to create a framework and/or a task map for non-expert users. There may be more to gain in making data more usable for non-experts users than for expert users.

6.

What are the research problems to be solved in spatial data usability?

The development of spatial data sets is commonly regarded as the most expensive part in any GI project. At the present there is no universal method designed to quantify the degree of usability of the actual data sets. As we have seen in the previous sections, there are different elements considered to be synonymous with usability that are strongly linked to the original user’s specifications, the purpose of the project and the nature of the problem. Perhaps a more holistic approach is needed to improve the chances of the same data sets being used by a wider audience. In particular, a methodology is needed for mapping the usability of any database in terms of the five aforementioned groups of elements: marketing, quality, software and tools, human perception/cognition, and applications. It will be important not only to quantify how well a data set fulfils its original purpose, but also its ability to incorporate wider objectives and to wide its usability.


There is also a dynamic component in the definition of usability. As time goes by, some data sets will be more useful than others. This is a consequence of the nature of the elements being depicted and the temporal and spatial scale of the representations. Accordingly, considerations will have to be made about the best way of improving the use of the database. Again, a usability framework should provide milestones for the application of objective methods aiming to extend the lifespan of a database within reasonable limits. This methodology will affect the way data sets are produced, maintained and used. There is also considerable scope for the implementation of data modelling techniques like rule-based classifications, data mining, etc. Finally, there is an important element linked to the way users perceive and use existing data sets. The same data presented in an unfamiliar way to the user seem to fail their usability. The human interface is an important area for constant improvement. Some visualisation like 3-D representations, virtual reality and animations seem to bridge the gap between data producers and data consumers. Therefore, these techniques improve the likelihood of these data being fully understood and used.

7.

What should the research priorities be?

Research priorities should seek to define a scientific methodology to quantify data usability that can be measured and compared with each other. The methods already developed in the field of the Information Technology are based on performance testing, beta sites, expert reviews, cognitive walk-troughs, heuristic evaluation, satisfaction questionnaires, and user interviews. However, there are commercial restrictions on the application of usability measures and the methods developed for evaluating them. Related research topics to spatial data usability have also been discussed in the NCGIA (National centre for Geographic Information and Analysis) meeting on Use and Value of Geographic Information. They were mainly related to impediments to GIS diffusion and adoption, cost of information, assessing the value of information, willingness to pay for information, and the effects of improved information in decision making. The main focus was on the evaluation of the benefits of geographic information analysis using Geographic Information Systems. Although we can make use of some of the techniques already developed, there is a need to study the mismatch between user’s expectations and the usability of our data sets. This distance tends to be variable in time. Therefore, we need an objective methodology to define the usability of our data according to the five groups of elements in our continuous effort of improving usability (See Section 3 for the list of elements). A coherent mapping of the state of our data sets will show us clearly the current and potential directions for improving them and identifying their milestones. We have identified seven research issues: 1. What can we learn about usability from the engineering community? 2. How can we measure and evaluate data usability? 3. Can we develop formal rules for ensuring data usability?


4. How the elements of usability change over time? 5. Is there a recipe or cookbook for usability linked to user tasks? 6. Are there case studies that show good and bad practice of usability data? 7. How can we define the differences between spatial and non-spatial data usability? The AGILE working group on Spatial Data Usability is being created to look at the above research issues and contribute to the GI European research agenda. For further information contact Monica Wachowicz, Centre for Geo-Information, Wageningen UR, [email protected].

8. Future developments The Working Group considered three possible pilot cases. The first one is the use of visualisation techniques to improve the access to census data in Israel. Three dimensional and web-based tools may be an intuitive interface to allow a better understanding of census data by non-expert users. In Great Britain, Forest Research will carry out a second case study. The organisation has a considerate amount of data collected to support research on the modelling of the forest growth and development. Now, under the ‘Freedom of Information Act’, these data sets have to be made accessible to the general public. This example will help us to define a methodology to design a system to make scientific information accessible to a wider public, to monitor their use of the data, and so provide feedback to improve the system. The Centre for Geo-Information, Wageningen UR will conduct the third case study. A review of the literature on usability will be carried out and its relation with GI will be investigated. An inventory of "good" and "bad" cases of spatial data usability will also be investigated.

8.

Acknowledgements

The authors would like to thank all participants for all of their support for this workshop. We would also like to thank AGILE for the opportunity to discuss spatial data usability issues in a broader venue.

References Grudin, J. 1992. "Utility and Usability: Research Issues and Development Contexts". Interacting with Computers. 4(2). 209-217. ISO. 1998. Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs) - Part 11: Guidance on Usability. International Standard ISO 924111:1998(E). 15.03.1998. Geneva. International Organization for Standardization (ISO), Technical Committee (TC) 159. Nielsen, J. 1993. "Usability Engineering". Boston. AP Professional.