Abstract. The paper describes the development environment REx, be- ing a collection of tools used in the construction and development of multilayer statement ...
Statement Networks Development Environment REx Wojciech Cholewa, Tomasz Rogala, Pawel Chrzanowski, and Marcin Amarowicz Department of Fundamentals of Machinery Design, Silesian University of Technology, Konarskiego 18, 44-100 Gliwice, Poland {wojciech.cholewa,tomasz.rogala,pawel.chrzanowski, marcin.amarowicz}@polsl.pl http://www.kpkm.polsl.pl
Abstract. The paper describes the development environment REx, being a collection of tools used in the construction and development of multilayer statement networks applied in diagnostic expert system shell. Described is the process of inference and its implementation based on a multilayer network of statements that make it possible to include the construction of a knowledge base by a group of experts, a hierarchical organization of knowledge, possibility of taking into account knowledge from many sources. Keywords: Multimodal Statement Networks, Knowledge Representation, Knowledge-Based Systems, Probabilistic Reasoning, Uncertain Reasoning.
1
Introduction
Inference is the process of formulating conclusions on the basis of available information. This task, most often done by humans, can be implemented by a computer system. This is done by a number of solutions to carry out the process of inference in an algorithmic manner and / or based on the use of knowledge. Among the latter, particularly important are systems that enable decoupling the process of inference from the considered domain. This requires to develop interference systems that use independent knowledge bases. Inference systems are currently being intensively developed in many fields of science. An example might be their use for the purposes of technical diagnostics, with the primary task identifying the technical state of the object based upon all available information. This task is particularly difficult for objects which are subject to slow-changing processes of wear. The increase in complexity of technical artefacts for both their structure and dynamic processes occurring in them contributed to the excessive number of available data on the object. Since these data are at varying degree covariate to changes of state, defining knowledge about relations between the observed outputs of the object, and the technical state is difficult. One example are turbine P. Jędrzejowicz et al. (Eds.): ICCCI 2011, Part II, LNCS 6923, pp. 30–39, 2011. c Springer-Verlag Berlin Heidelberg 2011
Statement Networks Development Environment REx
31
generators, for which the description of the possible definitions of the symptoms can be very complex and involve a variety of mechanical, magnetic and aerodynamic interactions. To the difficulties of relationships also comes uncertainty of the data which is caused by measurement errors, random factors, and sometimes the approximate nature of the data. Hence the need to develop inference systems allowing the identification of the state under conditions of uncertainty. The difficulty of articulating the knowledge defining the symptoms for complex objects as well as the development of methods of numerical modeling of multidimensional data, contributed to the development of diagnostic methods using models of objects. One example is the so-called model-based diagnosis, which is based on analyzing the differences between the results of observation of the object and the results of the model tuned to this object and allows simulating the operation of the object with the given condition [2]. In the case of diagnostics subject to slow-changing process of wear the most common approach is to use object models as simulators to allow for the acquisition of additional data about the object. Performing active diagnostic experiments, conducting a simulation experiment (using the object model), with desired technical states and observing environmental interaction allows acquisition of data corresponding to the states, and thus understanding the relationship between states and the corresponding symptoms. Data obtained in this way represent the implicit knowledge about the object. This knowledge can still be uncertain. This is due to the fact that highly tuned models of object are also inaccurate models (e.g., do not take into account the unknown and difficult to assess affects of environment on the object). A convenient way to represent implicit knowledge, derived on the basis of a simulation experiment or diagnostic tests are diagnostic models. These models have been developed mostly by machine learning, and show mapping object between model interaction with the environment and the state of the object, and can be applied in model-based inference systems. Effectiveness of the reasoning process depends on the availability and quality of knowledge about the object. This leads to the need to simultaneously take into account the sources of explicit knowledge expressed by experts as well as implicit knowledge [2]. The integration of knowledge for model based inference systems is a difficult task requiring the development of complex models taking into account the knowledge derived from many experts and many data sources. This task can be simplified by considering a number of simple models with lesser degrees of complexity representing the selected sources of knowledge. The main disadvantage of this approach is the need for the synthesis of responses from different types of incompatible individual models, showing conflicting data. Examples are diagnostic models created by experts in expressing different opinions on the considered domain. The issue of recognition of the technical state of complex objects is a difficult task that requires consideration of incomplete, inaccurate and approximate data. Moreover, inferences about the state are usually performed in conditions of imprecise, incomplete and partially contradictory knowledge. This leads to
32
W. Cholewa et al.
the need for the development and application of tools enabling to take into account the nature of the inference process. One of the convenient solutions for conducting inference under uncertainty and the possibility to include knowledge from many sources are multilayer statement networks.
2 2.1
Multilayer Statement Networks Graphical Models
Multilayer statement networks belong to the class of graphical models which use representations of the domain description in the form of graphs. Individual variables are represented by graph vertices, and the dependencies between them by graph edges. Network models are models most often connecting two fields of science, graph theory and the area under which it is possible to apply the inference process. Examples are probabilistic graphical models using probability theory, where all variables are considered as random variables. The task of inference in the network models boils down to searching for an equilibrium network by changing values of specific variables [1]. For example, the search for equilibrium in a probabilistic graphical models lies in the fact that when you change the specified values of random variables, resulting for example from the observation of the object, the answer is obtained on the probability distribution of the variables that describe such state. This action corresponds to the inference carried out using a diagnostic model. The basic advantage of network models is their transparency. Representation of dependencies between variables allows easy understanding of the described domain, and therefore its editions. This means that graphical models can be identified and tuned by hand, as well as automated using a set of machine learning algorithms allowing identification of the structure and / or network parameters based on data obtained during the numerical experiment. These models, as opposed to numerical models of black boxes, do not require re-learning when connecting new examples to the set of learning data. The process of inference in network models as opposed to inference based on numerical models of black boxes is not necessarily unidirectional. It is possible to both place known values for corresponding variable states and the launching of a process requesting the reading of values of corresponding variable symptoms. Thanks to that it is also possible to indicate, among other things, the most likely symptoms, which should appear in the event of a particular technical state. This is a particularly desirable property in expert systems requiring explanations of the inference process being carried out. 2.2
Statements and Statement Networks
Graphical models such as probabilistic graphical models can be used as inference modules of expert systems [3]. Their application requires the development of an appropriate way to communicate with users of these systems. It is necessary
Statement Networks Development Environment REx
33
to pay attention to the possible use of statements occurring for example in the form of logical sentences. To obtain an unambiguous interpretation of statements, taking into account the considered context, may be accompanied by adequate explanation of their interpretation. The use of statements is particularly convenient for systems intended to support technical diagnostics, where there is a clear necessity to interpret the data value (e.g., feature value of the measuring signals) and their changes [1] [4]. The statement is information on the recognition of expression resulting from observed facts or representing an opinion. Usually it is described by a pair [1] s =< c, v >
(1)
where c is the statement content, and v is the statement value. The content can be a declarative sentence, to which is attributed one of the logical values (true, false). Use of statements in expert systems allows the introduction of a complex system of aid, containing an explanation of used terms, links to sources and various comments. It is assumed that in inference systems statement content is constant and statement value can vary. This assumption allows the creation of thesauri, or sets of constant statement content. Thesauri help manage the explanations of statements content and control the degree of detail of explanations. Graphical models in which statements are represented by variables are called statement networks [1]. For the purpose of constructing a convenient statement network, it is possible to consider simple statements containing only one variant of content with its value or complex statements, which are specified as the pair: s =< c, v >=< c1:n , v 1:n >
(2)
where c is n the n-element vector of the statement contents variants, and v is the corresponding n-element vector of values for different variants of content. In the case of statement network it is assumed that variants of complex statements constitute an exhaustive set of mutually excluding elements. An example of complex statement content is the triple vector c1:3 , where [1]: c1 c2
= =
apple is red; apple is green;
c3
=
apple has a different color than red or green.
(3)
Methods of inference in statement networks are dependent upon the accepted definitions of statement values. Approximated statements are introduced in order to enable inference in imprecise, incomplete, and even contradictory information environment. It is assumed that the contents of these statements are accurate and constant; however, their values are approximate. Approximate values of statements can be defined as degrees of truth or degrees of belief in the truth of statements. It is possible to consider values of approximate statements as point values and interval values.
34
2.3
W. Cholewa et al.
Approximate Statement Networks
Approximate statements, which values are expressed as degrees of belief in the truth of statement, are used in statement networks represented as belief networks [4], [5]. Their use requires the definition of conditional probabilities tables for all nodes. The values of these tables are hard to determine. Another variant of the statement network, which allows approximate inference are approximate statement networks in which knowledge is represented as necessary and sufficient conditions [1]. If the belief in the truthfulness of the statement sp is always accompanied by the belief in the truthfulness of the statement sn , but not necessarily inversely, then sp is sufficient condition to establish sn . At the same time sn is necessary condition for sp . Information about the fact that sp as a sufficient condition for sn can be written as [1] b(sp ) ≤ b(sn ) (4) where, b(·) is a statement value. Approximate necessary and sufficient conditions are defined as presupposing that the dependence (4) can be satisfied with small permissible deviation δ. The value of an acceptable degree of approximation can be taken in different ways, for example as: – value δ equal for all considered conditions b(sp ) − δ ≤ b(sn )
δ≥0
(5)
– values δp i δn assigned individually for each statement sp and sn b(sp ) − δp ≤ b(sn ) + δn
δp ≥ 0 δn ≥ 0
(6)
– values δp,n assigned individually to each condition for a pair of statements sp and sn b(sp ) − δp,n ≤ b(sn ) δp,n ≥ 0 (7) The use of deviation δ written as in (5) (6) (7) allows the application of varying degrees of importance of statements. Systems containing inequalities (5) (6) (7) can be considered as statement networks, where statements are represented as nodes and each inequality is represented by the corresponding edge of the network. A statement network solution based on the use of statements of necessary and sufficient conditions when there is no contradiction in the network does not require the use of complex algorithms for searching for network solutions. In the case of a solving network, in which there is a contradiction, to seek solutions for the network could rely on finding the equilibrium minimizing the value of the criterion function written as [1]: 2 2 kp,n δp,n (8) e= p,n
where kp,n is a parameter determining the degree of importance of the condition representing influence of statement sp , on statement sn appearing in the approximate network.
Statement Networks Development Environment REx
2.4
35
Multilayer Statement Networks
It is possible to develop diagnostic models in the form of complex statement networks for complex technical objects. This task is difficult, however, requires consideration of various aspects of the operation of the object, as well as many of its components. A much more convenient way is to decompose the considered domain into subdomains, which can be described by simpler component models. It should be noted that the decomposition does not have to be related to the spatial structure of the object but for example to a selected aspect of its operation. It allows to create component models of a limited number of inputs (symptoms) and outputs (classes of states), which allows easier interpretation, identification and subsequent tuning [2]. A convenient tool for building component models and their subsequent synthesis, are multi-layered statement networks [1]. The structure of these networks is defined in a similar way as multimodal networks, where individual layers are treated as modes of multimodal network. Multimodal statement network is defined as a directed hypergraph, described by ordered triple < V, E, Γ >, where V is the set of all vertices of the hypergraph, E is a set of hypergraph edges and Γ is a set of mods of the hypergraph [7],[6]. The essence of the system is that individual modes of the hypergraph represent component networks for individual layers of the model and the selected nodes of the networks can occur in many layers. An example of a multilayer network is shown in Figure 1.
Fig. 1. An example of multilayer statement network
Synthesis of multilayered network response can be implemented in two ways:: – as a process of aggregation, where individual nodes act as independent instances in subsequent layers, and their values are determined in an independent manner, for example, when all the network layers were developed as statement networks, represented by statements of beliefs, or
36
W. Cholewa et al.
– as the reconciliation process, where each individual node acts as a single copy in all layers. This would require an appropriate adaptation of procedures for solution of each layer of the network. An example might be using approximate statement networks in all layers of a multilayered network. A characteristic feature of multilayered networks is that the individual networks created within the considered layer can be statement networks represented by different types of network models. The possibility of constructing a statement network as a set of component networks opens the possibility of collective creating of individual network layers, independently by several experts. Depending on the used decomposition of the considered domain, such networks can be built as networks of diverse structure, such as multiaspected or multiscaled networks.
3
Development Environment REx
Inference systems based on the use of multi-layer statement networks are a promising tool to enhance the process of identifying the technical state of complex objects. They take into account the characteristics and needs of diagnostic inference, among other things: – imprecise, incomplete and partially contradictory data and knowledge, – the possibility of taking into account knowledge from many sources, – no need to re-identify the model in the case of new examples in a set of training data, – possibility of backward inference, in order to carry out the process explanation for the conclusion reached, – simple and understandable way to communicate with users of the system using statements, the possibility of independent components to build models based on different sources, – greater transparency of knowledge represented by a multilayer statement network, easier interpretation, – identification and tuning of the component models and achieving the better generalization properties for these models, – the use of different decomposition of domains for the subdomains connected with various aspects of an object, or associated with the spatial decomposition of the object. Building expert systems to aid the diagnosing of complex objects requires to use special exchange of information between different modules of such systems. The application of the general concept of a blackboard is here specially reasonable [4]. The blackboard (Bulletin Board component) is a place where messages containing information about values of statements are available to receivers. The statements which appears on the table are active. It means that changes of statement values included in a model A can initiate sequences of operation which cause automatic changes of the values of other statements in models B,C ect. The table contains also information about the modules which are responsible for
Statement Networks Development Environment REx
37
Fig. 2. Components diagram of development environment REx. The current release of REx (http://www.kpkm.polsl.pl in section Projects) includes the components marked in bold frames.
the change of statements values. An interesting remark is that the blackboard can be understood as a hierarchically ordered database, designed for storing solutions generating by autonomous modules. The structure of the environment REx in the form of components diagram is presented in figure 2. It is an experimental environment in which it is possible to conduct appropriate comparative studies aimed at determining the most effective, from the viewpoint of the effectiveness of the process, inference tools. These include among others the research of: – aggregation processes in the application of many different types of networks which constitute the multilayered network. Obtained in this way results should help identify appropriate methods for determining the response of the inference system, – algorithms for solving the statement networks, including in particular the approximate statement networks, – algorithms for identifying the structure and network parameters on the basis of training data, – other aspects of the testing of interoperability of network layers, depending on the structure of multilayer networks, for example, multilayer networks represented as a multiscale network. The proper conduct of comparative research requires the development of a numerous sets of training and testing data and sets of examples of different networks, that can be used repeatedly during the mentioned studies and evaluating the performance of the inference system. For this purpose environment REx includes a repository of data and models (Benchmark and Model Repository) with differential network structures and differential training and testing data. Building complex models represented by component models that are created based on various sources of explicit and implicit knowledge required to ensure
38
W. Cholewa et al.
that the content of statements entered into the system is unequivocally interpreted by all individuals developing particular component models. The need to ensure the proper integrity of the developed set of statements is associated with the respective statement descriptions and avoiding the introduction of repeated, or semantically similar, contents of statements. The REx system includes the possibility of building thesauri for this purpose, creating appropriately classified dictionaries of accessible contents of statements, which can be built by many people working on a dictionary for the considered domain. Since statement collections can be numerous and the possibility of verification of the thesaurus in terms of formal and semantic correctness may be time consuming, appropriate methods of managing the various versions of the thesaurus are considered. The overall process of building a multilayer network involves the following steps: – introduce a set of possible statements for the considered domain through the creation of a new thesaurus. Defining the thesaurus involves building simple statements by indicating the type of values of the statement and the introduction of its content and the necessary guidance on the interpretation of the statement. In the next step, based on the set of simple statements it is possible to create complex statements in a similar manner. – define a new model of multilayer network and determine the number of layers, – define the individual layers of the network by developing component networks. This task, for each layer, requires the development of the structure of the graph, indicating the selected vertices representing particular statements and the edges between the vertices of the graph. The next step defines parameters for the network model if required for the considered type of network model used in the particular layer. An example might be the necessity to define a table of conditional probabilities for the layer described by the statement network represented as a belief network, – select merging strategy for layers (i.e. aggregate or reconciliate layers), – start-up of calculations. The environment REx was developed in R language. The use of this language gives, among other things, the ability to develop the environment and its components using free software.
4
Summary
Contemporary development of artificial intelligence allows the conduct of the process of inference with inaccurate, incomplete and even contradictory information. In the case of inference systems based on knowledge, which can also be inaccurate, incomplete and partially contradictory, inference is difficult. One example is to conduct inference on the technical state for a complex object. The performance of an inference system shell in this case depends on whether the requesting system allows both the consideration of the nature of the inference process, as well as, the quality and possible use for this purpose of various available sources of knowledge.
Statement Networks Development Environment REx
39
The paper describes a general concept of the development environment REx which is a set of tools used for the construction and development of multilayer networks used for diagnostic expert system shells. Described is the process of approximate inference realized by multilayered statement networks, which among others, gives opportunity to take into account knowledge from multiple sources. The paper describes the basic properties of statement networks and the ability to use approximate statement networks as component to build multi-layer statement networks. We report the construction of an experimental environment enabling REx to carry out comparative studies on both the testing of different algorithms and evaluation of the efficiency of the processes of inference. The developed version of the environment REx is available for download in the section Projects on the website of Department of Fundamentals of Machinery Design, at http://www.kpkm.polsl.pl Acknowledgements. Described herein are selected results of study, supported partly from scientific funds, as statutory research and research projects 4T07B05028 entitled„ Reverse models and diagnostic modeling”, N504478434 entitled„ Construction of diagnostic advisory systems using multiscaled statement networks” and from the budget of Research Task No. 4 entitled„ Development of integrated technologies to manufacture fuels and energy from biomass, agricultural waste and others” implemented under The National Centre for Research and Development (NCBiR) in Poland and ENERGA SA strategic program of scientific research and development entitled „Advanced technologies of generating energy.”
References 1. Cholewa, W.: Multimodal statements networks for diagnostic applications. In: Sas, P., Bergen, B. (eds.) Proceedings of the International Conference on Noise and Virbation Engineering ISMA 2010, September 20-22, pp. 817–830. Katholique Universiteit Lueven, Lueven (2010) 2. Cholewa, W., Korbicz, J., Kościelny, J., Patan, K., Rogala, T., Syfert, M., Witczak, M.: Diagnostic methods. In: Korbicz, J., Kościelny, J.M. (eds.) Modeling, Diagnostics and Process Control-Implementation in DiaSter System, pp. 153–231. Springer, Heidelberg (2010) 3. Dawid, F., Cowell, R., Lauritzen, S., Spiegelhalter, D.: Probabilistic Networks and Expert Systems. Springer, New York (2007) 4. Korbicz, J., Kościelny, J.M., Kowalczuk, Z., Cholewa, W. (eds.): Fault diagnosis: Models, Artificial Intelligence, Applications. Springer, Heidelberg (2004) 5. Jensen, V.J.: Bayesian networks and decision graph. Springer, New York (2002) 6. Heath, L., Sioson, A.A.: Multimodal networks: Structure and operations. IEEE//ACM Trans. on Computational Biology and Bioinformatics 6(2), 271–280 (2009) 7. Sioson, A.: Multimodal networks in biology. Phd Thesis, Virginia Polytechnic Institute and State University, Blacksburg, Virginia (2005)
