Intelligent machine agent architecture for adaptive ... - Semantic Scholar

6 downloads 34974 Views 2MB Size Report
Sep 3, 2011 - a Mechanical Engineering, University of Michigan, Ann Arbor, MI, USA b Automotive Components Technology Station, Nelson Mandela Metropolitan ... c Department of Mechatronics, School of Engineering, Nelson Mandela ...
Advanced Engineering Informatics 25 (2011) 783–796

Contents lists available at SciVerse ScienceDirect

Advanced Engineering Informatics journal homepage: www.elsevier.com/locate/aei

Intelligent machine agent architecture for adaptive control optimization of manufacturing processes Grant H. Kruger a,⇑, Albert J. Shih a, Danie G. Hattingh b, Theo I. van Niekerk c a

Mechanical Engineering, University of Michigan, Ann Arbor, MI, USA Automotive Components Technology Station, Nelson Mandela Metropolitan University, Port Elizabeth, Eastern Cape, South Africa c Department of Mechatronics, School of Engineering, Nelson Mandela Metropolitan University, Port Elizabeth, Eastern Cape, South Africa b

a r t i c l e

i n f o

Article history: Received 29 July 2008 Received in revised form 7 July 2011 Accepted 7 August 2011 Available online 3 September 2011 Keywords: Intelligent machining Agent architecture Intelligent manufacturing Process optimization

a b s t r a c t Intelligent agents have been earmarked as the key enabling technology to provide the flexibility required by modern, competitive, customer-orientated manufacturing environments. Rational agent behavior is of paramount importance when interacting with these environments to ensure significant losses are not incurred. To achieve rationality, intelligent agents must constantly balance technical (process) and economic (enterprise wide) trade-offs through co-operation, learning and autonomy. The research presented in this manuscript integrates methodological commonalities in intelligent manufacturing research and prognostics to design and evaluate a generic architecture for the core services of self-learning, rational, machining process regulation agents. The proposed architecture incorporates learning, flexibility and rational decision making through the integration of heterogeneous intelligent algorithms (i.e. neural networks and genetic algorithms) from fields such as machine learning, data mining and statistics. The architecture’s ability to perceive, learn and optimize is evaluated on a high-volume industrial gun drilling process. Ó 2011 Elsevier Ltd. All rights reserved.

1. Introduction Artificial Intelligence (AI) research has demonstrated its ability over the last two decades to meet many of the challenges presented by next generation manufacturing for achieving global competitiveness [1]. Heterogeneous, distributed intelligent control is leading to more flexible, responsive and intelligent machine tool systems. These systems are able to self-integrate into the overall manufacturing operation and address the impacts of process variation on the economics of manufacturing operations to realize global optimization [2]. Software Agent and Holonic Manufacturing systems have been said to provide one of the most promising computing methodologies for the development of these distributed, open and intelligent software systems [3]. The integration of these technologies in industrial control systems will allow increased robustness, scalability, and enhanced adaptability through their dynamic capability. This will allow higher levels of disturbance related uncertainty to be addressed [4]. Researchers have already attempted innovative strategies to integrate agent technology with manufacturing enterprises for enterprise collaboration, manufacturing process planning, scheduling and shop floor control, materials handling and inventory management. Additionally, interesting

⇑ Corresponding author. E-mail address: [email protected] (G.H. Kruger). 1474-0346/$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.aei.2011.08.003

possibilities and new kinds of Holonic Manufacturing Systems have also been investigated. The authors suggest Reference [5] for a detailed review of these topics. Advanced process control strategies require dynamic physical changes to processes (i.e. through process parameter adjustments). However, altering the process state may result in unforeseen costs and process uncertainty [6]. Understanding the economic impact of these process level control decisions on the overall manufacturing system is of key importance for optimal decisions to be made. Economic management factors influencing decisions include quality, throughput, cost, safety, flexibility and organizational limitations, while the technical factors include process specifications, conditions and controller parameters [7]. Researchers have suggested that groups of autonomous software agents should exist across a manufacturing organization to help make these complex decisions. By utilizing a communication mechanism they should be able to perform co-operative decision making in order to find the optimal solution to a set of organization wide goals. These goals can cover issues such as minimizing lost production volume due to breakdowns or minimizing production cost [4]. Of course, to prevent adverse situations, the decisions made by these agents must be rational compared to similar solutions made by domain experts. A decision made by a software agent can be said to be rational if the behavior exhibited in a certain situation is effectively similar to that exhibited by a human expert in the same situation [8]. For manufacturing environments,

784

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

Nomenclature G M P S U X Y

W b

H c m p u x a b g

performance goal matrix perceived data cube process parameter matrix process state matrix sensor data matrix experimental design data cube raw process response data cube domain knowledge model coefficients agent trade-off biases vector model coefficient matrix process constraint vector transformed feature vector process parameter vector single process cycle single sensor data vector model input vector model type (PU, UG, PS) bias toward a single decision factor result of cost-function

the ability to make rational decisions is critical for automated autonomous control, since they operate in real-time, dynamic environments and must handle concept drift and emergent situations without causing further problems. According to Nwana and Ndumu [9], rational agents include co-operation, autonomy and learning ability, even in the face of unforeseen interactions, incomplete and imprecise information [10]. For rational decisions to be made given all modern technical and economic factors, requires information sharing so that an understanding of the manufacturing organization and its various dynamics can be established through knowledge discovery, enabling the balancing of economic and technical factors to achieve an optimal production environment [11]. Therefore, the integration of the associated information processing sub-systems to perform the knowledge discovery, data acquisition, co-operation, decision making, planning and plan execution are of critical importance. Formal architectures are therefore necessary for intelligent machine agents to streamline implementations and optimization of high-level process control so that high-performance, interoperable, heterogeneous intelligent solutions can be built. The Foundation of Intelligent Physical Agents (FIPA) has addressed this need to some extent by developing the architecture shown in Fig. 1 The FIPA architecture primarily covers services and discovery of services and agent interoperability issues [12]. This architecture provides the tools necessary to achieve the technical (process) and economic (enterprise) integration required for next generation manufacturing. However, the agent core services required to utilize the enterprise information and interface with specific environments, such as rational behavior, are not covered. It does not cover agent lifecycle and management, mobility, domains, conversational policy, or agent identity. These aspects have been addressed by various researchers’, however; few have addressed self-learning agent core services. The benefit of the FIPA architecture can be seen for multi-agent systems and it seems feasible that similar benefits can be expected if best practices and standards can be developed for the internal components of domain specific agents. IA architectures incorporating inductive learning and rational decision making therefore need to be developed and evaluated to determine the best practices for optimal, autonomous, production process control that can selfintegrate into the larger manufacturing systems (i.e. shop floor process control, scheduling, maintenance, quality control, etc.) for future manufacturing operations.

h j m n1 n2 n3 n4 n5 n6 n7 n8 n9 n10 r t tk w z

sensor channel machining cycle number single transformed feature number of process cycles in Y number of process parameters in P number of samples per process cycle number of sensor data channels in U number of performance goals in G number of process state variables in S number of model replications number of domain knowledge functions number of production lines number of processes on a specific line index denoting transformation operator time time corresponding to a sample k width of transformation window index of sensor data feature

This manuscript addresses the need for self-learning, rational core agent service architectures. The design and performance evaluation is presented for a system capable of intra-cycle autonomous or semi-autonomous shop-floor process regulation. The architecture is highly modular and intended to maximize the instantiation and integration of various algorithms. It also provides a method for leveraging enterprise information through integration with the FIPA architecture. The proposed architecture is intended to be located in the adaptive control optimization (ACO) layer of the industrial process control hierarchy and aims to find the optimal tradeoffs between various technical and economic manufacturing goals. Commonly used homogeneous intelligent algorithms, such as neural networks (NNs) and genetic algorithms (GAs), are used to populate the architecture and automate the intra-process regulation of a high-volume gun drilling process to reduce machining costs by minimizing tool wear. Section 2 provides a literature review of various agent architectures developed by other researchers in the field. Section 3 presents the system level integration of the agent with the manufacturing environment. Section 4 comprehensively describes the generic agent core service architecture developed in this research. Section 5 provides a high-volume industrial gun drilling process as an industrial case study to evaluate to the architecture. Sections 6 and 7 contain a discussion of the results and the conclusion, respectively.

Fig. 1. Abstract Agent Structure According to FIPA 2000 Specification.

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

2. Literature review of agent architectures It has been proposed by many researchers that in the future agents will be distributed among multiple automation devices (i.e. PLCs). Therefore, architectures are required that describe the integration of these agents with manufacturing environments and enterprises. Previous experiments incorporating distributed intelligent control by Maturana et al. [13], Carrascosa et al. [14] and Tichy´ et al. [15] developed foundation architectures for highly distributed control agents, by expanding automation controller extensions. Each agent typically represents a process, machine or device and works with other agents to achieve a goal [6]. Several agent types exist, ranging from purely reactive (stimulus–response) to deliberative or goal-oriented (proactively reasons about their goals and actions) agent types. Agent architectures usually have two recognizable parts: a wrapper and body. The wrapper accounts for inter-agent communication and just-in-time reactivity, while the body carries out the agent’s main functionality [16]. Maturana et al. [17] developed a multi-agent system (MAS) architecture to implement a survivable and reconfigurable system to control a naval shipboard water chiller. Each agent comprising the architecture consist of five components: a planner, equipment (real world process) model, execution control (translate plans to actions and control logic events to response-context events for the planner component), and diagnostics for monitoring the health of a particular device by comparing the ideal model response to the real-world response for a given set of parameters. The architecture used Job Description Language (JDL) to communicate between various agents and devices. The water cooling plants (pipes, values, pumps, expansion tanks, sensors), loads (head generator, temperature sensor), and water services sub-systems were modeled as separate agents. An Agent Management Service (AMS) and directory facilitator (DF) were provided for agent registration, white and yellow page services. In related research, Maturana et al. [6] proposed an intelligent agent architecture, for distributed manufacturing organizations, and described a method utilizing tags to integrate the agent control software, SDE, soft controller, and simulation engine. The architecture is demonstrated in the context of an industrial-sized water cooling system. The research team utilized simulation to ensure the agent would have a suitable behavior if it were used on the real-world process. Further research performed by Maturana et al. [13] presented work toward further developing and formalizing their architecture for integrating an agent virtual machine with embedded process controller firmware. The goal was to transform the underlying system into a fully integrated and open heterogeneous system. They presented their research by automating a shipboard chiller unit to demonstrate the use of the agent virtual machine, which decentralizes controller intelligence for a more robust control environment. The agent was able to execute directly on the controller, interfacing with the low-level controller logic via a data table interface. The proposed agent architecture contained reasoning (dynamic interagent co-operation), parsing (FIPA inter-agent communication) and communication components (message encapsulation based on the Common Industrial Protocol standard). Tichy´ et al. [15] attempted to create a generic infrastructure of multi-agent systems (MAS) for real-time control of complex industrial systems. The static 3-tier hierarchical architecture was FIPA compliant and contained system-wide (maintain overall goals and strategic decisions), process-wide (functionally sub-divide system) and equipment-level (low-level real-time control, local planning and constraint evaluation) layers. The internal structure of each agent consisted of a planner, diagnostic, equipment and execution control module. The planner module scheduled activities,

785

allocated necessary resources, and developed the plans. The diagnostic module provided the planner with hardware status, while the equipment model provided a real-world model of the hardware and monitored control efficiency. The execution control model managed the physical execution of plans, synchronizing them with control logic programs and passed control logic events back to the planner module. Carrascosa et al. [14] developed a simulation prototype for a multi-agent mobile robot mail delivery system utilizing the SIMBA MAS architecture with the GAIA methodology. The results obtained by Carrascosa’s team showed that delivery efficiency was improved when a task planning agent was incorporated compared to random task assignment. This trend held true even when external environmental perturbations were introduced. Additional tests performed when removing continual agent collaboration and replanning showed decreased effectiveness of deliveries. The work clearly demonstrates the advantage of agent collaboration in dynamic environments over more traditional static implementations. The GAIA methodology addresses general aspects for agent-oriented analysis and design, and focuses both on aspects of the agent societal and internal agent design [18]. It encourages a developer to approach agent systems design as a process of organizational design and provides an agent-specific set of concepts which can be used to model complex systems. The key concepts in GAIA are roles, which can interact using well defined protocols and associated responsibilities, permissions, and activities. The GAIA design process involves defining a hierarchy of agent types required to represent the complete system, develop services models for the agent types based on functions, protocols, safety, etc. required by the agent types, and finally define acquaintance models to represent the communication links between agents [18]. The ARTIS agent architecture is a vertically layered, hybrid architecture of the intra-agent structure that was developed for environments requiring hard real-time constraints. It is able to guarantee response times through an off-line analysis of the agent specification [19]. Julian et al. [20] has utilized these agents as part of a proposed FIPA-compliant MAS architecture for the development of real-time multi-agent systems [20]. The ARTIS agents demonstrate their ability to determine the optimal solution given the instantaneous environmental state and have capacities for problem solving, adaptability and proactivity. The environment must initially be perceived through an array of sensors, followed by a reflex (low quality, high speed response) or deliberative (high quality response) process to generate a response for transmission to collaborating agents or effectors within a bounded time. The ARTIS agents therefore have a communication module, a set of internal agents (models of behavior/problem solving capability for IA to achieve goals), a set of beliefs (world model and domain knowledge) and a control module for the real-time execution of the internal agents. The internal-agent entities allow the abstraction of problem-solving knowledge in a modular way. Jennings et al. [21] created the Generic Rules and Agent-model Testbed Environment (GRATE). This is a general framework, for the construction of multi-agent systems for the domain of industrial process control. It embodies a significant amount of pre-existing knowledge related to the co-operation and control of the agent which can be utilized to expedite system building, therefore only domain-specific information is required to be added. GRATE agents consist of three pre-defined modules: the situation assessment module, co-operation module and control module. These modules are implemented as forward-chaining production systems and act on the user implemented domain level system. The domain level is application specific, and has previously been implemented to address problems such as detecting disturbances in electricity networks, locating faults and proposing remedial actions. The co-

786

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

operation and control layer is a meta-controller, which operates on the domain level system in order to ensure that its activities are coordinated with those of other agents. The situation assessment module acts as an interface between the local and co-operative control mechanisms. The control module ensures tasks are carried out based on their priority as classified by the situation assessment module and that all relevant information is obtained for task completion. The situation assessment module is also responsible for detecting the need to co-operation from other agents and invoking the co-operation module to establish the social interaction. The architecture also consists of an information store that provides a repository for all domain information which the underlying system has generated or which has been received as a result of interaction with other agents in the community [21]. Jennings et al. [22] continues their research by presenting the development of an agent-based infrastructure for managing business processes, called ADEPT (Advanced Decision Environment for Process Tasks), and how they were applied to a business process for providing quotes to customers for network services. The architecture also provides a method for structuring the design and development of a MAS as well as internal agent structure for business process management systems. It identifies the key concepts in this view as autonomous agents, negotiation, service provision, service level agreements, resource management, and information sharing. The ADEPT system provides algorithms, interfaces, language definitions, and structures for realizing the key concepts of business processes. The architecture internal to each ADEPT agent consists of an ‘‘agent head’’ with functional components to perform communication, service execution, situation assessment, and interaction management. The ‘‘agent head’’ architecture is similar to the GRATE and ARCHON agent models and is responsible for managing the agent’s activities and interacting with peers. The architecture also defines an ‘‘agency’’ which represents the agent’s domain problem solving resources. Work by Christensen [23] outlined an open, standards-based architecture for holonic manufacturing systems which is capable of fulfilling the economic and technical requirements for global adoption, deployment and support. He states the technical requirements include a distributed, open architecture component based approach for encapsulation and to provide IP protection, portability and reuse. The components of the proposed system include control and automation, machine and process interface, communication, human/machine interface, and software agents. The system proposes the use of low-level and high-level co-operation domains with functions comprising these domains arranged in ‘‘service stacks’’. Low-level co-operation refers to normal, non-holonic control and automation functions, while high-level co-operation refers to the integration of these functions into holons through the use of software agent technology. Examples of low-level co-operation functions include the control and automation of physical equipment (i.e. function blocks), real-time inter-controller input–output and communications, and the associated human machine interfaces. The high-level control architecture addresses functions such as intra-holon autonomous task planning and inter-holon co-operation (i.e. negotiation and co-ordination of mutually agreed upon tasks for process improvement or fault recovery). Bussmann [24] compares the ideas and concepts of holonic and agent-oriented manufacturing, showing that both paradigms have different, but complimentary views on manufacturing control and that a combination of both is beneficial. He states that holonic manufacturing systems deal with the overall structure of the manufacturing process and in particular with the integration of equipment, control, and workers, whereas multi-agent systems concentrate on the design of the information processing in a control system and its implementation. He proposes that agent technology may be used to enhance holonic manufacturing systems. Bussmann goes onto

demonstrate his thesis by using agent technology to design and implement holonic information processing in a manufacturing system. He presents an agent-orientated information processing architecture for a holon that includes, a core consisting of social and individual decision making, behavioral control (physical world), co-operating techniques, communication (physical communication) and organizational techniques. Work by Brennan et al. [25] addressed holonic control devices that interface with physical manufacturing equipment as well as the communication necessary to allow these agents to negotiate/ reason about manufacturing performance and dynamically adjust the process to recover from abnormal operation. The agents are formed using a hierarchical approach consisting of deliberative, data table, control function, physical and simulation layers. The Deliberative Layer contains two forms of functionality: application specific (user defined) and generic. The generic decision making functionality is realized through the Planner (construction of control-action steps via negotiation with other local or remote agents using agent communication language), Process Model (meta-knowledge about problem solving domain to evaluate current or predict future process states), Execution Control (manage plan execution) and Diagnostics (local self-assessment activity to verify the correct operation of the hardware) modules. The agent interfaces through a data layer to interface with the control function of the hardware controller. The Control Functions Layer is the user defined application logic that controls the activities of the physical hardware or process. A Simulation Layer is also described for the simulation of the physical hardware and process entities. Vrba et al. [4] developed a flexible and fault-tolerant system based on agent technology that utilized radio-frequency identification (RFID) tags to track workpieces moving though a production system. The agent system was tested in a laboratory environment with commercially available automation equipment and was shown to perform successfully. The system consisted of product, work cell, conveyor belt, and diverter agents. The diverter agent was able to change the destination of workpieces depending on the required operation to perform or in response to system failures. It contains a routing table that is developed at start-up about which work cells can be reached and the associated costs to compensate for breakdowns or variations in structural layout of the factory floor. The product agent negotiates with the other agents to achieve its own goals, which is to arrange its transport and obtain the correct processing at each work cell in the correct sequence [4]. Goh et al. [26] designed an autonomous agent network methodology using clusters of distributed cooperating IAs that dynamically modeled overall production operations to promote agility for manufacturing enterprises, for adapting to market changes and unforeseen circumstances. A very general overview of the internal agent architecture was given, and a more detailed description and discussion of their application to the production environment, but no experimental results were provided. Heinrich et al. [27] designed, but did not provide experimental results, for a distributed supply-chain multi agent system for production planning (scheduling and optimization) of manufacturing operations. The goals and behaviors of each of the agents comprising the system were discussed, but no learning seems to have been included and details as to their internal architectures and implementations were not provided. Feng et al. [28] developed a prototype MAS integration framework to demonstrate manufacturing system interoperability with higher-level applications. The system uses the FIPA architecture for communication and each agent contained rules to support the integration of manufacturing planning, predictive machining models and manufacturing control. Even though modeling and

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

optimization agents are briefly discussed, no internal architecture and description is provided for their internal architecture. Djurdjanovic et al. [29] presented ongoing and completed research on a watchdog agent prognostics toolbox for multi-sensor assessment and prediction of machine or process performance. Sensory processing and autonomous feature extraction methods were developed to facilitate a plug-and-play approach where the agent can run autonomously. Many experimental results were presented as examples of implementations of the system, however, no internal architectures or implementation details were provided. Hao et al. [3] presented an open, flexible and scalable autonomous agent and MAS development environment which can be used to build integrated and coordinated engineering applications from agent level reusable component packages and programming tools. It features loosely coupled communication, multiple message queue handling, finite state machine based conversation control, and multi-threaded execution. The architecture was applied to the design of a wheel-axle assembly to validate the prototype software environment. The CoSy Architecture Schema Toolkit (CAST) developed by Hawes and Wyatt [30] supports the construction and exploration of information-processing architectures for intelligent systems such as robots. The system uses parallel working memories to implement novel intelligent systems using flexible methodologies. Hilari et al. [31] successfully used a soccer simulation to demonstrate the utility of a formal specification for an IA architecture based on the biological immune system concept. The ‘‘immune system’’ is used to arbitrate several behaviors and facilitate interagent communication. Shen and Norrie [32] presented a hybrid agent-based approach and implementation to organize an enterprise’s activities and those of its customers, suppliers and partners into an open dynamic environment. Networks were used to facilitate the sharing and exchange of information and knowledge related to customer requirements, products, production, and services, etc. Li et al. [33] developed a three agent approach to facilitate the integration of production planning and scheduling functions approach. The improvement of this methodology was shown on a simulated production line. From these reviews it can be seen that many researchers have been developing various software agent methodologies and architectures. However, many of the systems presented utilize pre-defined knowledge as their primary source for making decisions. In this work we develop a self-learning deliberative agent architecture for manufacturing process control.

3. System level agent integration Fig. 2 illustrates the manufacturing enterprise wide integration of the proposed agent. It can be seen that the various departments exist on a common network backbone, with each department containing its specific software systems to perform their required function. In each of these departments we have indicated the agent type responsible for managing the activities specific to that department. Each agent has the possibility to interact with other agents by establishing a communication channel. Communication in the multi-agent system is in most cases based on a message passing technique [15]. The Agent Communication Channel (ACC) provides message routing across the network. To access the ACC each agent must register themselves and their capabilities with the Agent Management System (AMS), which provides yellow page services. Once the agents are registered at the startup they are not organized. The agents must establish their functional hierarchies based on their requirements assisted by a capabilities search by the DFs as specified by the FIPA architecture. Each DF agent provides a list

787

Fig. 2. Enterprise wide integration of proposed software agent architecture.

of agents that have a certain requested capability. During registration, each agent registers itself with the local directory facilitator (LDF) in its department, which provides white page services. The DFs exchange information automatically and seamlessly to create the MAS. As recommended by Tichy´ [15], it is advantageous to use a single global directory facilitator (GDF) with multiple LDFs, to distribute the network load imposed by capability searches as well as improve the robustness of the agent system as a whole. If the GDF becomes unavailable, then a new GDF is selected from one of the LDFs. We propose the integration of agents at each departmental level, such that the Production and Planning Agent (PA) department will set the goal for each process (P) or production line. A sub-function of production is quality control that ensures the parts produced meet the engineering specifications. The Purchasing Agent (UA) is responsible for ensuring that the necessary materials, components and equipment are ordered in the correct quantities and arrive on time. The Inventory Agent (VA) is responsible for holding all tools, spares, raw materials and equipment required to service the manufacturing process as well as providing a buffer should problems occur during the purchasing process. The Maintenance Agent (MA) is responsible for reactive, pro-active and/or predictive servicing of any faults that occur with the processes. The Shop Floor Agents (SA) are responsible for ensuring the goals of the PA are executed, which includes informing the MA of the machine state, as well as informing the VA of current and future requirements. Of course the final quality of a manufactured item depends in part on all the processes utilized during its production. Therefore, each SA must negotiate with other SAs to find the optimal trade-off between local process optimization and global quality. An example of this is if a process is able to perform its operation twice as fast as a process further down the production line. This may result in a buildup of work-in-progress (WIP) stock. Even though this may not seem like a problem, it causes the overall production system to become less reactive to customer demand, since all the WIP must be completed before another part can be produced. In addition, typically part quality is only checked periodically at certain locations on the line. Therefore, a significant portion of the WIP may be produced as scrap or need to be reworked before the problem is detected. It is therefore important that the SA communicate with the UA and VA to ensure the production schedule according to the PA is well co-ordinated. The VA will in turn communicate with the UA regarding obtaining nec-

788

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

Fig. 3. Enterprise aware FIPA core service agent architecture for the adaptive control optimization of manufacturing processes.

essary inventory. Should any problems arise, such as insufficient inventory; the various agents will need to negotiate with the PA in an attempt to find the most optimal course of action. The following section will discuss the core agent service architecture developed as part of our research to realize a self-learning SA for manufacturing processes.

4. Experimental agent service architecture Fig. 3 presents the proposed architecture for the core service component for the ACO layer to realize the intelligent machining developed in this research. It is well known that intelligent machines require the base components of perception, cognition and execution for intelligence [11]. In addition to the data to operate on, these three concepts define the foundation of our architecture. The episodic memory stores autobiographical event data, such as specific experiences (events, times and environmental conditions), as feature tuples in datacube, M. Each tuple represents nuggets of information, extracted from the input–output relationships of raw process data. The semantic memory contains concept-based knowledge unrelated to specific experiences. This memory contains learned coefficients, H, and explicitly stated expert/domain knowledge model coefficients, w, constraint vector, c, delimiting the agents’ environment, and bias vector, b, to adjust the agent’s decision making process. The episodic and semantic memories together form the declarative (explicit) long-term memory of the agent forming the complete set of knowledge regarding the agent’s environment. Enterprise integration and MAS interoperability is achieved through the manufacturing metadata communication interface,

which adheres to the FIPA abstract architecture. This interface allows access by other agents and systems to agent’s declarative memory to utilize its knowledge or influence its behavior. The procedural memory (methods for learning sensor-motor skills) of the IA is comprised of the perception, f1, modeling, f2 and decision/action computation, f3, phases. The computer numerical control (CNC) and adaptive control (AC) layers are grouped with the process, f4, so the system takes the perspective of an operator optimizing the process, by adjusting process parameter vector, x. Initially, the model of the system, f4, is unknown. For a given process parameter vector, x, the process will return the three dimensional process response datacube, Y, containing samples of the sensor data, process parameters, states and process performance metrics, for each machining cycle performed. The IA must design a set of experiments, X, to obtain sufficient data from f4 to discover the true model of the system. Intelligence in this architecture is achieved through physical decomposition agent encapsulation, where the agent independently learns to represent f4 through experiential learning. Learning is complete when the optimal coefficients, H, are found to parameterize model f2 so it accurately represents f4. Ultimately f2 should approximate f4, turning experiences into usable knowledge, so hypotheses about the input–output relationship, M, for various candidates of vector x can be generated. As the number of sensors and machine cycles increases Y may become significantly large and a method is required to search for and extract any useful information contained within. To achieve this, data mining is integrated within the perception phase of the IA framework as an extension to the Open System Architecture for Condition Based Maintenance (OSA-CBM) layer specification. The combinations of these methods forms the per-

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

ceptual skill required to analyze data cube Y, and discover embedded technical and economic relationships that could be used to optimize the machining process. Rational IAs must disregard irrelevant sensor and related data that may cause bias, so that significant interactions between technical (actuator control) and economic (process performance) metrics can be found [34]. These learned interactions facilitate rationality through the action computation phase, where hypotheses are generated using the learned and implicitly specified expert knowledge. Experiments are designed to test them, guiding the next iteration of data collection, growing the agent’s knowledge of the process and moving toward the goal state.

X½t ¼ f3 ðf2 ðf1 ðY½t  1Þ; H; WÞ; c; bÞ

ð1Þ

Eq. (1) provides a general description of the encapsulation and sequential transformations performed on the data as it flows through the agent as illustrated in Fig. 3 In the equation, X[t] represents the matrix containing the experimental plan for the next batch of machining cycles comprising learning iteration t. This matrix and can be thought of as the result of a series of sequential transformations of the input data cube, Y[t1], obtained from observations from previous learning iterations. Each transformation in the architecture is represented by a function, f. Process constraints, c, are represented by a vector of constants that can be directly mapped to process parameters or indirectly using domain knowledge functions defined by model parameter matrix, w, and biases, b. The domain knowledge, constraints and biases only apply to the action computation phase, and are specifically utilized in the cost function to hypothesize about the performance of the process given a certain process parameter vector. Matrix, H, contains the coefficients for the models representing the learned knowledge. Additionally, domain knowledge could also be used as an integral part of the metaknowledge regarding the learning process (i.e. NN step sizes, or GA mutation probabilities). The following sections will expand on the internal structure of each phase of the architecture. 4.1. Agent action computation phase This phase of the architecture forms the IAs reasoning, hypothesis generation and planning capability as illustrated in Fig. 3 It consists of a cost-function, f3,1, search algorithm, f3,2, and experimental planning layers, f3,3. Initially the agent has no knowledge of the process and can only rely on domain knowledge and constraints to create the initial experimental design for the first learning iteration. As the machines understanding of processes dynamics grow in future learning iterations, a more controlled and optimal manufacturing process will be obtained [2]. The search algorithm proposes various actuator settings for the costfunction to evaluate in terms of expected response, risk (i.e. breakage, scrap, and downtime), domain knowledge, and constraints. A hypothesis, p, is thus generated regarding the optimum process parameters. The experimental designer is responsible for developing a series of experiments, X, to either prove or disprove the proposed hypotheses. Experiments can be scheduled when sufficient time and production capacity is available to buffer unforeseen events, thereby mitigating the impact of unscheduled downtime. Depending on the process, experiments could be scheduled using previously tested process parameters to ‘‘create’’ additional capacity temporarily, to enable higher risk experimentation to be performed, but possibly increasing tool cost (increased wear). This situation is possible since higher feedrates generally result in reduced cycle (machining) time, which leads to increased production rate and hence additional ‘‘work in progress’’ parts to buffer any upstream failures.

789

4.1.1. Agent performance evaluation cost mapping function The learned relationships from the technical and economic modeling phases must be combined with constraint and domain knowledge to evaluate hypotheses suggested by a search process regarding optimal process parameters. gðpÞ ¼ min p2R

b1 f 2;1 ðf 2;1 ðp;H1 Þ;H2 Þ þ b2 f 2;1 ðf 2;1 ðp;H1 Þ;H3 Þ þ b3 Vðf2;1 ðp;H1 ÞÞ þ b4 Vðf2;1 ðf2;1 ðp;H1 Þ;H2 ÞÞþ b5 Vðf2;1 ðf2;1 ðp;H1 Þ;H3 ÞÞ þ b6 f2;1 ðp;c; W1 Þ þ ::: þ b5þn8 f2;1 ðp;c; Wn8 Þ

!

ð2Þ

Eq. (2) shows the implementation of the multi-objective cost function used to determine the hypothesized effect of each process parameter vector. This method is very similar to that used in the work performed by Runarsson and Yao [35]. In the equation, the result is a scalar quantity, g, representing the degree to which the solution proposed by vector p is optimal. The goal of the search algorithm is to minimize g. The variance operator is represented by, V(), f represents the mean of the function output for a given coefficient matrix H, b variables are scalar biases representing weighting of the various trade-offs (risk versus reward), f2,1 represents the general model structure comprising the modeling phase that describes the real-world system. The weighting of each term determines the manner in which the combinations of criteria contribute to the optimum search path [36,37]. The term, f 2;1 ðf 2;1 ðp; H1 Þ; H2 Þ maps process parameters to sensor data (model PU parameterized by H1) and the predicted sensor response to goal (economic) variable (model UG parameterized by H2). This key relationship provides the predicted process response for each goal given a vector of process parameters, p. The variance operator is used to predict experimental risk (model uncertainty) by observing the variance in the model coefficient search over successive training iterations. The model parameterized by H3 is used to predict the process state from the process parameters. The models parameterized by w indicate implicitly specified domain knowledge describing well known real-world relationships. Along with the constraint vector, c, the domain knowledge can delimit or guide the search, due to machining, tooling, economics and safety factors. 4.1.2. Experimental planning for pro-active data collection Because it can be expensive and wasteful to develop a broad model of the whole search domain, searches must be planned using distance formulation, hazard zone modeling, and constraint and domain knowledge [36]. Design of experiments (DOEs) provide a mathematical framework for process parameter (factor) settings to discover their underlying influence and interactions with the minimum runs while excluding experimental ‘‘noise’’ [38,39]. This is essential in manufacturing due to the cost of performing experiments. Fraleigh et al. [39] and Barton et al. [40], also suggest DOEs are necessary for successful modeling of operating processes. The DOE should: (1) grow the internal process model, (2) move the process closer to its goal state, and (3) focus or expand the search depending on the required/allowed granularity for the next learning iteration [41]. In critical processes, the range the initial DOE covers should be kept small, around the area of the current process parameter settings, and each iteration should slowly translate or expand across the response surface in an attempt to avoid tool or machine damage, or the manufacture of scrap parts. However, this may lead to the process getting stuck in local minima. By using GAs or a similar global optimization algorithm this could be overcome. Initially, for a new instantiation, the IA needs to be bootstrapped before it can start generating hypotheses. This process basically entails that the initial process parameters should be designed by a skilled operator to minimize the experimental risk, or the agent must rely on any pre-defined domain knowledge and constraints (i.e. expanding experimental space to cover maxi-

790

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

mum region as defined by the technical limitations of the process). Of course the latter technique has significant risk associated and it is recommended that a skilled operator at least review the proposed design before execution is begun. Execution of the DOE will result in the generation of new data describing process performance. The following section will discuss the perception process the agent utilizes to convert this acquired raw data into experiences and subsequently into knowledge. 4.2. Perception and data mining relationships from process data sources The perception and data mining phase, f1, of the proposed IA architecture is illustrated in Fig. 3. The process parameters (independent variables), process sensor data and process goal data (dependent variables), and data from other agents across the enterprise as analyzed for any information they main contain. The data mining algorithm is responsible for providing an optimal sensor data set characterizing each goal variable for the learning algorithm to model. Only essential data must be included so better model generalization, using fewer training samples, can be achieved. This improves learning speed and leads to improved classification [42]. To enable this functionality, the proposed structure is loosely based on various components from the Open System Architecture for Condition Based Maintenance (OSA/CBM) architecture, which defines a standard for moving information in a condition-based maintenance system to improve interoperability and integration of design changes. It defines generalized layers and interfaces, not imposing any requirement on the internal structure, for the management of data acquisition, data manipulation, state detection, health and prognostics assessment and advisory generation phases, and forms a good basis for the lower levels of this layer. Concepts suggested by Brophy et al. [43] are also incorporated to realize multi-sensor process monitoring and analysis, resulting in four major areas that constitute this phase: sensor selection, primary transformation (i.e. Fast Fourier Transforms (FFT), wavelets), secondary transformation (derivative, integral, etc.) and tertiary transformation (statistical analysis techniques). The feedback loop, indicated by the dotted line shown in Fig. 3, phase f1, can be used to implement wrapper feature extraction for perceptual skills learning as in humans.

 Yn1 ðn2 þn4 þn5 þn6 Þn3 ¼ Pn1 n2 n3

Un1 n4 n3

Gn1 n5 n3

Sn1 n4 n3

 ð3Þ

Eq. (3) describes the three dimensional data cube, Y, used to transfer real-world data to the perception phase of the architecture and to the modeling phase. This data cube is formed from the data cubes of process parameters, P, sensor data, U, performance goals, G, and process states, S. Variable n2 represents the number of process parameters the agent is able to manipulate to bring about change to the real-world process, n4 is the number of sensor channels monitoring the real-world process, n5 is the number of performance goals to meet, n6 is the number of variables describing the state of the real-world process, n3 is the number of samples obtained per process cycle, n1 is the number of process cycles. The only difference between input and output is that before perception is performed datacube U represents raw sensor data, and after perception, U, represents an optimal set of transformed sensor data to be used for building models of the physical process. Post-perception this data cube is represented by matrix, M, and stored as an experience in the episodic memory. In either case, each tuple completely describes the process at the instant of capture through, for example, process outcome, CNC error/status code or sensor data. State data can be used to build classifiers for process idiosyncrasies

(i.e. resonance, tool breakage detection) [44]. This allows the agent to avoid experimental runs in areas where the finished part may have unacceptable quality (i.e. dimensional variation, poor surface finish) or excessive machine tool degradation [45]. 4.2.1. Sensory signal feature extraction and transformation Data pre-processing is necessary to remove noise, anomalies, and missing data to ensure data accurately represents the realworld process [41]. Process data is then partitioned to facilitate the determination of intra-cycle temporal effect on process performance [46]. Feature indexes also need to be converted from the time domain to an alternate domain independent of process parameter effects (i.e. feedrate). In addition, features may be transformed to the frequency domain for closer-interrogation. Time series (statistical metrics) and frequency-domain (FFT, wavelets) transformations have previously been used in research by Axinte et al. [47], as transformations to maximize the information extracted from the features. From both the time and frequency domain transformation of each feature, characteristics are extracted. These characteristics may include, various statistics from the time-domain signal (i.e. quadratic mean, variation), or frequency domain representation (i.e. bandwidth, distribution). These extracted characteristics may also be combined with other characteristics using mathematical operators such as multiplication or division (e.g. p1, p2, p1p2) to improve their combined performance [48]. An example of this would be the combination of the vertical and torsion force recorded by a dynamometer to calculate the perceived kinetic friction between tool and work material.

mr;h;j ½z ¼ fr ðuh;j ½tk  w; :::; uh;j ½t k Þ

ð4Þ

Eq. (4) details the extraction and transformation used for the IA to extract a single discrete time-based sensor data stream vector from the raw sensor data cube. In the equation, u represents the timebased sensor data vector obtained from data cube U, h is the sensor channel, j signifies the machining cycle from which the data should be obtained, w is the width of the data to use for a single window, m is a scalar representing a transformed feature, with tk representing the time sample corresponding to the extracted feature, fr is the operator applied to the sensor data vector, u, to obtain the transformed feature, r is the index (type) of the transform to apply to the data. It can be seen that domain z has a much lower sampling frequency compared to the original data, reducing the number of data points. However, feature transformation typically will result in higher dimensionality than the original data set, which tends to increase the sparseness of the dataset. An illustration of the feature extraction process is provided in Fig. 4(a) for time-based sensor data. It can be seen that the high resolution time-series signals have been reduced to only 9 sequential features, and the original 5 sensor channels will become 20 channels after 4 statistical transformations are applied. Therefore, dimensional reduction is necessary to remove any unnecessary information contained in the raw data. 4.2.2. Dimension reduction Data features representing quantities influenced by the goal variables must be identified and/or combined to reduce the dimensionality of the data and provide a signal highly correlated to the goal variable, in order to optimize the model building process [49]. Multiple variables may measure the same driving principle governing the behavior of the process. In this case, a group of variables exhibiting similar variance can be grouped and replaced with a single new variable [50]. There are many linear and non-linear techniques to achieve this. Probably, the best known linear technique is Principle Component Analysis (PCA), however non-linear variations of PCA also exist (i.e. kernel PCA). PCA can convert large

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

791

Fig. 4. Normalized sensor waveforms illustrating the gun drilling process response for a typical machining cycle are shown. Features extracted by the agent according to Eq. (4) are shown as well as the comparison in sensor signals between new (dotted line) and worn (solid line) tools. The signals for 10 successive cycles were averaged and the standard deviations from this average was calculated and represented on the plot as the double lines. One machining cycle comprises both a blind hole (z = {1. . .4}) and through hole (z = {6. . .9}). Feature z = 5 represents the spindle jog for relocating to the position of the 2nd hole. (a) Spindle motor current trend showing 9 variables describing each transformed feature z with width w; (b) RMS spindle vibration; (c) Feed motor current; (d) Coolant pressure (e); Coolant flowrate.

amounts of correlated input data to a set of statistically decorrelated orthogonal principle eigenvectors, which are then sorted by decreasing variance [50]. Fig. 5 illustrates the PCA process on a set of sensor data. It can be seen that the 20 possible sensor and transformation combinations have been grouped into two variables (PC1 and PC2), which can represent the state of the process. Because this is a linear algorithm, linear or slightly non-linear correlations must be assumed. If non-linearity exists minor principle components (PCs) may contain important information and data

would need to be linearlized prior to PCA processing [49]. From the reduced dimensional dataset, a decision needs to be made about which features and principle components are most highly correlated to the current goal variable during the process of feature selection. 4.2.3. Feature selection Identifying features that influence the process performance is essential for good models to be built [51]. Unrelated features can

792

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

chain between the process parameters (technical) and process performance (economic) to obtain the optimal matrix, M. The optimal matrix will be passed to the modeling phase to convert this relationship into a predictive model, allowing the agent to learn about the process. 4.3. Modeling manufacturing process relationships The goal of modeling is to build a predictive model data cube, M, stored in the episodic memory. Fig. 3, f2, illustrates the internal structure of the modeling phase of the agent. The agent consists of three model classes: the process parameter-sensor data (PU), sensor data-goal (UG) and sensor-state (US) class formed for each link in the inference chain. The PU and US models represent the technical mappings and the UG model represents the economic mapping. The learned relationships are encoded as the defined model structure, f2,1, and parameters stored in the semantic memory, H. These models describe the system allowing insightful learning to take place by using predictions to accelerate the learning process [41]. The more accurate predictions are, the better the system will learn and achieve the required trade-offs between the technical and economic factors. Cook et al. [52] suggests that in order to ensure high quality predictions at the maximum prediction horizon, multiple models should be used. By using multiple models collected from successive iterations of the leaning cycle, each is responsible for describing a sub-set of the process space or in the case of overlap, improving the reliability (repetition priming) of a prediction in the overlap area. Updating a model on every instance may not be accurate in a noisy domain. This shortcoming can be avoided by employing incremental batch learning methods [34]. By learning the system’s behavior and propagating it into the near future through simulation forecasting, the system can predict future emergent behavior and minimize or prevent otherwise unforeseen perturbations in production. n7 X f 2;1 ðHa ; xÞ ¼ 1 f2;1 ðHa;i ½t; xÞ n7 i¼1

Fig. 5. Point cloud and vector plot showing result of secondary transformation of drill data, (a) PCA eigenvectors of transformed sensor data for feature 2 of each process cycle, showing detection of process operating point shifting for five different process parameter combinations; (b) contribution of transformed features, m (Eq. (4)), to 1st and 2nd PC created from the 2nd sensor feature from each process cycle for the drilling process (values for transformation index, r, correspond to 1, RMS; 2, standard deviation; 3, skew; 4, kurtosis; values for sensor channel, h, correspond to 1, spindle current; 2, feed current, 3, vibration; 4, coolant pressure; 5, coolant flow rate).

degrade the model performance by including trends due to unknown process variation [52]. The fewer features or inputs the less training samples are needed as the search space is smaller, reducing the number of experiments that are needed [51]. The resultant feature vector must optimally link each goal variable with a relevant process parameter vector via a sensor feature. The final selection of inputs and outputs depends on the real-world application and goals that the agent must work with, and must be refined as the agent learns about the process [41]. The information about relevant process parameters obtained through the initial screening experiments can be used by the experimental designer in the action computation phase to design more optimal DOEs. The process parameters, PCs, goal metrics, process states matrix obtained from this iteration of the perception phase are stored in the episodic memory along with the data from previous iterations, forming a set of experiences the agent has perceived in its environment. The goal of feature selection is to establish an inference

ð5Þ

Eq. (5) shows the general model structure for f2,1. A matrix, Ha,i, is used to parameterize the mapping, which the agent determined by applying a learning algorithm to matrix, M, t represents the current learning iteration for a specific time interval. Variables, a, and, i, represent the model type (PU, UG, or US), and model iteration, respectively, The variable n7 denotes the number of models used and vector x, the process parameters or sensor data depending on the model type. 4.4. The action computation phase continued: hypothesis generation For the second iteration of the action computation phase, the agent does not need to rely on domain knowledge alone, and can now utilize the learned models as a component of the cost-function. Ideally, hypotheses more specific to the machine tool will begin to emerge, presenting more optimal process parameters for realizing a rational trade-off between the specified manufacturing goals. The following section provides a real-world example of the implementation of the proposed agent architecture. 5. Industrial drilling process experimental setup The learning ability of the proposed IA architecture was implemented and evaluated on a high-volume industrial gun drilling process. Drilling occupies approximately 30–40% of the metal removal processes in machining and it is one of the most common machine tool operations [53]. The process is shown in Fig. 6, and

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

793

Clamp Drill guide

Accelerometer Workpiece Drill

Coolant jets

Spindle housing

Fig. 6. High-volume gun drilling process in cast-iron including drill, spindle, guide and accelerometer.

uses a Carbide 2-flute 10 mm diameter spiral twist drill (DK460UF GT100) to drill two 250 mm deep holes (a blind and though hole) in a cast-iron block (Brinell 179–229) every machining cycle. Pressurized coolant (48 bar) was applied through a series of jets onto the machining area and workpiece. The company was struggling to control increased manufacturing costs due to excessive tool wear using trial-and-error approaches and therefore a more rigorous methodology was sought. In addition to rapid advances in cutting tool coatings and chipgroove geometries, tool wear and fracture are inherently complex, non-linear processes, which complicate cutting process optimization [45,54]. Research by Abu-Mahhouz [55], found that tool wear changes drill point geometry leading to unbalanced cutting forces, and wandering or torsional vibration. To provide insight into the process dynamics, sensors are needed to monitor key physical quantities. To monitor tool wear Franco-Gasca et al. [56], and Ertunc [53], suggested using spindle current, while Fu, Ling, and Tseng [54] suggested motor impedance be used for tool wear monitoring as more power is consumed for worn tools. Brophy et al. [43], suggests non-symmetric drill wear can be detected through vibration analysis. However, Ertunc [53] and Jantunen [57] suggest that feed force is a better measure of tool wear compared to torque signals. To optimize tool wear a method to monitor the tool wear and process state are needed to provide insight into the physical process. It can be seen that there are multiple sensor configurations that could be used to achieve this. For our research, a Wavebook 516 8-channel 16-bit datalogger (IO-Tech, USA) was interfaced to a laptop, which simultaneously filtered and sampled all the sensor signals at 20 kHz. Current transformers, IHA-25 (Sypris, USA), were used to measure a single phase of the spindle and feed motor current and an accelerometer, CMCP1100 (STI, USA), was stud mounted to the spindle housing in vertical orientation. A PN3022 pressure and SI1004 flow sensor (IFM Efector, USA), monitored the coolant supply. A trigger signal from the Power Mate I numerical controller (NC) (Fanuc, USA) that was controlling the overall machine tool sequence initiated recording at the start of each machining cycle. Normalized values were used during analysis as the sensors were not able to be calibrated to specific mechanical units due to production demands. For the initial IA bootstrapping, the agent generated a DOE based on pre-defined domain knowledge, illustrated in Fig. 7(a), as the five vertical groupings of the light yellow circular data points. At least three levels were used for each parameter so some non-linearity in the process response could be detected. Two runs for each parameter combination were performed to facilitate excessive variation detection and repetition priming. Additionally,

Fig. 7. Drilling process technical–economic model inference chain including variance maps for experimental risk assessment.

economic (cycle time) and technical (spindle speed) constraints were imposed on the allowable process parameter selection. Two constraints were imposed by the production department as indicated in Fig. 7(a), by the area covered by green stars and red diamonds, respectively. The first constraint specified that the cycle time for the process should not go below 60 s to ensure adequate parts were manufactured for the overall manufacturing process to remain economically viable. The minimum feedrate (540 mm/ min) corresponding to this cycle time was determined using a domain knowledge function. The second constraint specified the maximum advance per revolution (0.2–0.35) per the tool specification datasheet. The allowable spindle speed and feedrate range to prevent tool damage was again determined using a domain knowledge function. For the initial DOE the importance of these constraints were slightly relaxed, after review by an experienced process engineer, to allow for a slightly larger response surface to be obtained. The data obtained from this initial DOE was provided to the perception phase of the IA, after which data cleaning was performed by filtering, min–max normalizing and storing the result in the short-term memory of the agent. For feature transformation, statistical metrics that have been successfully used in drilling sensor feature extraction were implemented for this phase of the architec-

794

G.H. Kruger et al. / Advanced Engineering Informatics 25 (2011) 783–796

ture and include the mean, RMS, standard deviation, variance, skew, kurtosis and gradient of the sensor signals [47]. Features were not combined, so that a more intuitive presentation of the agent’s performance could be determined. Tool position was used as the domain for the drilling process sensor data to prevent feedrate changes affecting the feature extraction process. For each drilling process cycle 180 sensor data features (5 sensors  9 features  4 transformations) were obtained. Fig. 4 shows the positive and negative standard deviation of the sensor data over 10 sequential cycles for both new and worn tools obtained from the drilling process. As explained by Eq. (4), the window over which each feature, mr,b,j(z), is extracted is shown by the enumerated grey bars (z = {1, ..., 9}). There is a clear distinction between a new (