Orchestrate Legacy Programs in Grid Yi Wang1, Jian Cao1, Minglu Li1, Ruyue Ma2, Feilong Tang1 1Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200030, China {wangsuper, cao-jian, li-ml, tang-fl}@cs.sjtu.edu.cn 2School of Computer Science and Technology, Shandong University, Jinan 250100, China
[email protected] Abstract With the development of Grid Technologies, Many researches and projects have done to deploy and integrate legacy programs into service-oriented Grid architecture. However, the legacy programs orchestration is still scarce of good approaches. In this paper, we overviews the architecture and legacy program execution mechanism of CGSP, presents an abstract workflow specification named ChinaGrid Workflow Description Language (CGWDL) which can be transformed into BPEL, and depicts the life cycle of legacy programs orchestration based on CGSP. An image processing application is used to validate the feasibility and efficiency of our solution.
The rest of this paper is organized as follows. Section 2 overviews the architecture of CGSP and describe the legacy program execution mechanism with CGSP. Section 3 presents the framework of our grid workflow system, analyze the mapping mechanism of CGWDL transforming to BPEL and the solution of enabling BPEL support WSRF. A demo for legacy programs orchestration is discussed in section 4. The last section concludes the whole paper and points out some future works briefly.
2. Overview of CGSP 2.1 Architecture
1. Introduction Grid computing offers seamless integration of various resources in a geographically distributed environment. One of the biggest obstacles in the wide-spread use of Grid technology is the existence of vast legacy programs and applications written in different programming languages that are not accessible as Grid Services. Some approaches to deploy and execute legacy programs into Grid environment have been proposed [1, 2]. As the Grid becomes stable and commonplace, problems put forward by grid users are much more complex and need be settled by the composition of multiple legacy programs. A more efficient approach for design and execution of legacy programs orchestration is needed. The Condor-G system [3] and DAGMan [4] build atop the GRAM service, but improve on it by enhancing its reliability and supporting the execution of Directed Acyclic Graphs. Similar task is introduced in [2]. But these solutions can not support BPEL [5], which is widely accepted. We propose a multi-layer workflow management system based on CGSP which can orchestrate legacy programs. A graphical workflow design tool is used to define abstract workflow with the form of ChinaGrid Workflow Description Language (CGWDL). Users can depict their requirement on the computing resource for each legacy program with Job Submission Description Language (JSDL) [6] during the design time. When a grid user wants to enact the process, an interpreter can find the appropriate computing service for each legacy program, and transform the CGWDL process to the form of BPEL. A BPEL engine will enact the BPEL process finally.
Figure 1. CGSP Architecture
ChinaGrid Support Platform (CGSP) is a grid middleware developed for the construction of the ChinaGrid [7]. Figure 1 outlines the CGSP architecture in a logic view. Main components of CGSP are Security Manager, Execution Manager, Information Center, Data Manager, Domain Manager and Grid Monitor. All these components are WSRF-compliant services deployed on CGSP Service Container. A user-friendly Portal
1-4244-1312-5/07/$25.00 © 2007 IEEE
5998
and a Grid Parallel Programming Interface (GridPPI) are also provided in order to enable user to use CGSP more conveniently.
2.2 Legacy Program Execution with CGSP The basic solution for legacy program execution with CGSP is General Running Service (GRS), which is a special WSRFcompliant Web Service and deployed on each computing resource in ChinaGrid, e.g. a cluster. GRS implements the virtualization of computing resource, provides uniform submission and execute interface for legacy programs task. With GRS approach, Grid user must appoint the GRS address when they submit the request to execute legacy program. Grid users often have some special demand on the computing resources, such as CPU numbers. However, users may not know which resource can satisfy their demand in grid environment. In CGSP, the solution is using Job Submission Description Language (JSDL) to define the requirements of grid jobs. The Execution Manager can parse the JSDL files submitted by grid users, and find a suitable computing resource through Information Center. Then, the JSDL job can be executed on as a GRS job. Figure 2 shows a brief view of the execution process of the two job types. If grid user submit a GRS job, Execution Manager will transfer the legacy program to computing resource through Data Manager, and invoke the GRS to execute the program. Otherwise, if the job is a JSDL job, Execution Manager will parse the JSDL file, match the configure requirement in Information Center, find a satisfied resource, then transfer and execute the program. More details can be found in [8].
technology, the problems put forward by grid users are more and more complex and the solution usually involve many legacy programs. So, more effective mechanism is needed to orchestrate these legacy programs. For this purpose, we develop a two-layer grid workflow management system based on CGSP.
3.1. Framework
Figure 3. Workflow Framework
Figure 3 shows the workflow framework for legacy programs orchestration with CGSP. The workflow management system includes three main parts: A graphical workflow modeling tool is used to write a file with ChinaGrid Workflow Description Language (CGWDL), which is an abstract workflow description language. A CGWDL parser has two functions. Fist, it confirms the concrete GRS address for each legacy program activities, and transfers the legacy program to relative computing resource. Secondly, the parser transforms the CGWDL to concrete BPEL file and Process Deployment Descriptor (PDD) file which are used by the workflow engine. An extended ActiveBPEL engine is responsible to enact the process. ActiveBPEL engine is an open-source workflow engine support BPEL. We extend it to support invoking WSRF-compliant services. To use BPEL4WS as the concrete workflow modeling language, we must solve two problems: how to transform CGWDL to BPEL and how to enable BPEL support WSRF.
3.2. CGWDL to BPEL Figure 2. Legacy Program Execution with CGSP
3. Workflow-based Orchestration The approaches discussed in section 2 can only support single legacy program execution. With the development of grid
CGWDL is presented as an abstract workflow description language similar with BPEL specification. In CGWDL, we define two types of abstract legacy program activities: GRS activity and JSDL activity. Each legacy program activity has a description file: GRS activity along with GRS argument file and JSDL activity along with JSDL file. The two types of activities can be transformed to several concrete activities. Then, the
1-4244-1312-5/07/$25.00 © 2007 IEEE
5999
modeling tool transfers the CGWDL file and all the JSDL files and GRS argument files relative with the legacy program activities to CGWDL parser. The most important process is the transform of legacy program activities: (1) Find the concrete GRS addresses through GRS argument files (appoint by user directly) and JSDL files (find resource dynamically) as we mentioned in section 2, evaluate the addresses to relative legacy program activities in CGWDL, and record these services in PDD file; (2) Transfer legacy programs to relative computing resource; (3) Then, as be shown in figure 4, each legacy program activity usually can be divide three invoke activities and several assign activities in a BPEL file.
Figure 4. BPEL activities to invoke WSRF service
4. Implementation
3.3. Enable BPEL Support WSRF Since BPEL is proposed for a standard of Web service orchestration, it can’t support invoking WSRF-compliant service directly. Since GRS is a WSRF-compliant service, we must afford a mechanism to enable BPEL support WSRF at the case of not revising BPEL specification. To support WSRF, BPEL need to manipulate endpoint reference (EPR) and pass the reference properties element in the SOAP header for invoking WSRF services. Some researchers extends the BPEL specification by adding an attribute InputHeaderVariable on invoke activity element [9]. But, we don’t think extending the BPEL specification is a good solution. So, we extend ActiveBPEL engine and enhance its support for WSA so that it can manipulate EPR and dynamically assign an EPR to a partnerlink. A new service invoking engine which can pass the reference properties in the SOAPEnvelope should be also added. By designating the type of partner service (WS or WSRF) in Process Deployment Description file (pdd file), extended engine will be able to select appropriate service invoking engine to complete the invocation of corresponding type of service. Figure 4 outlines the solution of enabling BPEL support WSRF in extended ActiveBEPL engine. The first step is to instantiate a new WS-Resource by invoking the resource factory's createResource method. The next step, also the most significant is the partnerLink assign activity, which dynamically sets the subsequent partnerLink as the endpointReference returned by the resource factory identifying the newly created WSResource. Subsequently, the BPEL invokes an operation on a resource-aware Web service, specifying the WS-Resource identifier to be used. In our case the operation simply updates the resource's stateful value with one passed as an input parameter. Last, we demonstrate calling one of the WSRF's standard methods for retrieving WS-Resource properties to verify that the stateful resource had in fact been updated.
In this section, we discuss an image process demo to show the feasibility of our orchestrating solution.
Figure 5. Abstract Image Processing Workflow
Figure 5 is a graphical workflow model designed with workflow modeling tool. The functions of legacy program jobs presented in this figure are: Activity named JSDL Job1 is a JSDL job to reverse the color of an image Activity named GRS Job1 is a GRS job to smooth an image Activity named GRS Job2 is also a GRS job to sharpen an image Notice that the workflow has a “XOR” logic control node, which means the conditions of invocation of GRS Job1 and GRS Job2 are mutually exclusive. Figure 6 depicts the transforming process from CGWDL to BPEL. Figure 7 shows the result of this workflow, the left image is the original image, the middle one is the result image if the condition in XOR node is “choice=1”, and the right one is the result image if the condition is “choice=2”.
1-4244-1312-5/07/$25.00 © 2007 IEEE
6000
……
