to the application programmer as a potentially large message-passing virtual ... As the application should not be aware of the fact that a task has migrated, the ...
Quasi-asynchronous Migration: A Novel Migration Protocol for PVM Tasks Dan Pei
Dongsheng Youhui Zhang
Meiming Shen
(Department of Computer Science, Tsinghua University, Beijing 100084, P.R. China)
email: Abstract Process migration is a desirable and useful facility for the Network of Workstations (NOW). This paper presents a novel migration protocol called quasi-asynchronous migration for PVM tasks, which allows non-migrating tasks to execute during most of the time of migration. Message flushing and message delaying are the key mechanisms used in quasi-asynchronous migration. Because the protocol is implemented on top of PVM and at user-level, it is both transparent to users and portable. Both the analysis of the migration protocol and the experimental results show that quasiasynchronous migration has lower overhead than other migration protocols. Keywords process migration, Network of Workstations (NOW), PVM
1. Introduction Networks of Workstations represent an increasingly popular distributed platform for long-running parallel programs. Several Capabilities are desirable in order to harness workstations more effectively and reliably, including support for a widely accepted parallel programming environment, process migration, intelligent resource management, fault tolerance, and totally transparent support of these features. PVM (Parallel Virtual Machine) [1] is a widely used environment that allows a heterogeneous network of workstations to be programmed as a single computational resource. This resource appears to the application programmer as a potentially large message-passing virtual computer. The ChaRM [2] system, which is developed by the authors, is designed to provide these capabilities to PVM programs. However, process migration is the only emphasis in this paper. Process migration is the ability to suspend the execution of a process on one machine and subsequently resume its execution on another [3,4]. Literatures [2,3,4,5,6] point out that process migration can be applied widely. First, processes on heavy-loaded machines can be migrated to lightloaded machines without terminating the entire application. Second, an idle workstation could be utilized by others if its owner allows so, and all the foreign processes could be migrated off the workstation timely when its owner reclaims it. Third, during the execution of a long-running application, one machine may need system maintenance or shutdown. When these happens, the processes running on this machine can be migrated to other available machines, and moved back later if necessary. Fourth, an I/O bound process could be migrated to the machine where the data resides, decreasing the network traffic. Process migration requires the ability to capture the state of a process on one machine and restore it on another. Process state is the necessary information that must be captured and reconstructed in order to guarantee the correct execution of the process after migration. The state of a UNIX process should include the data segment, stack, and the program counter, processor status word, stack pointer, and information about active files, interrupts and signals, etc. [6] Each process in PVM, which is also called as a task, is assigned a task ID or tid that uniquely identifies each task in the virtual machine. These tids are used to designate the source and destination tasks for messages. From the point-of-view of PVM, a task is one of a set of tasks that makes up an application. As the application should not be aware of the fact that a task has migrated, the tids, which each task got when it enrolled for the first time, are used throughout the lifetime of the entire application. Moreover, no messages should be lost, and all messages should be received in the correct
1
order. In this regard, a PVM task’s state includes its tid and the messages sent to/from that task. Since the emphasis of this paper is migration for PVM tasks, the term “task migration” and “process migration” have the same meaning. The remainder of this paper is organized as follows. Section 2 describes the problems to be addressed by task migration and introduces some general approaches to these problems. In section 3, the implementation of task migration in ChaRM is described and the quasi-asynchronous migration is presented in detail. Section 4 gives some results of performance tests. Related work is presented in section 5. Section 6 concludes the paper and discusses future work.
2. Migration for PVM Tasks There are roughly three stages during migration. First, the task to be migrated (MP) is suspended and a skeleton task (SP) is spawned on the destination machine. The skeleton task provides the infrastructure to which the MP’s state can be transferred. Second, the MP captures its state and writes the state into a TCP socket connection with the SP. At the other end of the socket, the SP reads out the state and reconstructs it in its own address space. Lastly, after the state transfer, the MP terminates while the SP executes in the context of the MP. In General, the migrated task (denoted as SP, too) in PVM has a different tids than the MP has, because they exist on two different machines, at the very least. Since the tids are used to designate the source and destination tasks for messages, some messages will be lost because of migration, as shown in figure 1. In figure 1, migration begins at time t0 and ends at time t1. (Note that t0 and t1 of different tasks may differ a little).
MP
t1
t0 freeze
t0
t1
t0
MP
dead SP
SP
SP
m1
m4
m3
m5
m2 running NMP
recv(MP,m2) send(MP,m1)
t1
MP
recv(SP,m2) NMP
NMP
(a)
(b)
(c)
Figure 1 The lost messages due to migration. MP, SP, NMP denote the task to be migrated, skeleton task and non-migrating task in an application, respectively. Note that NMPs are running during migration in this figure.
In figure 1(a), a NMP sends message m1 to the MP after t1. Since message m1 can not “find” its receiver MP, and the SP can not receive it either, message m1 is lost. If a NMP tries to receive message m2 which is sent from the MP after t1, the NMP can not receive m2 because it is identified as from the SP, not the MP. Because message m1 and m2 are lost both because the MP and the SP have different tids, almost all the systems supporting task migration set up and maintain a tid mapping table from MP’s original tid to corresponding SP’s tid. When trying to communicate with the MP, a NMP first finds the corresponding SP’s tid in the tid mapping table, and use this tid to communicate with SP actually. Therefore, messages like m1 and m2 will not be lost. However, message m3 and m4 in figure 1(b), will still be lost even if above tid mapping mechanism is used because the SP’s tid will not be valid until t1. Hence, a message like m3 , which is sent from the NMP to the MP during the time between t0 and t1, cannot find the receiver. Similarly, message m4, which is sent from the NMP to the MP before t0 but has not been received by the MP
2
before t0, will be lost. Furthermore, the tid mapping mechanism itself will introduce another kind of lost message like m5, which is shown in figure 1(c). After mapping the MP’s tid into the corresponding SP’s tid, a NMP tries to receive m5 from the SP. Since m5 is identified as a message sent from the MP, it will not be received by the NMP. However, m5 will be received properly if there is no tid mapping mechanism at all. Obviously, if some messages are lost, the application will run erroneously. Thus, the primary problem to be addressed by task migration is how to ensure the correct delivery of these messages and the correctness of the entire application. Various systems take quite different approaches to address this problem. Hence, these approaches can be used as the defining point of different migration protocols.
Definition 1: Synchronous Migration During synchronous migration, both MPs and NMPs are suspended. Therefore, there is no m3 at all. Before the transfer of the MP’s state, every task blocks to receive m4 or m5. This protocol is simple to implement, but the migration overhead is very high since the whole application has to be suspended during the migration.
Definition 2: Asynchronous Migration Asynchronous migration allows a task to migrate independent of what the other tasks in the application are doing. In general, only MPs are interrupted by migration. However, if the communication between tasks is via DirectRoute [1], the NMPs that are communicating with MPs are also involved. The implementation of asynchronous migration, like in MPVM [4], has to introduce message forwarding mechanism and message sequencing mechanism in order to ensure the correct delivery of the messages shown in Figure 1. Despite its high efficiency, this kind of protocol does have some disadvantages: message forwarding and message sequencing are complicated to implement, and the source code of PVM has to be modified.
Definition 3: Quasi-asynchronous Migration A novel migration protocol presented and implemented in ChaRM is quasi-asynchronous migration. Although both MPs and NMPs are interrupted, like in synchronous migration, NMPs are allowed to execute during most time of the migration. NMPs only do some coordination at the beginning and the end of the migration. As the analysis in the next section and the experimental results in section 4 will show, a NMP spends much less time for migration in quasi-asynchronous migration than in synchronous migration. Because this protocol needs NMP’s coordination but has nearly the same migration overhead as asynchronous, it is called quasi-asynchronous migration. A more detailed comparison between quasi-asynchronous migration and the other two will be given in the next section.
3. Implementation of Quasi-asynchronous Migration Utilizing the checkpointing tool of libckpt [7], we implemented a checkpointing library, which directly support uni-process migration. Based on it, the quasi-asynchronous migration protocol for PVM tasks was implemented in ChaRM system. There are two techniques underlying the implementation of the protocol. First, by providing wrappers for all PVM functions, the protocol is implemented completely outside PVM. Migration codes are inserted inside the wrappers, without changing the interfaces of PVM functions. Therefore, no changes are required to the system with new versions of PVM. This technique is also used to avoid reentering the PVM library [4]. Second, a tid mapping table is maintained in every task’s address space. When communicating, a task looks up the table first to find the current tid of the destination task. Hence, an application programmer does not have to know whether there was some migration or not. As was discussed in section 2, message m1 and m2 will not be lost if the tid mapping mechanism is used.
3
C-Manager [Signal handler of MIGRATE_SIG] { (1)receive MIGRATE_MSG (including the tids and destination machines of MPs)
MIG
RAT
E
. . .
MIG_CO
(3)send M IG_CHECK_SIG signal and MIG_CHECK_MSG (includinm g, n , the tids and destination machines of MPs)
(The number of NMPs isn )
(The number of MPs ism )
(2)send M IG_COORDINATE_SIG signal and MIG_COORDINATE_MSG (includingm , the tids of MPs)
stage1
NMP
MP G _MS
MIG_C
HECK
ORDINA
_MSG
TE_MSG
. . .
[Signal handler of MIG_COODINATE_SIG] { (1)receive MIG_COORDINATE_MSG from C-Manager;
[handler of MIG_CHECK_SIG] { (1)receive MIG_CHECK_MSG from C-Manager;
(2)send FLUSH_MSG to each MP;
(2)create a TCP socket and bind it to a port address;
}/*The end of signal handler */
SH_
FLU
(4)send FLUSH_MSG to each NMP and each MP except itself;
SOC
(4)wait to reiceive MPINFO_MSG from all MPs. When a MPINFO_MSG is received, spawns corresponding SP on destination machine and sends SOCK_MSG (Including socket address and the port number) to it.
(3)send MPINFO_MSG(Inlcuding socket address and the port number) to C-Manager;
MSG
MPINFO_MSG
K_M
SP (on the destination machine)
SG (1)receive SOCK_MSG from C-Manager;
(2)esteblish the connection with MP;
an e Tr
(3)read out the process state from the socket and recontruct the state in its own address space;
sfer
(7)capture its state , and write them into the socket; (8)terminate;
( if try to send application a message to a MP then save the message to delaying_buffer) FLUSH_MSG
(6)waits to accept the connection with SP;
stage2
Stat
. . .
(5) wait to receive (m+n-1) FLUSH_MSGs; if received an application message, saves it to flushing_buffer;
. . .
}
/*in the context of MP */
RJN_M
SG
(4)send RJN_MSG(Including its original tid and current tid) to C-Manager;
(5)wais for the RJN_MSG from all SPs and updates tid mapping table;
[Signal handler of MIG_COODINATE_SIG] { (1) wait to receive m FLUSH_MSGs; if received an application message, save it to flushing_buffer;
s tage3 (6) send MIG_COORDINATE_SIG signal to all NMPs the second time.
Control Messages TCP Transfer
G
/HJHQG
(2)wait to recieve TABLE_MSG from C-Manager and update its tid mapping table;
(5)wait to recieve TABLE_MSG from C-Manager and update its tid mapping table;
(3)send out all the messages in delaying_buffer according to the updated mapping table;
E_MS
} /* the end of the signal handler */
TABL
(7) send TABLE_MSG (Including the updated tid mapping table) to all MPs and NMPs
}/*The end of signal handler */
}/*The signal handler of MIG_CHECK_SIG */
. . .
. . .
Figure 2 Quasi-asynchronous migration for PVM tasks. It is supposed that there are m MPs and n NMPs during a migration.
4
The protocol of quasi-asynchronous migration is shown in figure 2. One of the key components of the protocol is a managing process called C-Manager who controls the migration. Another important component of the migration protocol is Control Messages and Control Signals. They are special system messages and signals in the library of ChaRM for the primary purpose of managing task migration. Both the messages and the signals are invisible to the application code. The migration protocol can be divided into three stages: Initialization, State transfer, and Restart. In the first stage, when C-Manager received the triggering signal MIGRATE_SIG and MIGRATE_MSG, it delivers a combination of a signal and a message to each MP and NMP respectively. Once a MP catches the MIG_CHECK_SIG signal, it creates a socket for state transfer and sends related information to the C-Manager via MPINFO_MSG message. After that, each MP sends FLUSH_MSG, a special message, to all tasks in the application except itself. Then it waits till the FLUSH_MSGs from all other tasks are received. Simultaneously, once a NMP catches the MIG_COORDINATE_SIG signal, it sends FLUSH_MSGs to all MPs. Then it exits the signal handler and resumes normal execution. During the normal execution before the end of migration, if a NMP tries to send application messages to a MP, the message will not be sent really but is saved into the delaying-buffer. Once the C-Manager receives the MPINFO_MSG from a MP, it spawns its corresponding SP on the destination machine and sends the information needed for state transfer to SP via SOCK_MSG. In the second stage, if a MP has received all the FLUSH_MSGs , a TCP connection between the MP and its corresponding SP is established, and the state transfer begins. While state transfer is proceeding, the SP simultaneously assimilates the state as its own. After the transfer, the MP terminates while the SP executes in the context of corresponding MP. In the last stage, each SP sends a RJN_MSG that contains its original and current tid to the CManager. Once all RJN_MSGs are received, C-Manager sends a MIG_COORDINATE _SIG signal to each NMP the second time. The updated tid mapping table is sent to each MP and each NMP via TABLE_MSG. After a SP receives the message and updates its own mapping table, it exits the signal handler and resumes its normal execution. Once a NMP catches the MIG_COORDINATE_SIG signal the second time, it waits to receive FLUSH_MSGs from all MPs. Then it sends out the delayed massages in the delaying-buffer according to the updated mapping table. After that, the NMP exits the signal handler and resumes its normal execution. The protocol ensures that application messages will not be lost and they will be received in the correct order. Follows are two key mechanisms to achieve this: Message Flushing At migration time, a special FLUSH_MSG message is sent between the MPs and the NMPs. Because PVM guarantees the FIFO semantics of message delivery, task A’s receipt of the FLUSH_MSG message from task B implies receipt of all messages previously sent by B to A. This means that the MP can collect all the messages sent to it before FLUSH_MSG into its address space, so that they become part of the task’s state and can be retrieved after restart from the wrapper functions. Hence, message flushing can avoid losing m4 and m5. The number of FLUSH_MSGs sent at migration time is in the order of O(m*n), where m is the number of the MPs and n is the number of the NMPs. In order to reduce the overhead introduced by this mechanism, NMP’s message flushing does not proceed in stage 1 like MP but in stage 3. The correctness of the protocol is not affected since m5 will not be lost before a NMP updates its mapping table. Because it is often true that by t1, most of the messages sent from all the MPs to a NMP have already received by the NMP, the overhead of message flushing is minimized. Message Delaying. In the wrapper of pvm_send() function, the application messages sent to the MPs by a NMP during migration time, like m3 in figure 1(b), are delayed. The messages are saved to the delaying-buffer and will be sent out to the SPs by the NMP at the end of migration, according to the updated tid mapping table. The overhead introduced by message delaying mechanism is low. On
5
the one hand, a NMP usually continues to execute after calling pvm_send() function and does not concern whether the receiving task has received the message or not. Therefore, message delaying has little explicit affection on a NMP’s running time. On the other hand, a MP does not have to wait a longer time for the delayed application messages from the NMPs. This is because following two reasons: First, since a MP has to spend much more time in MIG_CHECK_SIG signal handler than a NMP does in MIG_COORDINSTE_SIG signal handler, the execution of a NMP is usually ahead of the MP’s. As a result, we can expect that the time during which a MP waits for application messages from NMPs, if any, should be less than there is no migration. Second, the delayed messages are sent out by the NMPs as soon as the mapping table is updated, while the a SP either has not resumed execution yet or just did so. In comparison with synchronous and asynchronous migration, quasi-asynchronous has some advantages. On the one hand, the overhead of NMPs is much lower in quasi-asynchronous migration than in synchronous migration. This is because that NMPs’ overhead in synchronous migration is almost the same as MPs’, while NMPs’ overhead in quasi-asynchronous migration is much less than MPs’. On the other hand, quasi-asynchronous migration also has some advantages over asynchronous migration. It seems that asynchronous migration has lower overhead than quasi-asynchronous migration because there is no explicit participation of NMPs. However, in asynchronous migration, when a message cannot find its receiver because of migration, it has to be forwarded. As a result of this message forwarding mechanism, message sequencing has to be used in order to guarantee correct delivery of messages. This means that the overhead is averaged by every communication involved with migrated tasks. Moreover, if the communication between tasks is via DirectRoute, the NMPs that are communicating with the MPs are also involved. This increases the migration overhead, while quasi-asynchronous migration deals with DirectRoute and InDirectRoute in the same way. Therefore, there should be no significant difference between quasi-asynchronous and asynchronous migration in the measure of NMPs’ overhead. However, it is much easier to implement quasi-asynchronous migration than to implement asynchronous migration. In summary, quasi-asynchronous migration has the advantages of both synchronous migration and asynchronous migration: it is easy to implement and it has low overhead.
4. Performance Results Hundreds of tests have confirmed the viability and stability of the quasi-asynchronous migration protocol. We also performed some tests on migration overhead. Several measures are defined as follows: z MP’s freeze time, is the amount of time from the time when the task on the source host was told to migrate to the time when it restarts on the destination host. During freeze time, the task is executing inside the signal handler of MIG_CHECK_SIG, that is, outside application code. z The TCP transfer time is the actual transfer time of the task’s state through the network. z MP’s coordination time, is the time MP spends on other work for the migration than transferring process state. The MP’s freeze time is essentially the TCP transfer time plus MP’s coordination time. z NMP’s coordination time is the time during which a NMP executes in the signal handler of MIG_COORDINATE_SIG. Table 1 and figure 3 show the migration overhead as measured within the network of four SUN Ultra2 workstations running Solaris 2.5 connected over an idle 100 Mb/sec Ethernet. Each workstation has two processors and 128MB main memory. The application used was a matrix exponentiation program consisting of a master task and eight slave tasks, which is a modified version of matrix
6
multiply program. During execution of the program, a task is migrated from one host to another, and the measures defined above were tested. The process state size in table 1 indicates the actual number of bytes transferred through the TCP socket connection at migration time. NMP’s maximal coordination time in the table is the maximal of all the NMPs’ coordination time. MP’s freeze TCP transfer time MP’s coordination time NMP’s maximal time (sec) (sec) (sec) coordination time (msec) 0.418 0.219 0.189 2.0 0.438 0.253 0.185 1.7 0.584 0.368 0.196 2.6 0.661 0.472 0.189 2.5 0.759 0.565 0.194 3.5 0.938 0.684 0.254 5.8 1.039 0.819 0.220 7.5 1.120 0.916 0.204 2.6 Table 1: Migration overhead for various process state sizes. RYHUKHDG VHF
Process state size (Mbytes) 1.68 2.00 3.01 4.00 5.00 6.01 7.01 8.02
IUHH]H WLPH 7&3 WUDQVIHU WLPH &RRUGLQDWLRQ WLPH
3URFHVV VWDWH VL]H 0%
Figure 3 Graph of Table 3. This graph more clearly shows the direct relationship between MP’s the state size, the TCP transfer time, the freeze time, and the coordination time.
As can be seen from the table, the process state size is the dominating factor in the migration overhead. As the process state size increases, the TCP transfer time, which is time consuming and dominates the freeze time, increases proportionally. This also shows that migration performance is limited by the bandwidth of the underlying network. However, MP’s coordination time for various state sizes does not show this relationship with process state, although they show an increasing trend when state size increases. This is because that a MP’s coordination involves little with MP’s state size. These relationships can be seen more clearly in Figure 3. NMP’s coordination time shown in Table 3 is 100 times less than MP’s freeze time. This confirms our analysis of quasi-asynchronous migration: NMP’s overhead is much lower than in synchronous migration. The advantage of quasi-asynchronous protocol over synchronous protocol is especially obvious when the state transfer time is prolonged because of busy network traffic or the increase of the state size. Furthermore, the result also shows that it is worth spending a few milliseconds on NMPs’ coordination in quasi-asynchronous migration in order to avoid the complicated implementation of asynchronous migration. The results of the tests on other applications are shown in Table 2. Embar, DZZ are computationintensive application, while ASE, Fractal, FFT are communication-intensive application. As can be seen from the table, there are no significant differences between computationally intensive and communicationally intensive applications in the measure of MP’s coordination time. This shows that the overhead introduced by the message flushing and message delaying mechanism are not significant.
7
NMP’s coordination time is a little greater in communicationally intensive applications than in computationally intensive applications because NMPs have to buffer and sent out more messages. Application State size (Mbytes) Embar 1.6 DZZ 2.2 Fractal 1.7 ASE 8.0 FFT 3.7 FFT 22.1 FFT 66.1
MP’s freeze TCP transfer time MP’s coordination NMP’s maximal time (sec) (sec) time (sec) coordination time (msec) 0.438 0.212 0.215 2.0 0.471 0.273 0.196 3.1 0.437 0.225 0.204 3.3 1.088 0.864 0.223 1.9 0.445 0.679 0.234 4.4 2.232 2.445 0.211 6.2 7.348 7.687 0.249 7.6 Table 2 The migration overhead for various applications
Despite the overhead of migration, the ability to migrate could allow PVM applications access to machines they could not have used otherwise, and could lead to large gains in overall performance. In order to demonstrate the advantages of task migration, some tests were performed. Twenty seconds after the matrix exponentiation program began executing, another unrelated program was launched on the fourth workstation. It took the application 176 seconds to complete execution. The experiment was repeated, except this time the task was migrated from the fourth workstation to an idle workstation. The run time dropped to 102 seconds. By dynamically balancing the load, the overall run time is minimized.
5. Related work Process migration implementations can be broadly categorized as either supported at the systemlevel or at the user-level. Chalotte, V system, Mosix, Sprite and Mach support process migration at system-level with the involvement of OS kernel. System-level implementation has the advantages of high efficiency, transparency, but has poor portability. An alternative method is user-level implementation, entirely at user-level using facilities available through standard UNIX system calls and libraries. The advantage of such an implementation is that there is no need for kernel modification, making it portable to various UNIX flavors. ChaRM[2], MPVM[4], Condor[5,6], Dynamic PVM[8] , CoCheck[9], DPVM[10] fits into this category. ChaRM was first implemented on SUN Sparc20 running Solaris 2.4. It has since been ported onto Solaris 2.5, Linux, and AIX 3.2 MPVM is an enhanced PVM. It modified the source code of PVM and implemented asynchronous migration protocol. The drawback of this protocol is the need to introduce message forwarding and message sequencing to ensure the correct behavior of a PVM application, the enormous effort which is necessary to incorporate these changes in the current version of PVM, and the additional executing overhead in normal case after migration. In contrast to MPVM, ChaRM and CoCheck are both implemented completely outside PVM. This has been achieved by providing wrappers for all PVM function calls. Therefore, no change is required to the system with new versions of PVM. Furthermore, ChaRM has a specific manager process C-Manager that controls the entire system. It is has more portability to various message passing systems than selecting Resource Manager of PVM as the controlling process, like what CoCheck does. Condor uses a checkpoint/rollback mechanism to achieve migration. It saves the checkpoint of the process onto disks and suspends the process. Once an idle node is found, the suspended process will be reconstructed on the idle node. Using the uni-process checkpointing and recovery tools of Condor, CoCheck implemented synchronous migration, with high migration overhead. Similarly, DPVM [9] also use the tools of Condor. In order to support process migration, DPVM requires that
8
every message must first pass through a managing process existing on every node. Since messages have to pass through both the managing process and PVM daemon, additional overhead is added on normal case. ChaRM achieved an implementation much simpler than MPVM and much efficient than CoCheck with quasi-asynchronous migration. In contrast to Condor and DPVM, ChaRM and CoCheck have the additional advantage of requiring almost no disk I/O. In everyone of above systems, migration is transparent to the application programmer as far as the PVM interface is concerned, no matter whether source code of PVM was modified or not.
6. Conclusion and Future Work ChaRM environment comprises of a set of run-time libraries. PVM applications that should benefit from ChaRM’s features simply need to be recompiled and re-linked with the ChaRM libraries. ChaRM has an enhanced console of PVM to enable the users to issue manual or semi-automatic migration request. That is, user can explicitly specify both the tasks to be migrated and the destination nodes with a special command; or he can let the system to find the destination nodes. In addition to the supplied control command, ChaRM offers an API with which the users can request migration within their applications. Hence, the user is capable of integrating dynamic load balancing into his applications if he likes. Fully automatic migration, with which ChaRM monitors the entire system and makes migration decision, is under development now. There are three important features of ChaRM’s migration. First, it is transparent to application programmer. Second, the quasi-asynchronous migration protocol has good portability. It can be implemented on top of other massage-passing systems, like MPI, as well as PVM. Third, by introducing message flushing, message delaying mechanism, quasi-asynchronous migration attains the advantages of both low overhead and simple implementation. We have already implemented the checkpointing and rollback recovery mechanism in ChaRM system in order to provide the feature of fault tolerance for PVM application. A dynamic resource scheduler is under development in order to manage the resource of the Networks of Workstations more efficiently and intelligently. Process migration for MPI tasks will also be supported in the near future. References [1] V.S. Sunderam, G. A. Geist, J. Dongarra, and R. Manchek. The PVM concrrent computing system: Evolution, experiences and trends. Parallel Computing, 20(4):531-545, April 1994 [2] Wang Dongsheng, Zheng Weimin, Pei Dan. ChaRM: a Checkpoint-based Rollback Recovery and Migration System. Journal of Software, January,1999 [3] M.Rasit Eskicioglu. Design Issues of Process Migration Facilities in Distributed Systems. IEEE Technical Committee on Operating Systems Newsletter, 1989,4(2):3~13 [4] Jeremy Casas, Dan L. Clark, et al. MPVM: A Migration Transparent Version of PVM. Computing Systems, 1995,8(2):171~216 A Hunter of Idle Workstations. In: Proc. of the 8th IEEE Intl. Conf. on [5] J. Litzkow, et al. Condor Distributed Computing Systems.Los Alamitos,CA:IEEE CS Press,1988.104-111 [6] T. Tannenbaum and M. Litzkow. The Condor Distributed Processing System. Dr. Dobb’s Journal, 1995, (2):40~48 [7] James S. Plank, Micah Beck, Gerry Kingsley, and Kai Li. Libckpt: Transparent checkpointing under Unix. In Proceedings of the 1995 USENIX Technical Conference, pages 213-224, January 1995. [8] L. Dikken, F.van der Linden, J.Vesseur, and P. Sloot. DynamicPVM-Dynamic Load Balancing on Parallel Systems. In: W. Gentzsch and U.Harms, editors, High-performance Computin and Networking, International Coference and Exhibition Volume II: Networking and Tools, volume 797 of Lecture Notes in Computer Science, pages 273-277. Springer Verlag, Berlin, April 1994.
9
[9] Stellner and J. Pruyne. Resource Management and Checkpointing for PVM. In: Proc. 2nd Europe PVM User Group Meeting. Lyon :Editions Hermes,1995:31-136 [10] Ju Jiubin,Wei Xiaohui,Xu Gaochao et al. DPVM: an enhanced PVM supporting task migration and queuing, Chinese Journal of Computers, 1997,20(10):872~877
10
