Charles McCabe, San Francisco Chronicle. Jim Haynes. Computer Center, UC Santa Cruz. CA 95065; Bitnet, haynes@ucscc;. Arpa. haynes@ucscc ucsc.edu.
“Any clod can have the facts, but having opinions is an art!’ Charles McCabe, San Francisco Chronicle
Jim Haynes. Computer Center, UC Santa Cruz. CA 95065; Bitnet, haynes@ucscc; Arpa. haynes@ucscc ucsc.edu
Three-dimensional computer performance Measuring computer performance by focusing on CPU performance expressed in MIPS (millions of instructions per second) has led to numerous advances, such as multiprocessing, parallel processing, and array and vector processing. However, performance of general-purpose computers depends by definition on the applications they are attempting to solve. No general-purpose computer will perform at the same level for all applications. Therefore, CPU performance alone is an inadequate measure of computer performance, especially in the arena of extensive 1 / 0 processing such as real-time computing. We need a three-dimensional measure of overall computer-response performance. A computer architecture that relies heavily on CPU performance needs to balance its internal design to satisfy the fast response of the overall computer system. Hypercubes, for example, which have powerful message-passing array-processing units linked in a cube with no common memory, will disintegrate rapidly to an external interrupt to the computer. Therefore, CPU performance must become one of three measures-or dimensions-of system performance. The two other dimensions are high 1 / 0 throughput and high interrupt-handling capability. We define a 3D computer as a system capable of receiving inputs from an external real-world process, performing the required data processing, and outputting the response back to the process fast enough to meet process requirements. In addition, a 3D computer must be able to respond to external interrupts generated by 1 / 0 devices and other systems. Therefore, a 3D computer can be characterized by three key features: (1) sufficient speed of CPU computation (2) an efficient interrupt-handling capability (3) high 1 / 0 throughput July 1988
M3 measure The performance of general-purpose computers is typically measured in terms of MIPS or Mflops (millions of floating-point operations per second). However, a 3D computer may have high CPU performance in MIPS, but poor interrupt-service capability and low 1 / 0 throughput. Therefore, we propose a measure that better suits 3D computers and takes into account all three features mentioned above. If we measure CPU speed in MIPS-1, interrupt-handling capability in MIPS-2 (millions of interrupts per second), and 1 / 0 throughput in MIPS-3 (millions of 1 / 0 per second), then the proposed measure that incor orates all three features will be MIPS or M3. The range of values for M3 in advanced 3D computers is CPU speed: 2-60 MIPS-1 interrupt-handling capability: .01-.2 MIPS-2, and 1 / 0 throughput: 1-10 MIPS-3.
P
These three features are not independent and should be analyzed together. For example, if a 3D computer has the following as its maximum features: CPU speed: 5 MIPS-1 interrupt-handling capability: 0.1 MIPS-2, and
1 / 0 throughput: 5 MIPS-3 it does not mean that these three features can be achieved simultaneously. In fact, increasing one feature automatically degrades the other two. Therefore, to accomplish a 3D architecture and provide a faster overall system response, all M3 components need to improve. Let’s ignore 1/0 throughput for a moment and analyze the relationship between CPU speed and interrupthandling capability. This analysis is possible because most modern 3D computers have separate processors to handle 1 / 0 operations, so 1 / 0 throughput depends very little on the other two features. Figure 1 shows a typical interdependence between CPU speed and interrupt-handling capability for a 3D and a non-3D computer. Note that CPU power decreases in both cases when interrupt loading increases. However, the performance of a non-3D computer degrades much faster than that of a 3D computer. Assuming a conventional computer in which a single CPU performs application programs and I 0 processing, we can represent the M measure as a plane in a three-dimensional space, as shown in Figure 2.
I
h
U
al al
Q v)
3
a
2 7
0
E 5
MIPS-2 (Interrupt handling capability) Figure 1. Typical interdependence between CPU speed and interrupt-handling capability for a 3D and a n o n 3 D computer. 59
Benchmarks
S20.000
In a single-task environment, standard benchmark programs such as Whetstone, Dhrystone, and Linpack typically measure just the CPU speed. T o measure the system response and performance of 3D computers, new types of benchmarks should be developed. We have developed the M3 benchmark, which simulates a 3D environment and measures CPU speed as a function of interrupt loading. It consists of a combination of hardware and software elements: (1) a pulse generator to generate interrupts at user-selectable frequencies, (2) a user-coded interrupt-handler routine, and (3) the Whetstone benchmark. The M3 benchmark was run for various interrupt-loading factors between zero and 100,000 interrupts per second on two Modcomp Classic CPUs: the CL 11/75 and the CL 32/85. We used the MAX 1V and MAX-32 real-time operating systems, respectively. Figure 3 shows the relative system performance. In addition to relative performance, two other measures are useful for 3D computers: program overhead and system overhead. Program overhead ( P )is a measure that defines how much time it takes to complete an application program at a specific interrupt-loading compared to the time it takes to complete it at no interrupt-loading :
p = -tn - to t0
=
1.6
100
=
37.5%
The obtained results for S show that the interrupt-handler spends 37.5 percent of the total elapsed time servicing interrupts, and the rest dedicated to the application program. When measuring the performance of 3D computers, we can calculate the program and system overhead for different interrupt-loading. Note that S can be larger than 100 percent, in which case the saturation point has been reached. While the M3 measurements reflect the three-dimensional nature of the computer, a simplified measure, called “big M,” can represent a single factor combining MIPS-1, MIPS-2, and MIPS-3. First, we can transform the three-dimensional measure from Figure 2 into a two-dimensional measure, showing a vertical axis for the CPU
speed in MIPS-1 and a horizontal axis for task/seconds (combining interrupt capability and 1 / 0 throughput). We then define “big M” as an equivalent MIPS that combines MIPS-], MIPS-2, and MIPS-3. As advanced 3D computers become more complex, with multiprocessing, parallel processing, array processing, etc., it is important to develop a good, novel 3D architecture and a realistic methodology to evaluate and objectively measure M3 of future 3D computers. A future issue of Computer Architecture News will describe and show results from a novel computer architecture using the 3D concept. Guy Rabbat, Borko Furht, Ronald Kibler Modcomp/AEG, Ft. Lauderdale, Fla.
MIPS-1 (CPU speed)
/bx
Non-3D computer
3D computer
Figure 2. The M3 measure, represented as a plane in a three-dimensional space, for a 3D computer and a non-3D computer.
t
MIPS-1
1.6- 1 - 100 = 60% 1
In other words, this benchmark took 60 percent longer to complete under the specified interrupt-loading. System overhead ( S ) defines what percentage of the total elapsed time is devoted to handling interrupts:
Using the same values as in the previous example, S becomes
60
~
100 [%I
where to is the execution time under no interrupt-loading, and tn is the execution time at a particular interruptloading. Suppose that a benchmark program completes in one second at an interruptloading of zero, and in 1.6 seconds at an interrupt-loading of 20,000 interrupts per second. The program overhead for this interrupt loading becomes P20,Ooo
1.6- 1 =
‘.’MIP;
MIPS-3
Figure 3. Relative system performance of two Modcornp Classic CPUs running the M3 benchmark for various interrupt-loading factors between zero and 100,OOOinterrupts per second. COMPUTER
