Software Process Decision Support: Making Process ... - CiteSeerX

4 downloads 109957 Views 62KB Size Report
to support the software project management control function. This forward-looking ..... green or blue is considered to be good performance by the client. For this ...
Software Process Decision Support: Making Process Tradeoffs Using a Hybrid Metrics, Modeling and Utility Framework David M. Raffo

Warren Harrison

Joseph Vandeville

School of Business Administration Portland State University Portland, OR 97207-0751 +001-503-725-8508

Department of Computer Science Portland State University Portland, OR 97207-0751 +001-503-725-3108

Northrop Grumman Corporation Integrated Systems Sector Melbourne, Florida

[email protected]

[email protected]

ABSTRACT In this paper, we present a “forward-looking” decision support framework that integrates timely metrics data with simulation models of the software development process in order to support the software project management control function. This forward-looking approach provides predictions of project performance and the impact of various management decisions. Tradeoffs among performance measures are accomplished using Outcome Based Control Limits (OBCLs) and are augmented using multi-criteria utility functions and financial measures of performance to evaluate various process alternatives. The decision support framework enables the program manager to take corrective action as necessary on a project with the simulation model providing insight on potential performance impacts of the proposed corrective actions. A real world example is presented.

Categories and Subject Descriptors D.2.9 [Software Engineering]: Management – Software process models, Cost estimation, Productivity.

General Terms Economics, Management, and Measurement

Keywords Software Process Modeling, Project Management, Simulation, Software Measurement Repositories, Multi-criteria decision making

1. INTRODUCTION In recent work, Raffo developed the Process Tradeoff Analysis (PTA) Method. This method builds on previous work by Kellner et al. at the Software Engineering Institute (SEI) Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies

are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SEKE '02, July 15-19, 2002, Ischia, Italy. Copyright 2002 ACM 1-58113-556-4/02/0700...$5.00.

[email protected]

[3, 4, 5] by developing a quantitative approach to evaluating potential process changes in terms of development cost, product quality, and project schedule [7]. The core of the PTA method addresses evaluating process alternatives quantitatively by developing stochastic simulation models of each process alternative. These models explicitly capture process-level details including complex interdependencies among process components. The PTA Method has been applied to real-world process change problems at leading software development firms. [7, 8, 9]. This work has predominately been applied to the software project management planning function [7; 9]. The goal of our current research is to develop a “forwardlooking” approach that integrates timely metrics data with simulation models of the software development process in order to support the software project management control function. This forward-looking approach provides predictions of project performance and the impact of various management decisions. The concept for this framework was first reported in [10] and focused on the novel role played by the flexible metrics repository. In this paper, we focus on the overall decision framework using OBCLs, utility functions and financial performance measures which is new. In addition, in the previous work, a numerical example was not shown. Tradeoffs among performance measures are accomplished using Outcome Based Control Limits which are augmented by multicriteria utility functions and financial measures of performance (e.g. net present value, payback period, etc.) of various process alternatives as compared with a baseline. By combining metrics and predictive models, a more comprehensive performance picture of the project can be achieved than by using metrics alone. Moreover, the predictive models can support managers as they attempt to re-plan and make process tradeoffs as they work to bring a project back on track. In terms of supporting decisions, the OBCLs give managers a visual gauge as to how the project is tracking to its goals along multiple measures of performance. Using the multi-criteria utility functions and/or financial performance measures enable the manager to tradeoff among several alternatives.

- SEKE '02 - 803 -

Table 1 - Model Estimates of Project Performance Mean

Std

Total Effort (Cost) in Person Months

418.5

5.54

Project Duration (Schedule) in Months

26.2

2.03

Number of Remaining Defects (Quality)

77.4

3.68

Table 2 - Project Performance Targets and Limits Target

Blue Limits

Green Limits

Yellow Limits

Red Zone

(From table 1)

Upper

Lowe

Upper

Lower

Upper

Lower

(+5%)

(-5%)r

(+15%)

(-15%)

(+30%)

(-30%)

Total Effort (Cost) in Person Months

418.5

439.4

397.6

481.3

355.7

544.0

292.9

544.0

292.9

Project Duration (Schedule) in Months

26. 2

27.5

24.9

30.2

22.3

34.1

18.4

34.1

18.4

Number of Remaining Defects (Quality)

77.4

81.2

73.5

89.0

65.8

100.6

54.2

100.6

54.2

In this paper, we present an illustrative example using the combined simulation model and metrics repository decision support framework. The model described in this paper has been applied at a leading software development firm. The data have been masked to maintain company confidentiality requirements. The purpose of this example is to show how updated information can be gathered through a metrics repository and then used to support quantitative management of the software development process as required by the Quantitative Process Management and Software Quality Management Level 4 Key Process Areas (KPAs) of the Capability Maturity Model (CMM) [2, 6] and to make tradeoffs among process options.

2. THE SCENARIO UNDER CONSIDERATION The software development firm has been contracted to provide a major enhancement (approximately 32 KLOC) to an existing system. The enhancement consists of modifying six Computer Software Configuration Items (CSCIs) with modifications to each CSCI ranging between 4 and 8 thousand lines of additional code (KLOC). A software process model (as described below) has been used to estimate the three performance measures of cost, schedule and quality performance for the proposed project using past project data augmented with updated estimates for productivity, efficiency, and project size among other performance measures for the proposed project. These estimates were used in the bid process. The client organization has now accepted the bid. As a result, the estimates have now become the contracted project performance levels by which the project will be evaluated.

Above

Below

(>+30%) (