Using MCDA and Bayesian Belief Networks to Aid Decision Making Based on Software Project Management Antipatterns Dimitrios Settas, Ioannis Stamelos Dept. of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece {dsettas,stamelos}@csd.auth.gr
Vassilis C. Gerogiannis Dept. of Project Management, Technological Education Institute of Larissa, Larissa, Greece
[email protected]
Abstract Antipatterns in software project management are mechanisms that describe commonly occurring solutions to problems that generate negative consequences. The Bayesian Belief Network (BBN) approach has been recently proposed for modelling software project management antipatterns. BBNs provide a solution for software project managers, who would like to model the cause-effect relationship that underlies antipatterns by considering their inherent uncertainty. Although a BBN model can be used to aid decisions by observing the effect of uncertainty on a specific attribute of interest (such as software quality), in many circumstances decisions based on multiple criteria need to be made. Multi-criteria decision aid (MCDA) can be applied to improve the effectiveness of management decisions. A BBN model ignores the multiple, and often conflicting, criteria that surround a managerial decision (such as development time and total collaboration costs). In this paper such an approach is exemplified through a specific BBN model of an antipattern complemented by MCDA. The final resulted model is helpful in evaluating the impact of developer personalities and temperaments on communication, collaboration viability and effectiveness in pair programming. Keywords: Bayesian Belief Netowrks, Multi-criteria decision aid, Software project management, Antipatterns
1. Introduction Software project management plays a major role in software development that is required for the coordination and guidance of analysts, programmers and testers. Ineffective project management has been recognised as a major factor contributing to software project failures [Fox and Wayne (2005)]. Success or failure of a software project can be attributed to the incorrect handling of one or more project variables: people, technology and process [Hughes and Cotterell (1999); Brown et. al. (2000)].
There exist many uncertainties in these variables and current software engineering techniques are unable to eliminate them [Fan and Yu (2004)]. Managing variables uncertainty is very important because this would lower the risk of the project being unsuccessful. It is highly recommended [Hughes and Cotterell (1999)] that software project managers should reconsider the techniques they used on successful past project, before making decisions on current projects. Furthermore, software project managers should avoid using techniques they applied on unsuccessful past projects. Antipatterns [Brown et. al. (1998); Laplante and Neill (2006)] are particularly well suited in such cases by describing commonly occurring solutions to problems regarding dysfunctional behaviour of managers or pervasive management practices that inhibit software project success. These mechanisms can manage aspects of a software project more effectively by bringing insight into the causes, symptoms, consequences, and by providing successful repeatable solutions. An antipattern is related with two solutions. The first is a solution with negative consequences and the other is a refactored solution, which describes how to change the antipattern into a “healthy” solution [Brown et. al. (1998)]. By listing project management antipattern catalogues, project managers can identify potential problems and provide a refactored solution, in a practical and reusable manner [Belbin (1996)]. A project management antipattern can be regarded as the result of incorrect decisions/actions, which are made on the part of the manager, because of the lack of knowledge or experience in solving a particular problem. These decisions could also be made because a manager simply made a decision based on faulty information. Therefore, decision makers can decide to use the refactored solution of an antipattern, which describes how to change the antipattern into a correct solution. A software project manager may utilise machine learning models to observe the effects of uncertainty on a specific project variable and make a decision to use an appropriate refactored solution. For example, a project manager can build a Bayesian Belief Network (BBN) model of an antipattern to observe the effect of uncertainty on software quality [Settas et. al. (2006)]. However, this approach handles only one part of the information that the manager may exploit before deciding to use a refactored solution of the identified antipattern. A software project manager may be also interested in other criteria, like cost and functionality, which can not be provided by the BBN model of the antipattern [Fenton et. al. (2004)]. In this paper, we use antipatterns to overcome the problems related with possible dysfunctional behaviour of management practices that inhibit software project success. To take into account wider decision problems, based on multiple criteria, the role of BBNs is complemented with a multi-criteria decision aid (MCDA) approach. We exemplify this approach using an already defined antipattern that is called “shaken but not stirred” [Settas et. al. (2006)]. This is a software project management antipattern, expessed in a BBN model, that evaluates the impact of programmers’
personalities and temperaments on communication and collaboration. We chose this approach to demonstrate the power of antipatterns modelled through BBNs and MCDA. The paper is divided in 4 sections, which are organized as follows: section 2 describes the background, the related work and the literature review used in our research. Section 3 describes the proposed approach which unifies BBNs and MCDA for antipattern modelling. Finally, in section 4 we summarize our findings and draw our conclusions.
2. Background and Related Work 2.1 Background In contrast to conventional software project management approaches, agile approaches put emphasis on a process view of human collaboration through the lifecycle of a software development project [Larman (2004)]. A traditional, phasebased approach identifies a sequence of steps to be followed, whereas an agile approach considers a software project as a series of relatively small tasks conceived and executed in an adaptive manner. Extreme Programming (XP) [Beck (1999)] is arguably the best known and most popular agile development methodology. XP emphasizes team work by motivating the software development process to constantly respond to changes of user requirements, even late in the project lifecycle. In an XP project, managers, users, and developers are all part of a coherent team dedicated to deliver quality software. Software development processes rely heavily on a communications effort, and improving communications is one of the most effective ways to improve the development process. “Pair programming” is possibly the most controversial aspect of XP [Stephens and Rosenberg (2003)]. According to the pair programming style, every line of code should be created by a pair of developers, sitting side by side. Use of patterns and antipatterns may be proved very applicable in an XP project [Laplante and Neill (2006)]. The former refer to best practice solutions, while the latter refer to common mistakes a software development team may make as they try to respond to user requirements. Antipattern catalogues, have covered several management, personality and environmental antipatterns [Brown et. al. (2000)]. By documenting XP antipatterns, project coaches and managers are able to solve a newly encountered problem by applying a previously successful solution to a particular problem. In a pair programming practice, the aim is to transfer skills and knowledge between the two developers. However, there exist underlying human factors that need to be identified and understood, in order to control, predict and understand the problems that arise from the limited guidance that has been so far provided for specific
occasions. Therefore, an active community has been developed around XP in order to address such issues using antipatterns [Kuranuki and Hiranabe (2004)].
2.2 Related Work Bayesian Belief Networks (BBNs) have been recently proposed for modelling software project management antipattern uncertainty [Settas et. al. (2006)]. A BBN provides a natural, logical and probabilistic framework to depict project management antipattern uncertainty. The benefit of modelling a project management antipattern into a BBN is to bring the full weight and power of BBN analysis to bear on the problem of managing uncertainty in project management antipatterns. Furthermore, three considerations justify the decision of modelling antipatterns with the BBNs [Settas et. al. (2006)]: • The formalism of BBNs enables the development of a precise mathematical model of an antipattern. Project management antipatterns are effective in dealing with uncertainty and BBNs enable measuring and handling this uncertainty in mathematical terms. • The suggested model can be used by project managers to graphically depict and illustrate the effects of uncertainty on a project management antipattern. • The proposed BBN model can be used by project managers who have scarce statistical data of present or past projects and they wish to supplement the model using expert judgement. Multi-Criteria Decision Aid (MCDA) involves the selection of the best actions from a set of alternatives, each of which is evaluated against multiple, and often conflicting, criteria. Most of the existing MCDA methods only focus on decisions under certainty [Figueira et. al. (2005)]. However, Fenton and Neil [Fenton and Neil (2001)] have proposed a framework which complements BBNs using MCDA. The procedure they proposed consists of identifying objectives and perspectives for a decision problem and stakeholders, which leads to a set of actions/decisions, criteria and constraints. The proposed BBN is used to model a set of criteria and calculate a value for each criterion for a given action/decision. Using this method, a manager can apply traditional MCDA techniques to combine the values for a given action/decision and then rank the set of all possible actions. Furthermore, Watthayu and Peng [Watthayu and Peng (2004)] have proposed a decision framework using BBNs in order to model complex and uncertain interactions between criteria and other factors in a coherent and systematic manner. In this framework, a decision problem is represented by an Influence Diagram (ID), where each decision node represents a set of alternatives for a decision, a utility node represents a set of objectives (decision maker’s preferences), and chance nodes represent decision criteria and internal/external factors that may affect the decision criteria.
3. Complementing BBN Models of Antipatterns using MCDA 3.1 Software Project Management Antipatterns Brown et al. [Brown et. al. (1998); Brown et. al. (2000)] were the first to provide the structure of an antipattern. Following the informal presentation style of project management antipatterns, Laplante’s template [Laplante et. al. (20006] (Table 1) will be used to define an XP project management antipattern that is called “shaken but not stirred”. Table 1. Antipattern Structure A short name that conveys the antipattern’s meaning. Name CentralConcept The short synopsis of the antipattern in order to make the antipattern identifiable. The problems with the current practice. Dysfunction The expanded explanation including causes and consequences. Explanation A short term coping strategy for those who don’t have the Band-Aid influence nor time to refactor it. The first step for someone perpetuating the antipattern. Self-Repair The required changes in order to remedy the situation and their Refactoring rationale. An assessment instrument consisting of a list of questions for Identification diagnosis of the antipattern. The antipattern is defined as follows: Name: “shaken but not stirred”. Central Concept: A rush to use pair programming without ensuring that the developer pairs have mixed developer personalities and temperaments. Generally, any project manager or coach who does not use personality and temperament inventories. Dysfunction: This antipattern is attributable to a single individual. The project manager or the coach, who does not arrange the pairs according to mixed personalities, but usually cares only in having experienced programmers mixed with inexperienced ones. Explanation: We use the term “shaken but not stirred” because it reflects the fact that developers are only “shaken” in pairs according to their skills but they are not “stirred” according to their personality and temperament. The main causes are the lack of experience and knowledge of the project manager on mixing developer personalities. This clearly has a negative effect on knowledge and skill sharing hence affecting the development time, communication, collaboration, design correctness and the quality of the software.
Band-Aid: There is no band aid for this antipattern. No short term coping strategy can be applied to deal with this problem. Rearranging the pairs again, without using personality tests, will only make the situation more dysfunctional. Self-Repair: Project managers should mix the developer pairs accordingly as soon as possible. Therefore, they have to quickly identify symptoms such as poor communication and collaboration of the pairs and pay attention to the possible complaints of the developers. Refactoring: Personality and temperament sorter tests, such as the Keirsey Temperament Sorter test (http://www.keirsey.com/), can be used in order to identify and interpret developers’ personalities and temperaments. Pairs should be rearranged immediately, according to their personality and temperament, in order to improve the software quality by improving pair communication, collaboration and pair effectiveness. Identification: The following questions should be answered with a “Yes” or “No”. Does the XP team consistently deliver low quality software? Is there evidence of lack of shared knowledge of the project design? Have the developers reported any communication/collaboration problems to the project coach? Do the developers insist on working individually? If you responded “Yes” to one or more of these statements, your organisation is probably suffering from “shaken but not stirred”.
3.2 The BBN Model of the Antipattern Software project management antipatterns have been recently modelled using BBNs [[Fenton et. al. (2004); Settas et. al. (2006)]. The proposed model of the “shaken but not stirred” antipattern is illustrated in Fig. 1. Table 2 explains the variable names that were used in the BBN model.
Figure 1. BBN Model of the “Shaken but not Stirred” Antipattern
Table 2. Explanation of BBN Model Variables BBN Variable name Mixed Personality: An indicator of heterogeneous developer personality Mixed Temperament: An indicator of heterogeneous developer temperament according to Keirsey’s (http://www.keirsey.com/) descriptive groupings. Personality Type: 16 Personality types according to the Myers-Briggs (www.myersbriggs.org) type indication. Development Time: The total development time that took the pair to complete 2 tasks (measured in minutes). Total Communication Transactions: The total number of transactions in requirements gathering, specification and design changes, coding, unit testing, code and design reviews. Collaboration: The collaboration grade with the partner developer (points) Design Correctness: Measured in points obtained by each pair for both assignments. The points were measured on a ratio scale based on checklists to ensure objective assessment of the participant’s results Quality: This variable used the acceptance test scores of both tasks as an indicator.
3.3 A Decision Making Example An agile software project manager, who follows XP guidelines in his/her project, would like to rearrange the developer pairs in the best possible way. The manager wishes to improve software quality. Obviously, communication and collaboration of the developers have to be improved and development time must be reduced. The design also has to be of high correctness as well as conflicts between developers, their motivation and possible absence and other political or cost factors must be taken into account. The problem is to decide whether to: • Rearrange developer pairs heterogeneously according to their skills (i.e., advanced developers mixed with novice developers). • Rearrange developer pairs using programmers of roughly the same ability [Williams and Kessler (2002)]. • Apply the “shaken but not stirred” antipattern [Settas et. al. (2006)] that proposed to rearrange developers heterogeneously according to developer personalities and temperaments [Sfetsos et. al. (2005)].
3.4 MCDA and BBNs for Antipattern Modelling According to Fenton and Neil [Fenton and Neil (2001)], the objective of a decision problem is the ultimate reason for solving it. The objective is always with respect to a specific perspective, the most important component of which is the person or team
making the decision. The perspective of the problem incorporates not only the decision-maker, but also the stakeholders. These are the parties most affected by the chosen outcome and whose viewpoints will need to be considered in arriving at a decision. It is important not to take into account irrelevant stakeholders, if their viewpoints are irrelevant or their viewpoints are inconsistent with that of the decision maker. In an XP software project, for example, the end user of the developed software product is affected by the decision to rearrange the pairs of developers but there is no need for the manager to consider the needs of the end users. Three concepts usually play a fundamental role for analyzing and structuring the MCDA process, in close connection with the decision process itself [Figueira et. al. (2005)]. These are the possible actions we can take, a set of criteria and a set of constraints. In case of a single criterion, the best action is chosen. However, in case that several conflicting criteria need to be taken into account, we need to optimise the conflicting criteria. Constraints can guide decision makers to a decision and are the properties of the attributes that we regard as necessary. Constraints provide some help on choosing an action because any action that fails to satisfy a constraint for any criteria is rejected. To summarize the above MCDA key concepts in the “shaken but not stirred” antipattern: Objective: Rearrange the developer pairs in the best possible way. Perspective: Decision Maker: Software project manager. Stakeholders: The developers. Decision Problem: developer pairs.
Determine
the
most
suitable
rearrangement
of
Possible actions: Rearrange developer pairs heterogeneously according to their skills. Rearrange developer pairs using programmers of roughly the same ability. Apply the “Shaken but not stirred” antipattern. Criteria: Development time, total communicating transactions, collaboration, design correctness, quality. Constraints: Reduced development time, improved communication, collaboration and design correctness, increased software quality. External (uncontrollable) factors: Motivation, developers’ conflicts, absence, cost. Internal (controllable) factors: Development environment, project management involvement in the development process, personal communication with developers. However, when such a MCDA process is applied to a software project, the relevant criteria are not well defined and certain [Fenton and Neil (2001)]. BBNs can deal with the inherent uncertainty of the so called “vague” criteria. A vague criterion is often
decomposed into lower levels that are better defined. For example, software dependability is decomposed into safety, reliability, availability, and maintainability. Furthermore, in MCDA, once the criteria are defined it is assumed that for a given action their value is certain. However, some criteria require some uncertainty inference because they can not be computed with certainty. There are different types of factors that cause the need for uncertainty inference, such as external and internal factors. External factors cannot be controlled but influence the value of a criterion for a given action. All these factors together with the uncertain criteria may form nodes in a BBN which can be used to predict the values of uncertain criteria. For example, the finally resulting BBN model (Fig. 2) illustrates uncertain criteria with the external factors that impact on them.
Figure 2. BBN Model of the Impact of External Factors Internal factors, on the other hand, can be controlled and can influence the value of a criterion for a given action. In the construction of BBN models for uncertain criteria the internal factors also have to be included. An example of an internal factor that could be included in the BBN model (Fig. 2) is the amount of managerial involvement. Therefore, we could increase the involvement of the manager in case evidence of decreased motivation is received. Finally, the resulting BBN model can be used to calculate values of the uncertain criteria, combine values for a given action and then rank the set of possible actions.
4. Conclusion In this paper we have extended BBN models of software project management antipatterns by taking into account wider decision problems based on multiple criteria. In particular, we have exemplified this approach using the Extreme Programming (XP) “shaken but not stirred” project management antipattern. The final model provides more information, such as development cost and functionality, that the manager will consider before deciding to use the refactored solution of the antipattern. We have been interested in the issue of the impact of developer personalities and temperaments on communication and collaboration in pair programming, in order to demonstrate the power of antipatterns modelled through BBNs and MCDA. The proposed BBN-MCDA model can be used in situations where using Bayesian networks to observe the effects of uncertainty on a specific attribute to decide whether to use a refactored solution of an antipattern is not adequate. The subsequent multicriteria decision analysis can be followed to calculate a value for each criterion for a given action and, as a result, the most appropriate action to be discovered. It is suggested that a richer set of data from empirical investigations would be more helpful in representing project management antipatterns using BBNs and MCDA. Besides that, an advanced tool support will be very necessary for the analysis of complex BBNs using MCDA.
Acknowledgement The work presented in this article has been partially funded by the Greek Ministry of Education under the R&D project MISSION-SPM, in the context of the ARCHIMEDES II national programme.
References Beck, K. (1999). Extreme Programming Explained: Embrace Change. AddisonWesley. Belbin, M. (1996). Management Teams: Why They Succeed or Fail, ButterworthHeinemann. Brown, W. J., Malveau, R. C., McCormick H. and Thomas, S. W. (1998). AntiPatterns: Refactoring Software, Architectures and Projects in Crisis. Wiley Computer Publishing. Brown, W. J., McCormick H. and Thomas, S. W. (2000). AntiPatterns in Project Management. Wiley Computer Publishing. Fan, C. F. and Yu, Y. C. (2004). BBN-based Software Project Risk Management. The Journal of Systems and Software, vol. 73 (2), pp. 193-203.
Fenton, N. E., Marsh, W., Neil, M., Cates, P., Forey, S. and Tailor, M (2004). Making Resource Decisions for Software Projects. Proc. of the 26th International Conference on Software Engineering (ICSE 2004), IEEE Computer Society, pp. 397-406. Fenton, N. E. and Neil, M. (2001), Making Decisions: Using Bayesian Nets and MCDA. Knowledge-Based Systems, Vol. 14(7), pp. 307-325. Figueira, J., Greco, S. and Ehrgott, M. (2005). Multiple Criteria Decision Analysis: State of the Art Surveys, Springer Science. Fox, T. and Wayne, S. J. (2005). The Effect of Decision Style on the Use of a Project Management Tool: An Empirical Laboratory Study. The Database for Advances in Information Systems, vol. 36(2), pp. 28-42. Hughes, R. and Cotterell, M. (1999). Software Project Management. Second Edition, McGraw-Hill Publishing co. Kuranuki, Y. and Hiranabe, K. (2004). Antipractices: AntiPatterns for XP Practices. Proc. of the Agile Development Conference, IEEE Computer Society, pp. 83-86. Laplante, P. A. and Neill, C. J. (2006). Antipatterns: Identification, Refactoring and Management. Taylor and Francis. Larman, C. (2004). Agile and Iterative Development: A Manager’s Guide, AddisonWesley. Settas, D., Bibi, S., Sfetsos, P., Stamelos, I. and Gerogiannis, V. C. (2006). Using Bayesian Belief Networks to Model Software Project Management Antipatterns. Proc. of the Fourth International Conference on Software Engineering Research, Management and Applications (SERA 2006), IEEE Computer Society, pp. 117124. Sfetsos, P., Stamelos, I., Angelis, L. and Deligiannis, I. (2006). Investigating the Impact of Personality Types on Communication and Collaboration-Viability in Pair Programming -An Empirical Study. Proc. of the Extreme Programming and Agile Processes in Software Engineering Conference (XP 2006), Lecture Notes in Computer Science, LNCS 4004, Springer, pp. 43-52. Stephens, M. and Rosenberg, D. (2003). Extreme Programming Refactored: The Case Against XP. Apress, 2003. Watthayu, W. and Peng, Y. (2004). A Bayesian Network Based Framework for Multi-Criteria Decision Making. Proc. of Multi-Criteria Decision Making Conference (MCDA 2004), Whistler, B. C., pp. 6-11. Williams, L. and Kessler, R. (2002). Pair Programming Illuminated. Addison-Wesley.
