AWERProcedia Information Technology & Computer Science

4 downloads 238883 Views 405KB Size Report
CMMI level 3 certified defence industry company. First of all ... Information Technology & Computer Science. [Online]. 2013, 04, pp 257-262. Available from: ...
AWERProcedia Information Technology & Computer Science Vol 04 (2013) 257-262

3rd World Conference on Innovation and Computer Sciences 2013

Comparison of Three Software Effort Estimation Methodologies with Case Study Tülin Erçelebi Ayyıldız *, Computer Engineering, Başkent University, Ankara 06810, TURKEY. Altan Koçyiğit, Informatics Institute, METU, Ankara 06800, TURKEY. Deniz Peker, Savunma Teknolojileri Mühendislik ve Ticaret A.Ş., Ankara 06800, TURKEY. Suggested Citation: Ayyıldız, E., T., Koçyiğit, A. & Peker, D. Comparison of three software effort estimation methodologies with case study. AWERProcedia Information Technology & Computer Science. [Online]. 2013, 04, pp 257-262. Available from: www.awer-center.org/pitcs Received December 09, 2012; revised January 19, 2013; accepted March 04, 2013. Selection and peer review under responsibility of Prof. Dr. Fahrettin Sadıkoglu, Near East University. ©2013 Academic World Education & Research Center. All rights reserved. Abstract It is an important issue in the software industry to predict how much effort will be required for a software project. The more accurate the effort estimation is in the early product development lifecycle, the higher is the performance in utilization of the resources and in meeting the deadline requirements. There are a number of methodologies developed for effort estimation. Use Cases, which are used for describing and capturing the functional requirements of a software system, is very popular and Use Case Points (UCP) method is one of the widely used effort estimation methodologies both in the industry and academia. There are also other most widely used use case based effort estimation methods, which are called Adapted Use Case Point (AUCP) and Simplified Use Case Point (SUCP). In this study, fourteen software projects’ effort estimations are studied. These fourteen software projects are taken from a CMMI level 3 certified defence industry company. First of all, fourteen software projects’ effort estimations are made according to UCP, AUCP and SUCP methodologies, respectively. Then, these three methods’ results are compared to actual effort spent. As evaluation criteria, magnitude of relative error (MRE) is applied in this case study for detecting the estimation errors. According to results obtained, for the projects considered, SUCP methodology gives a higher correlation between the actual effort and estimated effort than the correlation obtained regular UCP method and AUCP methods applied. Keywords: Effort Estimation, Use Case Point, Adapted Use Case Point, Simplified Use Case Point;

_____________________________

* ADDRESS FOR CORRESPONDENCE: Tülin Erçelebi Ayyıldız, Başkent University and METU, ANKARA and 06800, TURKEY E-mail address: [email protected] / Tel.: +90-532-766-8596

Ayyıldız, E., T., Koçyiğit, A. & Peker, D.Comparison of three software effort estimation methodologies with case study. AWERProcedia Information Technology & Computer Science. [Online]. 2013, 04, pp 257-262. Available from: www.awer-center.org/pitcs

1. Introduction Estimation plays an important role in our everyday life thus; poor estimations are one of the major reasons for failures in most of the cases. Effort, cost and quality can be estimated either with using past historical records of an organization or with expert opinion. There are some well known effort estimation methods for software projects, and some of these methods utilize use cases written in requirements phase. Use Case Points (UCP) method is one of the widely used use case based effort estimation methodologies both in the industry and academia. There are also other widely used use case based effort estimation methodologies such as Adapted Use Case Point (AUCP) and Simplified Use Case Point (SUCP). The Karner’s original UCP method takes into consideration the complexity classification of an Actor and Use Case by counting the number of transactions in a Use Case that is classified as simple, average and complex, respectively. In the AUCP method the original UCP method was adapted to incremental development. This method is evaluated on a large industrial system with alteration of software. SUCP methodology aims to simplify the original UCP methodology by rejecting actor classification and decreasing technical and environmental factors. The objective of this study is to analyse the UCP based effort estimation methods and examine the impact of their constituents on the accuracy of the method. The rest of paper is organized as follows: Section 2 gives a brief introduction to the UCP based effort estimation methods. In Section 3, the effort estimations based on UCP, AUCP and SUCP methods are investigated for fourteen real life projects. Finally in Section 4, the results are evaluated and discussed.

2. Use Case Based Effort Estimation Techniques In this section, three of the widely used use case based effort estimation methods are summarized and discussed. These methods are:  Use Case Points (UCP) [2]  Adapted Use Case Point (AUCP) [5]  Simplified Use Case Point (SUCP) [6] 2.1. Use Case Points (UCP) UCP is the basic technique proposed by Gustav Karner [2] for estimating effort based on Use Cases. The method assigns quantitative weight factors (WF) to actors and use cases as according to their classification as Simple, Average and Complex. The UCP method is outlined in Figure 1. The sum of all the weights assigned to actors gives the Unadjusted Actor Weight (UAW). A similar way, Use Cases are assigned weights and are classified according to their complexity. The sum of the weights of the Use Cases gives the Unadjusted Use Case Weight (UUCW). Thus, Unadjusted Use Case Points (UUCP) is obtained from the sum of UAW and UUCW. UUCP = UAW + UUCW

(1)

258

Ayyıldız, E., T., Koçyiğit, A. & Peker, D.Comparison of three software effort estimation methodologies with case study. AWERProcedia Information Technology & Computer Science. [Online]. 2013, 04, pp 257-262. Available from: www.awer-center.org/pitcs

Figure 1. The UCP Effort Estimation Steps

Technical complexity of the projects considered is represented by TCF, which consists of 13 technical factors. These technical factors are assigned to some specific weights, and sum into TCF. Development resources are represented by EF, also referred to as experience factors. The UCP model describes eight such factors contributing to the effectiveness of the development team. After computing TCF and EF, they are multiplied with the UUCP to yield Adjusted Use Case Points (AUCP) as follows. TCF=0.6 + (0.01*TFactor)

(2)

EF=1.4 + (-0.03*EFactor)

(3)

AUCP =UUCP *TCF*EF

(4)

Therefore the size of the project is obtained. There are some methods proposed to convert the size obtained from the use case evaluation to required effort. For example; 20 person hours (Phper UCP) are given for 1 UCP in the Karner’s method. At the end, effort is estimated as follow: Effort= AUCP*20 person hours

(5)

259

Ayyıldız, E., T., Koçyiğit, A. & Peker, D.Comparison of three software effort estimation methodologies with case study. AWERProcedia Information Technology & Computer Science. [Online]. 2013, 04, pp 257-262. Available from: www.awer-center.org/pitcs

2.2. Adapted Use Case Points (AUCP) In this study, the original method was adapted to incremental development model. Authors claimed that the impact of the TCF is small. The EF is not relevant for their project since there are small numbers of changes from one release to another. As a result, their TCF and EF =1. Therefore, in this study TCF and EF are fixed to 1. Moreover, authors stated that all actors assumed to be average in classification because classification has little impact on the final estimation result. Therefore, in this study all actors are assumed to be average. Indeed, all use cases considered as complex initially, then use cases are broken down to simple and average based on transactions. 2.3 Simplified Use Case Points (SUCP) This method is aimed to simplify the original UCP method. Authors, proposing SUCP, claim that original UCP method has some drawbacks about TCF and EF. They state that these factors lack standardization and they are subjective. To investigate the impact of the TCF and EF they compared the accuracy of the effort estimation based on the Karner’s original UCP, unadjusted UCP and unadjusted Use Case Weight (UCW). In order to examine the possibility of reducing the number of TCFs and EFs, they performed the factor analysis [3]. The purpose of the factor analysis was to discover groups of potentially overlapping factors within TCFs and EFs that could be subrogated for more general factors. As a result 21 factors of TCFs and EFs are reorganized into a simplified model that consists of 4 TCF and 2 EF. They categorized 4 TCFs under the title of Efficiency, Operability, Maintainability and Interoperability. 2 EFs are Team Experience and Team Cohesion. Researchers also discuss the UAW’s contribution on the effort estimation with UCP. In order to examine the influence of UAW, they calculated actual effort with UCP and without UCP. The observed situation is almost the same for both variants. Therefore, they claimed that UAW is negligible and the minor impact on the effort estimation with UCP [6]. Therefore, UAW is not considered in this method. 3. Case Study In this paper, size measurement and effort estimation for fourteen real life software projects are investigated. These fourteen projects are taken from a software development company of CMMI level 3 in Turkey. These projects are defense industry projects and due to confidentiality reasons the details of the projects are not presented in the paper and the projects are named as A,B,C,D,E,F,G,H,I,J,K,L,M,N. First of all, we estimated the effort for the software projects according to Karner’s original UCP, AUCP and SUCP methods. We compared the three results to actual effort, in order to determine the accuracy of the effort estimation methods.

260

Ayyıldız, E., T., Koçyiğit, A. & Peker, D.Comparison of three software effort estimation methodologies with case study. AWERProcedia Information Technology & Computer Science. [Online]. 2013, 04, pp 257-262. Available from: www.awer-center.org/pitcs

In this study productivity factor is assumed to be equal to 20 person-hours for all fourteen projects and for all three methods. As evaluation criteria, magnitude of relative error (MRE) is applied in this case study. MRE is measured as: MRE 

AE  EE AE

(6)

where AE is the actual effort, and EE is the estimated effort. MRE is considered such that the accuracy of an estimation technique is inversely proportional to the MRE [4]. The results are given in Table 1. Table 1. Case Study Results

UCP

AUCP SUCP

Estimated Effort 11595 12670 10801 Project A MRE 0,09 0,16 0,02 10561 10561 10561 Actual Effort

Estimated Effort 7230 6840 6860 Project H MRE 0,12 0,06 0,06 6439 6439 6439 Actual Effort

Estimated Effort 14770 13992 11840 Project B MRE 0,12 0,06 0,09 13105 13105 13105 Actual Effort

Project I

Estimated Effort 8335 7840 7565 MRE 0,15 0,08 0,04 7210 7210 7210 Actual Effort

Estimated Effort 6177 6304 5532 Project C MRE 0,06 0,08 0,04 5819 5819 5819 Actual Effort

Project J

Estimated Effort 6824 6140 5764 MRE 0,27 0,15 0,08 5336 5336 5336 Actual Effort

Estimated Effort 10848 12660 10545 Project D MRE 0,31 0,51 0,26 8342 8342 8342 Actual Effort

Project K

Estimated Effort 6320 5140 6196 MRE 0,13 0,08 0,10 5597 5597 5597 Actual Effort

Estimated Effort 1388 2760 1860 Project E MRE 0,35 0,27 0,14 2165 2165 2165 Actual Effort

Project L

Estimated Effort 3412 3802 2457 MRE 0,14 0,27 0,17 2989 2989 2989 Actual Effort

Estimated Effort 1816 2660 2235 Project F MRE 0,22 0,12 0,05 2354 2354 2354 Actual Effort

Estimated Effort 13480 11910 12360 Project M MRE 0,19 0,05 0,09 Actual Effort 11286 11286 11286

Estimated Effort 5180 5820 4960 Project G MRE 0,10 0,24 0,06 4667 4667 4667 Actual Effort

Project N

Estimated Effort 3812 3645 3284 MRE 0,42 0,36 0,22 2678 2678 2678 Actual Effort

261

Ayyıldız, E., T., Koçyiğit, A. & Peker, D.Comparison of three software effort estimation methodologies with case study. AWERProcedia Information Technology & Computer Science. [Online]. 2013, 04, pp 257-262. Available from: www.awer-center.org/pitcs

According to the results, in nine projects (A,C,D,E,F,G,I,J,N) SUCP method gives effort estimation more closer to the actual effort as compared to the Karner’s original UCP method and AUCP. In the three of remaining five projects (B,K,M) AUCP method gives the better results among all methods. In project (H), the same result obtained by SUCP and AUCP methods that is AUCP method is not worse than the SUCP method. Finally, in the remaining one project (L) a better result is obtained with Karner’s original UCP method. Within the lights of the results obtained in this case study we can say that, for the projects considered, SUCP method gives much better results compared to other methods. For all fourteen projects UCP method gives the worse estimation among the all three alternatives. 4. Conclusion In this paper the Karner’s UCP method and UCP based effort estimation methods, which are called SUCP and AUCP, were investigated based on the data collected from 14 software projects. The aim was to analyse the UCP based effort estimation methods and examine the impact of their constituents on the accuracy of the method. According to results obtained, for the projects considered, SUCP methodology gives a higher correlation between the actual effort and estimated effort than the correlation obtained regular UCP and AUCP methods applied. As a future work, our findings will be investigated by increasing the number of projects utilizing UCP based method for effort estimation increases. References Alpaydın, E.; Introduction to Machine Learning (Adaptive Computation and Machine Learning), The MIT Press, 2004 Karner, G., (1993). Metrics for Objectory. Diploma thesis, University of Linkoping, Sweden.No. LiTH-IDA-EX-9344, 21. Kim, J.; Mueller, C.; Factor Analysis: Statistical Methods and Practical Issues, Sage Publications, Inc., 1978 Jørgensen, M.; Østvold, K.M., (2004). Reasons for Software Effort Estimation Error: Impact of Respondent Role, Information Collection Approach, and Data Analysis Method, IEEE Transactions On Software Engineering, Vol. 30, No. 12. Mohagheghi, P., Anda, B., Conradi, R.; Effort Estimation of Use Cases for Incremental Large-Scale Software Development. ICSE May 15-21, 2005. Ochodek, M., Nawrocki, J., Kwarciak, K.; Simplifying effort estimation based on Use Case Points. Journal of Information and Software Technology, 0950-5849 © 2010 Elsevier B.V

262