An empirical investigation on the relation between ... - Semantic Scholar

8 downloads 1469 Views 197KB Size Report
financial resources, the common practice of small software companies is only devoted to produce requirements analysis models, which are often adopted to ...
An Empirical Investigation on the Relation between Analysis Models and Source Code Comprehension Carmine Gravino and Genoveffa Tortora

Giuseppe Scanniello

Dipartimento di Matematica e Informatica University of Salerno Via Ponte Don Melillo, +39089-963503

Dipartimento di Matematica e Informatica University of Basilicata Viale Dell'Ateneo, Macchia Romana +390971-205881

[email protected], [email protected]

[email protected] companies to take decisions in their development practices [1]. For example, the support of models produced in the early phase of the development process may be crucial in the system testing [4], where a tester must acquire knowledge about the objects (e.g., application features) he/she has to test. However, there are many activities that involve the comprehension of software systems, in general, and the comprehension of source code, in particular [3].

ABSTRACT This paper presents a controlled experiment to investigate whether the comprehension of source code increases in case novice software engineers use abstract software models produced in the early phase of the software development, i.e., requirements analysis. The study has revealed that there is not significant difference in the comprehension of source code achieved by using or not abstract software models.

This paper presents a controlled experiment to investigate whether the comprehension of source code increases in case novice software engineers are provided with software models produced using the behavioral modeling.

Categories and Subject Descriptors D.3.2 [UML]: Assessing the relation between the use of class and sequence diagrams and the comprehension of object oriented software systems.

2. THE EXPERIMENT In this section we describe the experiment according to the guidelines suggested by Wohlin et al. in [5].

General Terms Documentation, Design, Experimentation, Human Factors.

2.1 Definition and Context The paper focuses on the comprehension of source code by using software system abstract models. In particular, we consider the method suggested in [2], where analysis modeling is composed of: functional model, analysis object model, and dynamic model. Functional model focuses on the functionality, while analysis object model focuses on the concepts that will be manipulated by the system. Finally, dynamic model concerns the behavior of the meaningful use cases presented in the functional model.

Keywords Software Maintenance, Controlled Experiment, UML.

1. INTRODUCTION The Unified Modeling Language (UML) represents one of the main employed notations for object oriented software analysis and design modeling. In particular, several concerns can be modeled during the lifecycle of a software system using suitable diagrams to describe its functional behavior, static structure, and dynamical behavior [2].

The controlled experiment aimed at assessing whether functional model, analysis object model, and dynamic model improve the comprehension of the source code of a given software system. To this end, we investigated the effect of using the methods SM (i.e., Software Models) and NO_SM (i.e., NO Software Models), to perform two comprehension tasks. Subjects used functional, analysis object, and dynamic models to accomplish the comprehension task when using SM. On the other hand, source code was applied to accomplish the task with NO_SM. We considered as functional models the structured description of use cases, while the UML class diagrams were employed to abstract analysis object models. To model the dynamic aspect we considered the UML sequence diagrams.

Due to the short time to market and the limited human and financial resources, the common practice of small software companies is only devoted to produce requirements analysis models, which are often adopted to comprehend source code in order to perform maintenance operations. The costs and benefits of using abstract software models based on UML and the impact of program documentation have been marginally investigated in the past. These studies are needed however to allow software

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC’10, March 22-26, 2010, Sierre, Switzerland. Copyright 2010 ACM 978-1-60558-638-0/10/03…$10.00.

The research question we formulated and investigated here is: Does the combined use of software models produced in the analysis phase increase the source code comprehension? The experiment was conducted within a laboratory at the University of Basilicata with volunteers. In particular, 16 third year Bachelor students in Computer Science were involved.

2365

To conduct the empirical analysis we used the non-parametric Wilcoxon test.

2.2 Hypotheses Formulation To assess the research question we have formulated the following null hypothesis:

2.5 Influence of the Method

Hn0: The combined use of software models produced in the analysis phase does not significantly improve the comprehension of the software system source code with respect to the only use of source code.

The Wilcoxon test revealed that the null hypotheses Hn0 cannot be rejected. In fact, the obtained p-value was 0.172. To further investigate this concern, we analyzed the correct answers of the subjects within the laboratory runs. We noted that 3 subjects benefit more using SM, while 1 subject benefit more using NO_SM. The number of correct answers provided by the remaining 4 subjects was the same when using or not software models. Hence, the analysis of the influence of method suggests that even if better results have been obtained when software models are used the differences are not statistically significant.

In case the null hypotheses can be rejected with relatively high confidence, it is possible to formulate the corresponding alternative hypothesis, which admit a positive effect of the use of the software models.

2.3 Variables and Experiment Design In order to properly design the controlled experiment and analyze the results, we considered the following independent variables or factors:

2.6 Influence of the Task As the complexity of the tasks T1 and T2 could bias the results we have also analyzed their effect on the performance of the subjects of the group B. In particular, we performed the Wilcoxon test in order to verify whether there is statistically significant difference between the number of correct answers provided by the subjects within the groups A and B using the method NO_SM on the tasks T1 and T2, respectively. The results of the test revealed that there is not a statistically significant difference between the number of correct answers (p-value = 0.394) achieved by the groups A and B. Hence, the complexity of the tasks can be considered comparable and then does not influence the results.

Method: the factor on which the study is focused on, namely SM and NO_SM. Task: the comprehension tasks to perform. In particular, the following are the tasks the subjects were asked to perform within the laboratory sessions (also named runs): T1: Comprehending the functionality “Search a Song by Singer” of a Java system for managing the selling within a music shop; T2: Comprehending the functionality “Buy a Ticket Unsuccessfully” of a Java software system for managing the ticket reservation of a theater.

3. FINAL REMARKS The experiment revealed that better results have been obtained when software models are used. The difference is however not statistically significant. This result could be considered unexpected. In fact, we imagine that providing to a software engineer software models, he/she would obtain a better comprehension level. Furthermore, a careful reader may object to the fact that the involved subjects may affect the soundness of our study. Nevertheless, the business of a small software house, especially in Italy (the context where we would like to generalize the results), is based on practitioners with a Bachelor degree in computer science, thus considering the involved subjects not far from professional programmers.

To get a quantitative evaluation of the comprehension, subjects were asked to fill in a questionnaire for each task. Accordingly, we have considered the following dependent variables: Correct: the number of correct answers provided by the subjects on the comprehension questionnaire.

2.4 Design, Preparation, and Execution The subjects was randomly assigned to the groups A and B, and started to work in the first laboratory run on the task T1. In particular, the subjects of the group A were asked to perform the task T1 using only the Java source code. The subjects within the group B accomplished T1 and T2 using SM and NO_SM, respectively. The experiment design is summarized Table 1.

4. REFERENCES [1] Basili, V.R., Selby, R.W., and Hutchens, D.H. Experimentation in Software Engineering. IEEE Transaction on Software Engineering, 12,7 (1986), 733-743.

All the subjects attended an introductory lesson where detailed instructions on the tasks were presented and then they conducted the experiment according to the adopted design.

[2] Bruegge, B., and Dutoit, A. Object-Oriented Software Engineering Using UML, Patterns, and Java, Prentice Hall.

Regarding the experimental material, the subjects were provided with: (i) Handouts of the introductory presentation; (ii) Printout of the tasks: source code, the class diagrams, and the sequence diagrams (depending on the treatment), and the comprehension questionnaires; (iii) Some white sheets and a pencil.

Groups of subject A B

[3] Canfora, G., and Di Penta, M. New Frontiers of Reverse Engineering. In Proceedings of the International Conference on Software Engineering (FOSE’07) (Minneapolis, MN, USA, May 23-25, 2007). IEEE CS Press. [4] Sneed H.M. .Program Comprehension for the Purpose of Testing. In Proceedings of International Workshop on Program Comprehension (IWPC’04) (Bari, Italy, 24-26 June 2004). IEEE CS Press.

Method SM, LAB 1 NO_SM, LAB 2 T1, LAB1 T1, LAB1 T2, LAB2

[5] Wohlin, C. Runeson, P., Host, M., Ohlsson, M. C., Regnell, B., and Wesslen, A. Experimentation in Software Engineering - An Introduction, Kluwer, 2000.

Table 1. Experiment Design

2366