Fifth International Conference on Information Technology: New Generations
A MODEL BASED TESTING TECHNIQUE TO TEST WEB APPLICATIONS USING STATECHARTS Hassan Reza, Kirk Ogaard, Amarnath Malge School of Aerospace Sciences, University of North Dakota, Grand Forks ND 58202 USA
[email protected] Section 2, briefly discusses the related work. Section 3 describes modeling web applications with Statecharts using our process. Section 4 describes the five types of test coverage that may be applied to Statecharts designed with our algorithm. Section 5 surveys four tools for drawing Statecharts diagrams. Section 6 considers how concurrency in web applications might be modeled using our algorithm. Section 7 considers a possible way to model the back end of a web application using our algorithm. Section 8 considers test metrics for web applications. And in Section 9, we conclude our work.
ABSTRACT In this paper, we discuss a model based software testing method for testing of the web applications using StateCharts. To this end, we briefly describe the test coverage technique, editorial tools for drawing StateCharts, modeling concurrency in web applications, and test metrics. A simple web application has been used to show the feasibility of our approach. Index Terms—Statecharts, test coverage, Web applications, GUI testing, model-based testing
I.
INTRODUCTION
2.
Software testing is an important element of both the software lifecycle and quality assurance activities. In the simples form, testing is the dynamic execution of the software products in order to demonstrate the presence of errors. Software testing is costly, and it is considered as the ultimate barrier to the release of the software product. Model-based testing (MBT) is an automation of black box testing technique; its main dissimilarity from the conventional black box testing methods is that the test cases are automatically generated via software tools that exploit the expected behavioral models of the software under the test (SUT) [11]. MBT has brought many benefits. These include early detection of errors, which is considered to be very inexpensive. The main liability of MBT is to knowing how to model SUT. Testing web applications has shown to be costly because they are heavily event-driven. In this paper we discuss a model-based testing method for web applications that utilizes behavioral models of the SUT from StateCharts models originally devised by Harel [2],[10]. Statecharts models can be used both for modeling and generating test cases for a web application. The main focus of this work is on the front end design and testing of a web application. As such, we utilize the syntax of the web pages to guide the specification of the Statecharts. Our approach is a systematic way to test the front-end functionality of a web application. For the most parts, we are concerned with verifying that the links, forms, and images in the web application under test function according to the specification documents. The organization of our paper as follows. In
978-0-7695-3099-4/08 $25.00 © 2008 IEEE DOI 10.1109/ITNG.2008.145
RELATED WORK
We evaluated a number of visual models that could model web applications, including finite state machines, classical Petri nets, and Statecharts, based on three criteria: representation of structure, concurrency, and hierarchy. In this section we will discuss how we evaluated these three models as potential candidates, and why we chose Statecharts as our model. In [5], the authors discussed the model-based criteria for validating web applications using Design view (DView) for agile software development proces. The criteria only include all nodes, all edges, and all paths. In [7], a web application is modeled using finite state machines by determining the distinct states that the web application can be in and the transitions that can occur between those states. Each state is represented as a node in the finite state machine, and transitions are represented by directed edges. A finite state machine requires an initial state before it can be “executed.” The finite state machine being executed starts in the initial state and progresses to each successive state through firing of transitions until it reaches a final state. A small set of web pages may be modeled using finite state machines, but if we try to model a full-scale web application using finite state machines, the number of states and transitions in the diagram would be very high, making the finite state machine difficult to work with as a modeling tool. Furthermore, a small increase in the complexity of the web application would cause an explosion in the number of additional states and transitions. Finite state machines cannot represent
183
of a web page, that is, the elements of the web page that cannot be decomposed into smaller elements. The atomic elements include the buttons, form elements, and images contained in the web page. Each atomic element is represented by a blob in the Statecharts. Once a blob has been drawn for each atomic element, we can draw the blobs that contain those elements. We continue this process until the entire structure of the web page is represented. After this step is completed, we draw transitions between the blobs to represent the actions that the user may perform on the web page, i.e. clicking on a link or submit button. For each link in the web page, we draw a transition from the blob representing the link to the blob representing the content that will be generated. A link in a web page can be internal or external. An internal link redirects the user to another part of that web page. Internal links are represented in the Statecharts as loops. An external link directs the user to another page of content. This is represented as a transition between the Statecharts for two web pages. We should make note of the fact that the Statecharts for a web page does not represent a specific URL, but rather a set of content generated by the back end of the web application. This distinction is necessary because, in modern dynamic web applications, the same URL may generate different content pages based on the internal state of the web application. Blobs representing form submit buttons also have transitions to other Statecharts, because clicking on a form submit button will cause the back end to generate another web page. These transitions are labeled according to what action will be performed when the form submit button is clicked.
concurrency that may be present in a web application, since only one state in a finite state machine can be active at a time. The problem with above approach is that FMS is inherently flat and this may simply overwhelmed the modelers for the complex web applications with more that 100 states. Petri nets [9], on the other hand, are excellent modeling notations to represent concurrency. A Petri net could be used to represent multiple users browsing a web site simultaneously. Petri nets are a model for visually representing concurrency in a system [8]. Classical Petri nets are not widely used to model web applications, because flat Petri nets do not support the hierarchy inherent to web applications. We chose Statecharts [2] to model web applications for the purposes of testing because Statecharts met all three of our criteria. Statecharts can represent the structure of web applications, as we will demonstrate. Statecharts can also represent concurrency that may be present in a web application and the natural hierarchy of web applications. Table 1 compares finite state machines, Petri nets, and Statecharts based on these three criteria. Visual Model FSMs Petri nets Statecharts
Structure X X X
Concurrency X X
Hierarchy X X
TABLE I. COMPARISON OF MODELS
3.
MODELING WITH STATECHARTS
The process that we have developed for designing Statecharts for a web application is straightforward, and derived from the syntax of the HTML. Since our focus is on the front end of the web application, the language we are concerned with is HTML. We would like to model the web application as it appears to the user. We apply our algorithm to each web page in the web application individually, and then connect the Statecharts we have designed to form a representation of the web application in its entirety. We have introduced a formalized means for modeling web sites in order to facilitate a systematic approach to testing the front-end functionality of web sites. By using Statecharts to model the front end of a web site, rigorous criteria such as test coverage and test metrics can be applied to the testing process. Our approach also provides a systematic method by which the testing process itself can proceed. While we have restricted ourselves to modeling the front-end functionality of a web site using Statecharts in our case study, our approach could be generalized to model the back-end functionality of a web site, as discussed in Section VII. Our algorithm begins with the atomic elements
Fig. 1. Simple web page illustrating how a Statecharts is drawn for an HTML form.
The Statecharts representation of a simple web page in Fig. 1 contains a blob for the main web page, and a blob for each of the three possible web pages that may be generated based on the user’s selection. The blob for the main page contains three blobs: a blob representing the text displayed to the user, a blob representing the
184
drop-down list, and a blob representing the submit button. From the submit button’s blob, we have drawn three transitions, representing the three possible choices. Each of the three transitions is labeled with a guard condition corresponding to the choice it represents. For our case study, we modeled our department web site at the University of North Dakota. In Figs. 2 – 12 we show in detail how our algorithm works. For each individual page in our department web site, we have included a screenshot of the page and the corresponding Statecharts to model it. For each Statecharts, we represent the menu items for the corresponding web page by a blob. Images and text are also represented by blobs. The arrows drawn from the menu item blobs represent the transitions that occur when those menu items are clicked.
Fig. 5. Statecharts for the Academic Programs page.
Fig. 6. Screenshot of the About Computer Science page.
Fig. 2. Screenshot of the home page. Fig. 7. Statecharts for the About Computer Science page.
Fig. 3. Statecharts for the home page.
Fig. 8. Screenshot for the News page.
Fig. 4. Screenshot of the Academic Programs page.
185
Fig. 9. Statecharts for the News page.
Fig. 12. Statecharts showing some of the relationships between the pages.
In a linear organization, we begin with the home page of the application and test each successive web page. In a grid organization, we begin with testing the home page as well. We may then test successive web pages in a column-first or row-first order. In a network organization, we begin with testing the home page. Then we test all the web pages that are reachable (via links) from the home page. While testing a web application using our proposed method, we must use a bottom-up approach. We exclude the drivers that are customarily used with that approach, since drivers are superfluous in our case. We may observe that a top-down approach to testing web applications is not feasible. It is not feasible because there isn't an intuitive way to represent stubs for a web application. All-blobs is the weakest form of test coverage. All-transitions test coverage involves testing every transition in the Statecharts. All-transitions test coverage thus includes testing of all links and form submissions. Links are verified to insure that they are not broken links. The links are also tested to insure that they reference the correct web pages. All forms are submitted by clicking on their submit buttons to verify that the back end of the web application generates the correct content in response. When testing all-transitions, we test all the blobs at the starting point of the transition, since these blobs may affect which transition is fired, and then we test the transitions themselves. We may conclude that alltransitions is a stronger form of test coverage than allblobs, because in all-transitions we test most blobs as well as all of the transitions. Blobs that do not have transitions, such as orphaned web pages, would not be tested under this form of test coverage. All-transition-pairs test coverage tests every pair of transitions between two blobs. Paired transitions may occur in a network organization of web pages, where each page is linked to every other page. All-transition-pairs test coverage is not as strong a form of test coverage as all-transitions in most cases. It only tests transitions that occur in pairs. There may be isolated transitions between
Fig. 10. Screenshot for the Contact Us page.
Fig. 11. Statecharts for the Contact Us page.
4.
TEST COVERAGE CRITERIA
Once the Statecharts have been drawn for the web application, we can observe that there are five types of possible test coverage: all-blobs, all-transitions, alltransition-pairs, all-conditions, and all-paths. We will discuss each of the types of test coverage briefly. Allblobs test coverage involves testing every blob in the Statecharts separately. Web applications are very conducive to this type of test coverage. We can directly visit each blob in a web application's Statecharts without following transitions. This is possible because each blob represents either an atomic element, which may be tested directly, or a compound element, such as a web page, that may be tested by manipulating the GET and POST parameters that are passed to the web server. We test atomic blobs first, then the blobs that contain the atomic blobs, and so forth, until all the blobs are tested. Since we do not follow transitions in all-blobs test coverage, we must choose an order in which to test blobs. This is the integration strategy. The integration strategy may be selected based on the organization of the web pages in the application, i.e. a linear, grid, or networked organization, as described in [4].
186
concurrency is quite common. A web application may exhibit interleaved concurrency if, for example, the user clicks on a link and a new web browser is opened. In this case, the user may interact with either one of the two web browsers that are opened, but he cannot interact with both of them simultaneously. On the back end of a web application, however, we may find examples of true concurrency. How do we represent interleaved concurrency in a web application using Statecharts? This is beyond the scope of our article, but it is an interesting area for further research. Our model assumes that there is no concurrency, true or interleaved, present in the web application. If clicking on a link opens another browser window, we cannot represent this with our modeling system. In our modeling system, all actions result in a transition to another web page. In real world web applications, this is not always true.
blobs that are not tested. All-transition-pairs test coverage is ideal for web applications with highly interconnected web pages. Such a form of test coverage allows us to verify the connectedness of the web pages while testing blobs and transitions as well. All-transition-pairs test coverage may provide stronger test coverage than allblobs and all-transitions test coverage for some web applications, but not all. We cannot strictly determine where it belongs in our test coverage hierarchy, as we can with all-blobs coverage, for example. All-conditions test coverage tests every transition with a guard condition, i.e. transitions from form submit buttons. Clicking on a form submit button forces the back end of a dynamic web application to make a decision on what content to generate, based on the configuration of the form. Transitions that represent links to other pages do not represent conditions. Like all-transition-pairs test coverage, all-conditions test coverage provides stronger test coverage for some types of web applications, and weaker test coverage for other types. Thus we cannot assign it a specific place in our test coverage hierarchy. Finally, all-paths test coverage tests every possible path through a web application. All-paths are clearly the strongest form of test coverage, but it is infeasible for moderately to highly complicated web applications, as stated in [5]. There may be thousands of possible paths through a modern complex web application. We cannot realistically test all of them.
7.
Another area of web applications we have not considered yet is the back end of a web application. Our approach is concerned with testing the functionality of the front end—verifying links, verifying images, insuring that forms operate properly, and so forth. However, the back end of a web application is where a significant amount of the work is done. We will briefly outline a possible algorithm for designing Statecharts for the back end of a web application, and then examine the possible forms of test coverage for it. When modeling the back end of a web application, blobs can no longer represent web pages. Blobs must now represent the possible states that the back end can assume. Transitions in the Statecharts represent the transitions between these states. We need to design separate models for the front and back end, since the differences in their behaviors is significant. The back end of a web application may be programmed in a much larger variety of languages than the front end—Java servlets, PHP, and ASP.NET to name a few. The functionality of a typical back end for a web application is described in detail in [3]. But, while the language of implementation for the back end may vary, the functions that the back end performs are generally similar. The back end of a web application generates web page content based on the GET and POST parameters that are passed to it. The back end maintains state information, in contrast to the front end, which is largely stateless. State information is maintained through session variables that persist from one web page to the next. How do we represent these session variables in our proposed model? The session variables are not represented explicitly, but are rather incorporated into the possible states and transitions between those states. We may have a transition, for example, with the guard condition that x = 0. Whenever x is equal to 0, the transition fires and another state, represented by a blob, becomes active.
5. TOOLS FOR DRAWING STATECHARTS There are a number of software tools available for drawing Statecharts. We consider only four in our article, but more are available, either as commercial or free software. A complete discussion of software tools for drawing Statecharts is available in [6]. Rational Rose from Rational Software Corporation is the most popular tool for drawing UML diagrams, including Statecharts. In addition to the ability to draw any UML diagram, Rational Rose has features such as version control that are useful in software development and testing. Another software tool enabling the user to draw Statecharts is the UML StateWizard. The UML StateWizard is a Visual C++ add-on library that can be used for developing simulations. The Together Control Center from Borland is yet another set of tools for designing UML (and hence Statecharts) diagrams, similar to Rational Rose. And finally we consider the Poseidon software tool from Gentleware, which is part of an open source project for UML tools. Poseidon is available free to software developers and testers.
6.
BACK-END MODELING
MODELING CONCURRENCY
Statecharts can represent true concurrency through orthogonal states, as stated in [2]. True concurrency in a web application is rare, but interleaved
187
same models, which were developed to test the web application running under a particular browser, and apply these models to the testing of the web application under other browsers. Moreover, we have not yet found solutions to the problems of modeling concurrency and modeling the back-ends of web applications. Possible solutions have been proposed in our paper, but there is much that is left for future research.
The categories of test coverage for a back end of a web application that is modeled according to our outlined algorithm are not as distinct as these categories are for the front end. If we consider all-blobs coverage for the back end, for instance, there is no clear method to directly visit each blob without following any transitions, as there is in the front end. Each blob in the Statecharts representing a back end of a web application represents a state that is the culmination of previous states. So we cannot visit a particular blob without extensive manipulation of the state information for the back end. We could conceivably do so but we would have to assign specific values to session variables in order to activate the state represented by the blob. This is cumbersome, and not conducive to efficient testing of the back end of the web application.
8.
10.
ACKNOWLEDGEMENTS
The first author has been party funded with grants from Rockwell Collins.
11.
TEST METRICS
[1]
Another area of possible further research is the development of test metrics for the types of test coverage we have described for web applications modeled with Statecharts. Such test metrics would allow us to quantitatively show how much of the web application is tested by the five types of test coverage we have described. These test metrics would be statistical probabilities, rather than analytically-derived percentages, as stated in [1]. We might develop statistical probabilities for the amount of the web application that is tested by applying our algorithm to the testing of a large number of applications. The probability that a certain type of test coverage would test some percentage of a web application could then be determined with a reasonable amount of accuracy. The method proposed in [1] for evaluating the traditional forms of Statecharts test coverage, using fault seeding, could perhaps be adapted to evaluate the effectiveness of the forms of test coverage we have proposed.
[2] [3]
[4] [5]
[6] [7] [8]
9. CONCLUSIONS AND FUTHER WORKS
[9]
Testing web applications has proved to be a difficult task, due to the complexity and variation in structure of web applications. Applying traditional methods of modeling and testing to web applications that are designed for use with applications written in imperative languages may create a number of problems. In this work, we have shown how model-based software testing can be used. More specifically, we discussed how Statecharts [2] might be used to model web applications for the purposes of testing. The method that we have proposed in this paper is primarily for verifying the functionality of the front end. One area that we have not considered is browser compatibility. This cannot be feasibly modeled for the purposes of testing. If we used our method for testing browser compatibility, we would be required to use the
[10]] [11]
188
REFERENCES L. C. Briand, Y. Labiche, Y. Wang, "Using simulation to empirically investigate test coverage criteria based on statecharts," Proceedings of the 26th International Conference on Software Engineering, 2004. D. Harel, "On visual formalisms," Communications of the ACM, vol. 31, no. 5, pp. 514-530, 1988. G. Gousios, D. Spinellis, "A comparison of portable dynamic web content technologies for the Apache web server," Proceedings of the 3rd International System Administration and Networking Conference, pp. 103119, 2002. T. A. Powell, Web Site Engineering, Prentice-Hall, 1998. J. Alava, P. Clarke, “Model-based test criteria definition for validating annotated web applications,” Proceedings of the 2nd MoDeVa Workshop – Model Design and Validation, 2005. M. Chitnis, P. Tiwari, L. Ananthamurthy, “UML tools,” http://www.developer.com/design/article.php/1593811 A. Andrews, J. Offutt, R. Alexander, “Testing web applications by modeling with FSMs,” Software And System Modeling, vol. 4, no. 3, pp. 326-345, 2005. R. Johnsonbaugh, T. Murata, "Petri nets and marked graphs—mathematical models of concurrent computation," American Math. Monthly, vol. 89, no. 8, pp. 552-566, 1982. C. Petri, Kommunikation mit Automaten, University of Bonn, 1962. D. Harel. Statecharts: A Visual Formulation for Complex System. Sci.Comput. Program. 8(3):232-274, 1987. M. Utting, B. Legeard. Practical Model-based Testing: A tools approach. Morgan Kaufmann, 2007.
