Investigating Testing Approaches for Dynamically Adaptive Systems Work in Progress Fabiano C. Ferrari
Bruno B. P. Cafeo
Federal University of São Carlos São Carlos, SP - Brazil
University of São Paulo São Carlos, SP - Brazil
[email protected]
[email protected]
Joost Noppen, Ruzanna Chitchyan, Awais Rashid Lancaster University Lancaster, Lancashire - UK
{j.noppen,r.chitchyan,marash} @comp.lancs.ac.uk
1.
OVERVIEW
Software systems that can automatically adapt to changes in their environments – also know as Dynamically Adaptive Systems (DAS) – play increasingly important roles in society’s everyday life. The growing demand for such systems, which range from simple mobile interactive gaming applications to complex crisis management systems, requires specific development methodologies that encompass the full software life cycle. The EU-funded DiVA project (Dynamic Variability in Complex, Adaptive Systems) [1] aims at providing a toolsupported methodology for developing DASs. The DiVA framework addresses the management of dynamic variability through a combination of aspect-oriented and model-driven techniques. Within the DiVA project context, we are interested in developing testing strategies that can adequately handle the issues involved in testing DASs. In particular, the combinatorial explosion of variant composition alternatives, many of which probably unforeseen at design time, and the runtime nature of system re-configuration, makes testing a daunting task for software developers. We started our investigation by performing a systematic mapping study of testing approaches for DASs. The goals of our survey were two: (RQ1) identifying the challenges authors have described so far in the context of DASs testing; and (RQ2) identifying candidate testing approaches which can be customised to address the challenges within the scope of the DiVA project. In regard to the research question (RQ1), we enumerated the following challenges: 1. How to guarantee the correctness of systems that have never been tested in advance [5, 6, 13]; 2. How to detect and avoid incorrect system configurations defined at runtime [5, 6]; 3. How to test adaptive systems that run in a distributed environment [8, 7];
4. How to deal with user interference in the system configuration [7]; 5. How to anticipate all the relevant context changes and when they could impact the behaviour of contextaware applications [12]; 6. How to deal with context-dependent control and data flow in DASs (e.g. how to avoid “noisy contexts” resulting from sensor interference) [3, 4]; 7. How to deal with the exponential growth of system configurations that should be tested [13]. Note that some challenges can be more adequately tackled with approaches other than testing (e.g. challenge #2 is more about rule-based reasoning). Some others are domainspecific (e.g. challenge #3 that regards distributed systems and #6 that regards sensor-based systems). Nevertheless, the current state of the art shows that improving these systems’ reliability through testing still requires large effort from the research community and software practitioners. In regard to the research question (RQ2), some approaches address the reduction of the associated costs through builtin testing strategies [5, 6] or through the prioritisation of components to be tested [13]. Other approaches include: applying model-checking algorithms for automatic detection of faults related to distribution [8, 7]; analysing the control and data flow focusing on context-aware data [12, 11, 3, 4, 2]; and analysing multi-layer context-aware adaptive applications [9]. The next step of our research is identifying how we can leverage the current testing approaches for DASs in order to compose an adequate testing framework for DiVA systems. An important property of the DiVA approach is that it is strongly oriented to model transformations. Along the full system life cycle, models are enriched and transformed with proper tooling support [10]. This shall be taken into account in our upcoming research initiatives.
Acknowledgments This work is sponsored by the European Union as part of the DiVA project (ICT-215412).
References [1] Dynamic Variability in Complex, adaptive systems. Online. http://www.ict-diva.eu/DiVA/ - 28/02/2011. [2] A. Flores, J. C. Augusto, M. Polo, and M. Varea. Towards context-aware testing for semantic
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
interoperability on PvC environments. In SMC’04, pages 1136–1141. IEEE, 2004. H. Lu, W. K. Chan, and T. H. Tse. Testing context-aware middleware-centric programs: A data flow approach and an rfid-based experimentation. In FSE’06, pages 242–252. ACM, 2006. ISBN 1-59593-468-5. H. Lu, W. K. Chan, and T. H. Tse. Testing pervasive software in the presence of context inconsistency resolution services. In ICSE’08, pages 61–70. ACM, 2008. ISBN 978-1-60558-079-1. D. Niebuhr and A. Rausch. A concept for dynamic wiring of components: Correctness in dynamic adaptive systems. In ESEC/FSE SAVCBS’07 Workshop, pages 101–102. ACM, 2007. ISBN 978-1-59593-721-6. D. Niebuhr et al. Achieving dependable component bindings in dynamic adaptive systems - a runtime testing approach. In SASO’09, pages 186–197. IEEE, 2009. ISBN 978-0-7695-3794-8. M. Sama, S. Elbaum, F. Raimondi, D. S. Rosenblum, and Z. Wang. Context-aware adaptive applications: Fault patterns and their automated identification. IEEE Transactions on Software Engineering, 36(5): 644–661, 2010. ISSN 0098-5589. M. Sama, D. S. Rosenblum, Z. Wang, and S. Elbaum. Model-based fault detection in context-aware adaptive applications. In FSE’08, pages 261–271. ACM, 2008. ISBN 978-1-59593-995-1. M. Sama, D. S. Rosenblum, Z. Wang, and S. Elbaum. Multi-layer faults in the architectures of mobile, context-aware adaptive applications. Journal of Systems and Software, 83(6):906 – 914, 2010. ISSN 0164-1212. D. Vojtisek and F. Munoz. Diva Studio architecture. Online. http://www.ict-diva.eu/DiVA/results/ diva-deliverables - 28/02/2011. H. Wang and W. K. Chan. Weaving context sensitivity into test suite construction. In ASE’09, pages 610–614. IEEE, 2009. ISBN 978-0-7695-3891-4. Z. Wang, S. Elbaum, and D. S. Rosenblum. Automated generation of context-aware tests. In ICSE’07, pages 406–415. IEEE, 2007. ISBN 0-7695-2828-7. K. Welsh and P. Sawyer. Managing testing complexity in dynamically adaptive systems: A model-driven approach. In ICST ViDaS’10 Workshop, pages 290–298. IEEE, 2010. ISBN 978-0-7695-4050-4.
