Hospital Information Systems Management: Towards ...

Hospital Information Systems Management: Towards a comprehensive Maturity Model João Vidal de Carvalho 1, Álvaro Rocha 2, José Braga de Vasconcelos3 1

ISCAP, Instituto Politécnico do Porto, Porto, Portugal [email protected] 2 Departamento de Engenharia Informática, Universidade de Coimbra & LIACC, Coimbra & Porto, Portugal [email protected] 3 Universidade Atlântica, Oeiras, Portugal [email protected]

Abstract. In the present paper we put forth a preliminary research aimed at the development of an encompassing maturity model for the management of hospital information systems. The development of this model is justified to the extent that current maturity models in the field of hospital information systems management are still in an early development stage, and especially because they are poorly detailed, do not provide tools to determine the maturity stage nor structure the characteristics of maturity stages according to different influencing factors. Keywords: Stages of Growth, Maturity Models, Hospital Information Systems, Strategy, Management

1 Introduction Health care institutions and governmental organizations are starting to understand that the reasons underlying a certain inadequacy in the management of health processes directly relates to infrastructural limitations and their inefficient management [17, 64]. Hospital Information Systems (HIS) managers usually contemplate the errors that occurred in these organizations and wonder what could have been done to avoid them. They conclude that these errors are usually a natural growth and maturation symptom of organizations, and are often the result of the development that brought the organization to its current maturity stage [59]. This phenomenon of change fits the principles behind the growth stages theory and in the current context surrounding Information Systems (IS) of health organizations. Based on this presupposition which highlights the relevance of Maturity Models in the HIS fields, this research work is intended at the development of a maturity model that is especially adapted to the needs of Hospital Information Systems Management. To develop this new model, we carried out a preliminary study on IS Maturity Models and respective specificities. Based on this preliminary review of the State of the Art, concerning these two types of maturity models, we define a research methodology to propose and validate the new maturity model.

Cite as: Carvalho, J.V., Rocha, Á. & Vasconcelos, J.B. (2015). Hospital Information Systems Management: Towards a comprehensive Maturity Model, In Proceedings of the Mediterranean Conference on Information & Communication Technologies'2015, Saidia, Marocco, May 7 – 9, 2015.

Besides this section, this article is organized in fifth more. Accordingly, the second section proposes a preliminary systematization of the state of the art concerning maturity models in IS Management and HIS Management. The third section defines the problem underlying this research work. The fourth section presents the questions and objectives of the research. The fifth puts forth and explains the methodological approach adopted in the development of this research work and, finally, the sixth section offers some final remarks concerning this study.

2 Preliminary Review of Literature Information Systems Management (ISM) is the activity responsible for the tasks of an organization pertaining to Information Management, Information Systems and the adoption of Communication and Information Technologies (CIT) [1]. The maturity of this activity is a key factor for the success of organizations, to the extent that an IS is fundamental for their survival, competitive edge and success. In this context, there are several instruments that help the ISM achieve an enhanced maturity, namely the so called Maturity Models. Indeed, maturity models provide organization managers an important model for the identification of the maturity stage of an IS in order to plan and implement actions that will allow them to move towards an enhanced maturity stage, and thus achieve the proposed goals [57]. Maturity Models can be perceived as conceptual models, comprised by discreet stages that are used to identify "anticipated, typical, logical or desired evolution paths towards maturity" [5]. We observe that these models have been used in multiple areas to describe a wide variety of phenomena [10, 13, 31, 32, 40]. Maturity Models are sustained by the principle that people, organizations, functional areas, processes, etc., evolve, towards an enhanced maturity and following a development or growth process, which covers a number of different stages [57]. That is, Maturity Models are based on the theory of cyclic stages of growth, where the changes observed in an IS over the course of time occur in a sequential and predictable mode, covering a certain number of cumulative and hierarchically sequential stages, which can be described and linked to a specific level of maturity [7, 50, 57, 59]. In the same sense, Caralli & Knight [11] argue that maturity models provide organizations with a tool to address their problems and challenges in a structured way, offering both a reference point to evaluate their capabilities and a guide to improve them. Over the last four decades several maturity models have been proposed, with differences as to the number of stages, influencing factors and intervention areas. Each one of these factors identifies the characteristics that typify the focus of each maturity stage, that is, these factors work as reference descriptors or variables to characterize each stage and provide the necessary criteria to achieve a specific maturity stage [5].

1.1 Evolution of Maturity Models in IS Management Richard Nolan is considered the mentor of the IS maturity approach. Indeed, after studying/researching the use of IS in the biggest US organizations, Nolan proposed a


maturity model that initially included 4 stages [50]. Later, with a view to improve his first proposal, Nolan included two additional stages to the initial model [51]. In this second version, Nolan suggests that organizations start slowly in the Initiation stage, followed by a rapid spread in the use of ITs during the Contagion stage. Subsequently, the need for Control emerges, and this stage is followed by the Integration of different technological solutions. Data Management allows for development without increasing IS related costs and, finally, constant growth promotes the achievement of Maturity. Although this approach to the maturity models developed by Nolan, has been recognized as significantly ground breaking, it also raised a lot of debate and controversy within the scientific community. Several researchers have published studies that, on the one hand, validated and, on the other hand, expanded the model proposed by Nolan. Indeed, resulting from the researches in this field several researchers have proposed new models [e.g.: 15, 18, 27, 30, 39]. Amongst these new models proposed after the initial approach developed by Nolan, the most widely accepted, detailed and comprehensive is the Revised Model of Galliers and Sutherland [57, 10]. This model provides an improved perspective of how an organization plans, develops, uses and organizes an IS and offers suggestions towards an enhanced maturity stage. This model involves six stages of maturity and assumes that an organization can occupy different maturity stages in any given moment and be conditioned by influencing factors. Moreover, it presents the characteristics of the stages aligned with modern network organizations and offers a data collection tool to evaluate maturity [57]. More recently, after the model proposed by Galliers & Sutherland [18], other models have been resealed (e.g.: 4, 31, 45, 29], including a new Nolan model with nine stages of maturity [52], developed as an answer to the technological evolution in the IS field and its management. As to the field of IS Management, another solid example of a Maturity Model is the model developed by de Khandelwal & Ferguson [29], proposing nine stages of maturity and combining stages theory with Critical Success Factors. Notwithstanding, the model proposed by Galliers & Sutherland [18] is still perceived as the most complete and updated in IS management [59]. Additionally, these maturity models are still being used and implemented in multiple types of organizations and to different areas inside them. Mutafelija & Stromberg [44] refer that the concept of maturity has been applied to more than 150 areas inside IS. In fact, there are several examples of maturity models focused on different organization and IS areas, namely the maturity model for Intranet implementation, by Damsgaard & Scheepers [12]; the maturity model for ERP systems by Holland & Light [23]; and the CMMI maturity model for the software development process [63]. We can also mention maturity models for fields such as Software Management [3], Business Management [35], Project Management [9, 28], Project Portfolio and Program Management [43], Information Management [65], IS/ ICT Management [55], e-Business [15, 16, 19, 36], e-Learning [37], Knowledge Management [6, 38], BPM – Business Process Management [60], Enterprise Architecture [14, 46], etc. 1.2 Maturity Models for HIS Management Health related organizations, and more specifically Hospital IS Management organizations, are also increasingly adopting maturity models. This use is connected to


a growing provision of health care services based on electronic systems, supported by enhanced computer capacity and an increased ability to seize and share knowledge in a digital format. It is widely agreed that ISs offer significant opportunities for health care providers and health provision in general, as well as access to information required by users [59]. In this context, some maturity models emerged, namely the Quintegra Maturity Model for electronic Healthcare [64], proposed as a model that goes beyond the limits of an organization, incorporating every service linked to the medical process applied to each health care provider in each maturity stage. Another example of a maturity model in the health field is the HIMSS Maturity Model for Electronic Medical Record, which identifies different maturity stages in the Electronic Medical Record (EMR) of hospitals [20]. IDC (Health Industry Insights) has also developed a maturity model which describes the five stages of development in Hospital SIs. This maturity model has been used all over the world by IDC, both to evaluate the maturity of IS in hospitals and to compare maturity average differences between regions and countries in different continents [24]. To these models we can add the Maturity Model for Electronic Patient Record, directed to the system that manages every patient related information, that is, a system that manages the EPR (Electronic Patient Record) [54] and the maturity model for PACS by Wetering & Batenburg [67]. National health services from several countries have also started to develop and adopt Maturity Models. That is the case of the model created by the National E-health Transition Authority of Australia (NEHTA) [47] and baptized Interoperability Maturity Model (IMM). This model focuses on interoperability associated with technical, informational and organizational capabilities of the different players involved in health care services. Another example concerns the Maturity Model of the NHS Infrastructure Maturity Model (NIMM) [48]. This is a maturity evaluation model that helps NHS organizations carry out an objective self-assessment in terms of technological infrastructures.

3 The Problem Health care institutions and governmental organizations are starting to understand that the reasons underlying a certain inadequacy in the management of health processes directly relates to infrastructural limitations and their inefficient management [56, 64]. An analysis to the current health context clearly reveals the weight of the technological transition problem [64]. Moreover, operational information technologies have increased in complexity to answer the demands of the sector. This increase in complexity, in its turn, led to the integration of several new enterprise integration systems, processes and approaches, and the emergence of new companies providing services in this field. Consequently, many underdeveloped products and services are being consumed by HISs undergoing a process of change and demanding, more than ever, a degree of performance and effectiveness that will answer their needs. In this scenario, several questions that require a convincing answer emerge: • How can we know if we are doing a good job when managing these changes and monitoring their progress on an ongoing basis? • How can we manage the interactions of systems and processes in constant evolution?


•

How can we manage the impact of low interoperability, security, reliability, efficiency and effectiveness processes? • How can we evaluate the impact of current clinical and hospital software applications in the maturity development of their respective IS? We observe that the benefits brought by modern technology to the health field, and supported by better methods and tools, cannot be obtained via undisciplined and chaotic processes [21, 22]. That is why we believe that IS Management in health organizations must be carried out based on maturity models. Several maturity models have been proposed in the course of time, for personal evolution purposes, for the general evolution of organizations and for the evolution of the IS Management task in particular. The differences in these models lie specifically in the number of stages, evolution variables and focus areas [40, 59]. Each of these models identifies certain characteristics that typify the target of different growth or maturity stages and are implemented in different organizations. Where health related organizations are concerned, several maturity models are also proposed. Notwithstanding the specificities of these models that distinguish them from the models of other areas, these are still in an early development stage [40, 59]. In the research that was carried out, we observed that the models pertaining to the health field are poorly detailed, do not provide maturity measuring tools and do not structure the characteristics of maturity stages according to influencing factors. This reality offers an opportunity for the development of new maturity models focused on IS management in the health field that are capable of filling the previously identified gaps. Within the universe of the maturity models that we know, we believe that the model of maturity stages reviewed by Galliers & Sutherland [18] can serve as an inspiration and reference, both to define influencing factors and to develop a measuring instrument for Hospital IS maturity. Additionally, the very concept of Maturity Model is not devoid of criticism. For instance, Pfeffer & Sutton [53] argue that the purpose of maturity models is to identify a gap that can be filled with subsequent improvement measures. However, most of these models fail to describe how to effectively carry out these actions, as the closing of such gaps can be extremely difficult to illustrate. The strongest point of criticism, where maturity models are involved, is their weak theoretical basis [8]. Most of these models are based on "best practices" or "success factors” connected with processes from organizations that have shown favorable results. Therefore, although these practices are compatible with the maturity model, they provide no guarantee as to the success of the organization. There is no agreement surrounding the "real path” that will ensure a positive result [42]. According to deBruin & Rosemann [13] the reasons for the, sometimes, ambiguous results obtained with the maturity models stem from the insufficient testing of the models in terms of validity, reliability and generalization, as well as from the lack of documents addressing the design and development process behind this type of model. For this reason, it is fundamental to describe the work underlying the development of a maturity model based on an approach sustained by DSR (Design Science Research) principles.

4 Research Question and Objectives After describing the problem we elaborate the following research question:


•

What is the best model and respective stages of maturity to be applied in HIS Management? From this research question we can pose more specific questions: • What influencing factors, associated with the stages of maturity, are perceived as being fundamental in the health care field by stakeholders? • How can these factors be determined, quantified and integrated in the context of HIS maturity stages? To answer these questions we defined the following objectives: • Through a systematic literature review, the identification of the main maturity models adopted in IS and HIS Management and characteristics of their different stages; • The identification and characterization of a set of influencing factors that can be used in different stages of maturity of HISs; • The proposal of a conceptual model that will allow us to put into context, classify and describe the factors that influence the maturity stages of the HIS with the application of the Delphi Method; • The validation of the proposed conceptual model and methodology through the adoption of statistical techniques applied to the results obtained with the Delphi Method. • The development of an automatic tool that will allow us to identify the stage of maturity of a given HIS and the influencing factors that must be improved in order to achieve an enhanced stage of maturity.

5 Research Methodology In the context of a PhD project or any other type of research work, defining a methodological approach is fundamental for the researcher, in a way that it will allow him to put into context the activities to be developed, define how the work will be carried out and explain how the results will be measured and evaluated, with a view to validate the research. A research process entails the collection, analysis and interpretation of data, with a view to understanding a certain phenomenon. This process should be systematic and, within a certain framework and established guidelines, involve the definition of objectives, the management of data and the communication of conclusions [68]. Bearing in mind the research questions and the goals established for investigation project, the adopted approach will include the following methods: Systematic Literature Review; and Delphi Method.

5.1 Literature Review The starting point for the follow up of this research work will be the systematic review of the available literature in the field under study. A systematic literature review is an essential step in any research project, to the extent that it provides the researcher a solid basis to make the knowledge advance. On the other hand, a systematic literature review promotes the development of theories, the identification of fields where a


multiplicity of research projects, and the identification of areas that need to the studied [66]. In this project, the systematic literature review will try to reach the following research goals: Identify the main maturity models adopted in IS Management and describe its different stages; Review the State of the Art in HIS Maturity Models; Identify the success factors and the limitations of this type of Maturity Models; Analyze and debate different options for the development of a conceptual maturity model in the IS field; Identify and characterize a set of factors that influence maturity, to be adopted in each maturity stage of an HIS, and which will serve as a basis for our Delphi method study. By the end of the systematic literature review, one of the most important results, besides a detailed description of the state of the art concerning HIS maturity models, will be the identification of an initial set of influencing factors connected to different stages of the maturity model. This set of factors influencing maturity will be particularly useful for a subsequent input in the first round of the Delphi Method. 5.2 Delphi Method According to the herein presented methodological approach, one of the methods that will be adopted in this research project is the Delphi method. This method supports decision making based on the opinions and contributions of participants who are experienced or specialized in the field under study [33]. Moreover, this method has been widely accepted over the last decades in the IS domain, as its application in several studies demonstrates [e.g.: 41, 49, 61]. In this project, the application of the Delphi Method will have the purpose of identifying and describing the main influencing factors that are to be used in different stages of the maturity models. Based on the results obtained from the participants, we expect to develop and propose a conceptual model that will allow us to put into context and describe the factors influencing the stages of maturity of a HIS. The selection of the Delphi panel is usually one of the first stages in any study of this nature. Although there are no rules underlying the definition of a panel, according to Scheele [62], three types of participants must be included in order to achieve a good variety of opinions, namely: • Stakeholders: the elements who are involved and interested in the field under study and who are directly affected; • Experts: those who are experienced or specialized in a relevant field; • Facilitators: those who have the power to clarify, organize and synthesize. Additionally, and in certain circumstances, we may consider other types of participants, people who can provide alternative perspectives and points of view [62]. The predominance of each type of participant will depend on the characteristics of the study to be developed. In the context of the present research work, we intend to invite the following types of participants for our study panel: CEOs and CIOs of health organizations with implemented HISs; HIS projects leaders; Consultants and Professionals in the HIS field. As to the number of panel participants and according to the literature, there is no ideal size for a panel; a number of studies recommend a number between 10 and 50 participants [61]. In this project the definition of the panel will mainly depend on the contacts established and the invited entities.


6 Final Remarks In the present paper we put forth the initial stage of an research aimed at the development of an comprehensive maturity model for hospital IS management, justified by several existing limitations in current maturity models in the health field. A future work will involve systematic reviews of the available literature concerning maturity models for information systems management in general and hospital information systems management in particular, which will allow us to identify a set of potential influencing factors that must be considered during the initial development stage of the maturity model, to be defined with the Delphi Method. The resulting maturity model will be validated in a group of hospital organizations to be defined, and subsequently we will develop an automatic tool that will allow us to identify the stage of maturity of a given HIS and the path that must be pursued towards a growing maturity.

References 1. 2. 3.

4. 5. 6. 7. 8. 9. 10.

11. 12. 13.

14. 15. 16.

Amaral, L. e Varajão, J., Planeamento de Sistemas de Informação, 4th Ed. FCA (2007). Amaral, Luís, & Santos, Maribel Yasmina., Modelos de Estádios de Crescimento, Sistemas de Informação, 7, 41-59 (1997). April, A., Abran, A. & Dumke, R., Assessment of software maintenance capability: A model and its architecture, In Proceedings of the 8th European Conference on Software Maintenance and Reengineering (CSMR2004) (pp. 243-248), Los Alamitos CA: IEEE Computer Society Press (2004). Auer, Timo, Beyond IS Implemention: A Skill-Based Aproach to IS Use, Paper presented at the 3rd European Conference on Information Systems, Athens, Greece (1995). Becker, J., Knackstedt, R. & Pöppelbuß, J., Developing Maturity Models for IT Management, Business & Information Systems Engineering, 1 (3), pp. 213-222 (2009). Berztiss, A.T., Capability maturity for knowledge management. In: DEXA Workshop, IEEE Computer Society, 162-166 (2002). Bhidé, Amar V., The Origin and Evolution of New Businesses, New York, NY, USA: Oxford University Press (2000). Biberoglu, E., & Haddad, H., A survey of industrial experiences with CMM and the teaching of CMM practices. Journal of Computing Sciences in Colleges, 18(2), (2002). Brookes, N. & Clark, R., Using Maturity Models to Improve Project Management Practice. POMS 20th Annual Conference, May 1 to May 4, 2009. Orlando Florida USA (2009). Burn, J. M. A., Revolutionary staged growth model of information systems planning, Proceedings of the International Conference on Information Systems, Vancouver, British Columbia, Canada, pp. 395-406 (1994). Caralli, R., & Knight, M., Maturity Models 101: A Primer for Applying Maturity Models to Smart Grid Security, Resilience, and Interoperability. Software Engineering Institute, Carnegie Mellon University (2012). Damsgaard, J., & Scheepers, R., Managing the crises in intranet implementation: a stage model, Information Systems Journal, 10(2), 131–149 (2000). doi:10.1046/j.1365-75.2000. 00076.x deBruin, T., & Rosemann, M., Understanding the main phases of developing a maturity assessment model. Proceedings of the 16th Australasian Conference on Information Systems, Sydney, Australia (2005). Doc, IT Architecture Capability Maturity Model, Department of Commerce, USA Government Introduction, (2003). Earl, M. J., Management Strategies for Information Technologies, Upper Saddle River, NJ: Prentice Hall (1989). Earl, M. J., Evolving the EBusiness, Business Strategy Review, 11(2), 33-38 (2000).


17. Freixo, J. & Rocha, Á., Arquitetura de Informação de Suporte à Gestão da Qualidade em 18. 19.

20. 21. 22.

23. 24. 25.

26. 27. 28. 29. 30. 31. 32.

33. 34. 35. 36. 37. 38. 39. 40. 41.

Unidades Hospitalares. RISTI - Revista Ibérica de Sistemas e Tecnologias de Informação, 14, 1-18 (2014). doi: 10.17013/risti.14.1-18 Galliers, R. D., & Sutherland, A. R., Information systems management and strategy formulation: the ‘stages of growth’ model revisited, Journal of Information Systems, 1(2), 89–114 (1991). doi:10.1111/j.1365-2575.1991.tb00030.x Gardler, R. & Mehandjiev, N., Supporting Component-Based Software Evolution. In Aksit, M., Mezini, M. & Unland, R. (Eds.), Objects, Components, Architectures, Services, and Applications for a Networked World, Series: Lecture Notes in Computer Science, 2591 (pp. 103-120), Springer Verlag (2003). Garets, D., & Davis, M., Electronic Medical Records versus Electronic Health Records: Yes, there is a difference, Chicago, IL: HIMSS Analytics (2006). Gonçalves, J. & Rocha, Á., A decision support system for quality of life in head and neck oncology patients. Head & Neck Oncology, 4(3), 1 – 9 (2012). doi: 10.1186/1758-3284-4-3 Gonçalves, J., Silveira, A. & Rocha, Á.. A platform to study the quality of life in oncology patients. International Journal of Information Systems and Change Management, 5(3), 209220 (2011). doi: 10.1504/IJISCM.2011.044501 Holland, C., & Light, B., A stage maturity model for enterprise resource planning systems, The Data Base for Advances in Information Systems, 32(2), 34–45 (2001). Holland, M., Piai, S., & Dunbrack, L. A., Healthcare IT Maturity Model: Western European Hospitals - The Leading Countries (Tech. Rep. No. HI210231), Framingham, MA: IDC Health Insights (2008). Holland, M., Piai, S., & Dunbrack, L. A., Healthcare IT Maturity Model: Western European Hospitals - The Leading Countries (Tech. Rep. No. HI210231), Framingham, MA: IDC Health Insights (2008). NHS Infrastructure Maturity Model. http://www.connectingforhealth.nhs.uk/systemsandservices/nimm (Retrieved in Set/2014). Huff, Sidney L., Munro, Malcolm C., & Martin, Barbara H., Growth Stages of End-User Computing, Communications of the ACM, 31(5), 542-550 (1988). Kerzner, H., Using the Project Management Maturity Model: Strategic Planning for Project Management (2nd ed.). New York: John Wiley & Sons (2005). Khandelwal, V. & Ferguson, J., Critical Success Factors (CSFs) and the Growth of IT in Selected Geographic Regions, Proceedings of 32nd Hawaii International Conference on Systems Sciences (HICSS-32), USA (1999). King, J. & Kraemer, K., Evolution and organizational information systems: An assessment of Nolan´s stage model, Communications of de ACM, 27(5), 466-475 (1984). King, W. R. & Teo, T. S.H., Integration Between Business Planning and Information Systems Planning: Validating a Stage Hypothesis, In: Decision Sciences, 28(2), (1997). Kohlegger, M., Maier, R. & Thalmann, S., Understanding Maturity Models Results of a Structured Content Analysis Proceedings of I-KNOW and I-SEMANTIC 09. Graz, Austria: University of Innsbruck, School of Management, Information Systems, Austria (2009). Landeta, J., Current validity of Delphi method in social sciences, Institute of Applied Business Economics of University of the Basque Country (UPV/EHU), Spain (2005). Lavoie, D., & Culbert, S., Stages in organization and development. Human Relations, 31, 417-438 (1978). Levin G. & Nutt, H., Achieving Excellence in Business Development: The Business Development Capability Maturity Model (2005). Ludescher, G. & Usrey, M., Towards an ECMM (E-Commerce Maturity Model). In Proceedings of the First International Research Conference on Organizational Excellence in the Third Millennium. Estes Park: Colorado State University (2000). Marshall, S., E-Learning maturity model (2007). http://www.utdc.vuw.ac.nz/research/emm/ (Retrieved in Set/2014). Maybury, M.T., Knowledge Management at the MITRE Corp (2002) http://www.mitre.org. McKenney, James L., & McFarlan, Franklin W., The Information Archipelago – Maps and Bridges. Harvard Business Review, 60(5), 109-119 (1982). Mettler, T., A Design Science Research Perspective on Maturity Models in Information Systems. University of St. Gallen, St. Gallen (2009). Miaskiewicz, T. and Kozar, K., The Use of the Delphi Method to Determine the Benefits of the Personas Method – An Approach to Systems Design, Proceedings of the Fifth Annual Workshop on HCI Research in MIS, Milwaukee, 50‐54 (2006).


42. Montoya-Weiss, M. M., & Calantone, R. J., Determinants of new product perfomance: A

review and meta-analysis. Journal of Product Innovation Management 11(5), (1994). 43. Murray, A., Capability Maturity Models - Using P3M3 to Improve Performance. Vol 2;

Issue 0616-01-12 (2006). Available: www.outperform.co.uk (Retrieved in Set/2014). 44. Mutafelija, B. and H. Stromberg., Systematic process improvement using ISO 9001:2000

and CMMI. Boston: Artech House (2003). 45. Mutsaers, E., Zee, H. & Giertz, H., The Evolution of Information Technology, Information

Management & Computer Security, 6(3), 115-126 (1998). 46. Nascio, NASCIO Enterprise Architecture Maturity Model, Version 1.3, National

Association of State Chief Information Officers, December (2003). 47. NEHTA, NEHTA Interoperability Maturity Model. 2007 ed. Level 25, 56 Pitt Street,

Sydney, NSW, 2000, Australia.: National EHealth Transition Authority Ltd. (2007). National Infrastructure Maturity Model (Online) (2011), Available: http://www.connectingforhealth.nhs.uk/systemsandservices/nimm (Retrieved in Set/2014). Niederman, F., Brancheau, J.C. and Wetherbe, J.C., Information Systems Management Issues for the 1990s, MIS Quarterly, 15(4), 475‐500 (1991). Nolan, R., Managing de computer resource: a stage hypotesis, Communications of de ACM, 16(7), pp. 399-405 (1973). Nolan, R., Managing the crisis in data processing, Harvard Business Review, 57(2), (1979). Nolan, R. & Koot, W., Nolan Stages Theory Today: A framework for senior and IT management to manage information technology, Holland Management Review, 31, (1992). Pfeffer, J., & Sutton, R., Knowing "what" to do is not enough: Turning knowledge into action. California Management Review, 42(1), 83-108 (1999). Priestman, W., ICT Strategy 2007-2011 for The Royal Liverpool and Broadgreen University Hospitals NHS Trust. Trust Board Meeting 6th November (2007). Renken, J., Developing an IS/ICT manage¬ment capability maturity framework. In: Re¬search conference of the South African Insti¬tute for Computer Scientists and Information Technologists (SAICSIT), Stellenbosch, 53–62 (2004). Rocha, Á. & Rocha, B., Adopting nursing health record standards. Informatics for Health & Social Care, 39(1), 1-14 (2014). doi: 10.3109/17538157.2013.827200 Rocha, Á. & Vasconcelos, J., Os Modelos de Maturidade na Gestão de Sistemas de Informação, Revista da Faculdade de Ciência e Tecnologia da Universidade Fernando Pessoa, 1, 93–107 (2004). Rocha, Á., Maturidade da Função Sistema de Informação: Teoria de Estádios, Modelos e Avaliação. Porto, Portugal: Universidade Fernando Pessoa (2002). Rocha, Á., Evolution of Information Systems and Technologies Maturity in Healthcare. International Journal of Healthcare Information Systems and Informatics (IJHISI), 6(2), 2836 (2011). doi:10.4018/jhisi.2011040103 Rosemann, M. & deBruin, T., Business Process Management Maturity - A Model for Progression. In Proceedings of the 13th ECIS, May, Regensburg (2005). Santos, L.D.d., Factores Determinantes do Sucesso de Adopção e Difusão de Serviços de Informação Online em Sistemas de Gestão de Ciência e Tecnologia, Tese de Doutoramento, Departamento de Sistemas de Informação, Universidade do Minho (2004). Scheele, D.S., Reality Construction as a Product of Delphi Interaction, in: The Delphi Method: Techniques and Applications H.A. Linstone and M. Turoff (eds.), http://is.njit.edu/pubs/delphibook/ (Retrieved in Set/2014) (2002). SEI Software Engineering Institute, CMMI® for Development, Version 1.3, Improving processes for developing better products and services, (Tech. Rep. No. CMU/SEI-2010TR-033), Carnegie Mellon University (2010). Sharma, B., Electronic Healthcare Maturity Model (eHMM), India: Quintegra (2008). Venkatesh, V. et al., User acceptance of information technology: Toward a unified view, MIS Quarterly, 27(3), 425-478 (2003). Webster, J. & Watson, R.T., Analyzing the past and prepare for the future: writing a literature review, MIS Quarterly, 26(2), xiii‐xxiii (2002). Wetering, R., & Batenburg, R., A PACS maturity model: A systematic meta-analytic review on maturation and evolvability of PACS in the hospital enterprise. International Journal of Medical Informatics, 78, 127–140 (2009). doi: 10.1016/j.ijmedinf.2008.06.010 Williams, C., Research Methods, Journal of Business & Economic Research, 5(3), (2007).

48. NHS, 49. 50. 51. 52. 53. 54. 55. 56. 57.

58. 59. 60. 61.

62. 63.

64. 65. 66. 67.

68.
