Lessons Learned from England’s National Electronic Health Record Implementation: Implications for the International Community Ann Robertson
Kathrin M. Cresswell
Aziz Sheikh
eHealth Research Group eHealth Research Group eHealth Research Group Centre for Population Health Sciences Centre for Population Health Sciences Centre for Population Health Sciences The University of Edinburgh The University of Edinburgh The University of Edinburgh 0044 131 650 9459 0044 131 650 9241 0044 131 650 4151
[email protected]
[email protected]
ABSTRACT
[email protected]
General Terms
Background: National electronic health record (EHR) programs are increasingly being pursued across the world with the aim of improving the safety, quality and efficiency of healthcare. Despite significant international investments, and particularly in the light of reported “failures”, there is surprisingly little evidence on the specific and potentially transferable factors associated with the planning and execution of large-scale EHR implementations. England embarked on a National Program in 2002, characterized by “top-down”, central procurement of a few, standardized EHR systems. Objectives: To evaluate the national implementation and adoption of EHRs in English hospitals and derive lessons for this and other national EHR programs. Design: We conducted a qualitative case study-based longitudinal evaluation drawing on sociotechnical principles. Setting: Data were collected from 12 “early adopter” hospitals across England. Data sources: Our dataset consisted of 431 semi-structured interviews; 590 hours of observations; 334 sets of notes from observations, researcher field notes and notes from conferences; 809 hospital documents; and 58 national and regional documents. Results: A range of factors emerged as important. These included software characteristics and user involvement in shaping technology; realistic timelines, balancing the national EHR vision and stakeholder expectations; relationship building and communication; balancing national progress with allowing local accommodation; and maintaining central direction whilst permitting degrees of local autonomy. Conclusions: It is not possible to be prescriptive for achieving “successful” national EHR implementations. Nonetheless, we identify dimensions likely to be of greater significance than others, in a range of national contexts. We argue that design, based on users’ requirements, and accommodation of the technology in the healthcare setting need to occur on a small-scale first before building out to satisfy organizational, local health economy and national needs, and that this needs time. Our results will we hope offer evidence to inform national strategies for large-scale and expensive EHR ventures.
standardization
Keywords Electronic health records; national implementation; sociotechnical evaluation
1. INTRODUCTION In pursuing more efficient and safer healthcare to larger populations with increasingly complex conditions, there is now growing international interest in the potential of information technology (IT) (1;2). Costly and multifaceted electronic health record systems (EHRs) have been central to these efforts (3-7). These consist of longitudinal patient records that can be shared across care settings and associated clinical and management functionalities. However, to date many implementations of such systems and associated benefits have been achieved on relatively small scales. Here, systems have been extensively customized to local contexts. Challenges even in these smaller scale implementations have in the main related to difficulties of systems embedding with clinical and organizational workflows (2;8;9). In contrast to these smaller implementations, larger scale national ventures pose appreciably more challenges. But they also offer the potential for significant benefits, particularly in relation to secondary uses and larger scale data sharing. National EHR implementations are therefore internationally pursued. However, no nationally interoperable system or agreed-upon “best” approach to realizing benefits exists as yet. Strategies for national EHR implementations vary from connecting local systems in individual care settings to implementing nationally procured solutions (1;2;10). Nationally procured solutions are often designed with interoperability considerations from their inception. England has attempted this by procuring commercial EHR systems nationally and implementing these centrally in individual care settings (11-14). As there is limited evidence with regard to any large scale implementation approaches, it is vital that other countries learn from these experiences. This will help to ensure that the chances of realizing benefits are maximized. In the present paper, we focus on key lessons from the English experience of implementing nationally procured EHRs in hospitals. In doing so, we draw on our work evaluating the introduction of these systems in selected care settings.
Categories and Subject Descriptors D.2.11 Computer Systems Organization [General]: Systems Application Architecture Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. IHI’12, January 28–30, 2012, Miami, Florida, USA. Copyright 2012 ACM 978-1-4503-0781-9/12/01...$10.00.
685
abreast of formally planned changes and actual developments on the ground. We assigned lead researchers who collected and analyzed data in individual case study sites in order to allow immersion in local contexts and gain an in-depth insight into local contingencies. During data collection, emerging findings informed subsequent research activities, which helped to refine and test emerging hypotheses (24;25). This involved feeding back emerging findings to local implementation team members as the study progressed. Data collection continued until no new major themes emerged i.e. the point at which researchers felt they had gained sufficient insights into the complexities of each case during the time they were in the field. Data analysis was informed by sociotechnical principles, paying careful attention to how social and technical dimensions evolved over time. It was facilitated by regular analysis workshops amongst the wider team to discuss emerging findings and explore new potential avenues for investigation (15;16). We combined deductive thematic analysis (based on a substantive review of the empirical literature) with inductive thematic analysis to allow new themes to emerge from the data (15;26;27). After analyzing each case study separately, and integrating individual data sources within cases, we aggregated findings across cases and integrated these with findings obtained from the wider national environment (i.e. interviews with stakeholders outside the immediate hospital environment, and national documents).
2. METHODS We conducted a qualitative, longitudinal, real-time evaluation of the introduction of national EHRs into English hospitals drawing on sociotechnical principles that emphasize the mutually shaping relationship between social and technical factors (15;16). Data were collected from September 2008 until February 2011. Our detailed methods are reported elsewhere (17), but we give a brief overview below. We purposefully sampled 12 hospitals across England being amongst the first to implement nationally procured EHR software systems. We conceptualized these as case study sites in order to gain an insight into local processes and consequences of systems introduction (18-20). We used these insights to draw transferable lessons for other contexts (i.e. settings yet to implement EHR systems). In the light of the limited number of implementations taking place at the time of our data collection, we had to be somewhat opportunistic, but where possible sampled sites of varying demographics, software solutions and geographical locations (see Box 1) (21). Box 1. Summary of case study sites Eight acute, three mental health, and one community setting Six teaching sites and six non-teaching sites Five more autonomous sites Three different software systems across sites Locally, initial contacts were made with the Head of IT, who acted as the first point of contact for snowball sampling other local stakeholders including healthcare professionals who were not part of the implementation team (doctors, nurses, allied healthcare professionals), implementation team members, managers and administrative staff (22;23). Each interviewee was asked to recommend other potential interviewees and differing viewpoints were actively sought. In order to gain an insight into wider national processes surrounding the EHR introduction, we also opportunistically sampled other stakeholders not directly connected to case study sites but involved in the national implementation strategy. These were approached through recommendations from interviewees and through personal contacts. They included software developers, national implementers, independent sector representatives and governmental stakeholders.
3. RESULTS We collected data through 431 semi-structured interviews. In addition, we obtained 334 sets of notes from 590 hours of observations, researcher field notes and notes from conferences. These were complemented by 809 hospital documents and 58 national and regional documents. Overall we found that accommodation and technological design needed to occur on a local/organizational level first before satisfying organizational, local health economy and national needs. In doing so, we have identified five overarching themes, which we will discuss in the paragraphs below.
3.1 Software characteristics and user involvement in shaping technology Our results indicated that the most important pre-requisite for implementing complex EHRs was the existence of software that was usable, or could be ‘made usable’, and modified over time to suit the evolving needs of a variety of user groups. In case studies where software design reflected user needs or was customized to do so over time, users were more motivated and early benefits were more likely to be realized. Conversely, where system design was felt to be inadequate, this often resulted in increased workloads for users, who often spent a long time browsing the system to find information they needed due to slow loading of screens. In some cases there were also perceived adverse consequences for organizational functioning as clinical users stated that administrative tasks increased, whilst clinical time was compromised. Therefore, in some instances, patient throughput was reduced.
Our main method of data collection consisted of semi-structured audio-taped interviews reflecting the explorative nature of our research. The focus of these discussions was to investigate stakeholders’ concerns and experiences in relation to using and implementing national EHR systems. Where possible, interviews were conducted at two time points, with an approximately six month gap in between, in order to capture changes over time (e.g. whether a certain amount of embedding of the software had occurred and/or attitudes had changed). In doing so, we actively attempted to explore varying perspectives of a range of stakeholders. Interviews were complemented by observations of local strategic implementation meetings and use of the software in the respective care settings. This allowed exploring of informal processes and “real world” (as opposed to reported) issues and how these were tackled locally (e.g. how users would react to a slow-loading screen and what consequences this had for the running of clinics). Researchers also collected local documents (including project initiation and lessons learned reports) and national documents (including governmental reports and reviews of progress relating to the national EHR systems) in order to keep
Lack of system usability was overall particularly evident in systems that had been untested and not previously implemented (of which there was one out of three), whilst systems that had routinely been implemented in other contexts were often more usable. The only way to achieve system usability seemed to be by consulting users and incorporating their suggestions into system design. This was often done through designated representatives
686
implementations locally: “…this kind of slightly difficult contractual relationship where unless you throw it over the wall in the right format it’s not our problem… Yes and I actually know its not because your company is only going to make money if this…system is credible for the rest of the NHS [National Health Service] and it doesn’t look credible while you’re being very careful about whether you’re in it with us or not you know, you need to be over on our side of the fence, its needs to worry you as much as it worries us about getting this right and that has been quite a difficult...” (Interview, Manager) Other stakeholders of relevance included associated commercial and governmental bodies as well as IT professionals and healthcare workers themselves. These different groups were often not used to working together. Consequently, communication was often complicated with many stakeholders feeling that other parties were “speaking an entirely different language” to themselves. This was particularly apparent when healthcare professionals attempted to communicate their needs to managers and system developers. Again, this was slightly less pronounced in smallscale implementation settings, where designated members of the implementation team had established good personal relationships with software users.
who fed back user concerns to the implementation team. Sites where this was achieved often consisted of small-scale implementations but tended to have more motivated users, whilst sites where users did not feel listened to tended to have increasingly stressed and frustrated users.“But I’ve had a lot of frustrations with it and continue to have. I don’t really feel, cause we’ve handed information through the system to the [name of system] representative [each team has one of these to feed back problems], I don’t really feel any changes have been made really, I don’t know no, there was a slight change to the template.” (Interview, Healthcare Professional)
3.2 Realistic timelines, expectations and balancing the national EHR vision Across softwares and settings, we further found that stakeholders often stated that the ambitious implementation timelines hampered progress, with expectations far exceeding what was realistically achievable. The ambitious vision of nationally shared detailed EHRs over time changed to many stakeholders believing that it was more realistic to only share certain demographic information: “What would have been ideal is the shared care record vision which was one system like [name of system] … until six to nine months ago I still believed that we could deliver that. […] I think the only way of doing that now is an expanded summary care record or something like that…” (Interview, Manager) For example, users were often disappointed with the observed benefits of using national software that in many cases was perceived to slow down individual work practices. This was more pronounced with some software packages than others, but particularly so when users could not conceptualize the exact functionality (as some systems were still in development and could not be demonstrated). There was therefore uncertainty as to what benefits could be expected in the future and it was difficult to plan implementation activities. This was exacerbated by, in some instances, users reporting to have been presented with a system that looked different than the one they had seen in practice.
3.4 Balancing national progress with allowing local accommodation We further observed a tension between the need to show implementation progress locally and an incremental implementation approach to allow for local accommodation of the technological change to occur. The best way of helping to accommodate the changes brought about with a new EHR was characterized by an incremental implementation approach by most organizational stakeholders, although replacing patient administration functionality was by nature somewhat larger-scale. Pressure to show progress too quickly, on the other hand, seemed to result in unintended consequences at individual and organizational levels, such as the need to adopt workarounds and resulting knock-on effects. These included for example entering data into the system at a later point whilst taking notes on paper during the clinical encounter because the system was viewed as too time-consuming and clinical responsibilities took priority. As a result, the computer system was not as up-to-date as paper records. We found this to be the case across softwares and settings:“…it’s very frustrating when if you’re in a clinic with time pressures, you’ve got a patient sat there, you need to be getting on to treat them, you know, you might have another few in the clinic waiting… So what we’re doing you see there is writing it all down and going into [the computer system] when we have time later.” (Interview, Healthcare Professional)
3.3 Relationship building at all levels We also found that aligning the interests of diverse stakeholder groups was important throughout the national implementation. Relevant stakeholders did not only include those within the immediate hospital environment but also those outside the case study sites, such as developers, who incorporated suggested software changes. However, across settings we found that local relationship building between suppliers and users was often inhibited by centrally managed contracts resulting in delays in incorporating locally requested software changes due to complex bureaucratic processes. “Strange, very strange, and in some ways I think that’s what drives some of the frustration probably from both parties sitting at the far end is, you know, if we can’t engage with the customer how do we know we’re delivering something that’s going to be beneficial and vice versa, you know, if they’ve got to go through seven loops to get something fixed and then you get a question coming back, why do you want that, because you’ve not got that direct communication the frustration increases.” (Interview, Developer) Nationally negotiated contracts largely excluded individual hospitals. Therefore, implementation team members had limited power for making the systems usable in each environment due to national arrangements. Those that delivered the software were often more concerned with receiving monetary rewards as opposed to high quality
3.5 Central guidance and local autonomy Despite the need for new EHR systems to satisfy user and organizational needs, our results have also pointed to the importance of considering standards for larger scale interoperability. However, these concerns seemed to be secondary to most organizational stakeholders as local accommodation needed to occur first. Conversely, a primary focus on interoperability was felt to compromise local usability as the design of the software in many cases lacked tailoring to local needs. Nevertheless, most also acknowledged that a certain amount of political guidance and setting of standards was important for a national implementation in order to guide organizations, whilst imposing national systems was felt to be
687
unlikely to work: “… the framework and the technology…still needs applications and modifications to meet localized requirements… so there’s still a need for services deployment management and also certification of specific tools that would be used by various organizations.” (Interview, Independent Sector) Most participants, irrespective of setting and software functionality, also argued that a central authority overseeing implementation should focus on connecting local and natural groupings of healthcare organizations first in order to bring local benefits, before considering larger scale interoperability and data sharing.
Box 2. Summary of lessons learned from the English venture
The initial focus should be on making the software usable. This should be informed by users. Expectations need to be realistic. Benefits may take a long time to realize, and systems may initially slow down work practices and impact adversely on organizational functioning. Implementing EHRs in an ongoing process and needs to be expected to continue in line with changing local and national needs as well as political landscapes. Efforts should ideally begin with the user, before moving on to more general organizational and national requirements. A balance between customization and standardization is vital. Aligning efforts of various stakeholder groups is necessary for an integrated approach to the vision. Appropriate time and resources need to be allocated nationally to allow the process of local accommodation to occur. Centrally negotiated contracts may inhibit these desired developments. It is also important to note that we are reporting on the early stages of implementation and adoption only and can therefore not claim transferability of findings for the later stages of the English venture.
4. DISCUSSION Our findings echo many of the factors associated with EHR implementation and adoption in the literature (28-38), but provide unique insights into the worldwide first attempt to implement nationally procured software. The results therefore build on the existing literature by illustrating that a central management model and a primary focus on national integration can undermine the best efforts to make a system usable and fit a particular environment. This was apparent by systems lacking customizability and therefore not fulfilling the needs of individual users as well as organizations. As a result, the centralized implementation model has, over time, increasingly allowed more local input in system choice (39;40). We have therefore argued that accommodation and technological design needs to occur on a local/organizational level first and begin with the user. Only when local factors are attended to will a system be able to satisfy organizational (e.g. management and small-scale information sharing), local health economy (e.g. local information sharing) and national needs (e.g. interoperability and secondary uses). However, in large-scale national ventures there is a danger that an initial focus on interoperability may cloud this essential consideration of user involvement in shaping technological developments. A word of caution is however needed at this stage as there were clearly reasons for nationally procuring EHR solutions, particularly in relation to cost savings through largescale contracting and anticipated benefits associated with interoperability of standardized systems (41). The main danger then is that if efforts are too localized, there may be no coherent approach to implementation, resulting in potentially compromised interoperability. Arguably, a balance needs to be struck between both standardization and localized developments and there appear to be trade-offs resulting from a too concentrated focus on either. For example, despite nationally led strategies resulting in a likely optimization of technical integration and interoperability, there is a danger that these systems are not accepted and used by endusers as local motivations for use may be neglected. Conversely, if the emphasis is solely on building systems arising from local need, these are likely to be used, but major issues may arise when attempting to integrate systems on a larger scale. In the light of our findings, and the need to build on user informed system design, the second extreme scenario would, however, be preferable, as integration of systems appears secondary and local benefits may still be realized even if national systems integration is initially not achieved. Despite these important insights, our case-study based design may mean limited transferability of findings to other settings, but this was not our aim. We have instead focused on drawing transferable lessons for international efforts of implementing large-scale EHR systems (summarized in Box 2). Many of these are in line with the existing literature (2838).
5. CONCLUSIONS We have summarized the lessons learned from the English attempt to implement national-scale centrally procured EHR systems and outlined lessons that can be learned from these experiences to inform other international ventures. In doing so, we have argued that initial efforts should focus on designing technology that is fit for local use and satisfies user and organizational needs. Important, but secondary to these developments should be large-scale interoperability. We hope that the lessons drawn from our work and summarized in Box 2 will inform future efforts and help to ensure that large-scale EHR implementations proceed as smoothly as possible with the highest chances of fulfilling the promise of improving the safety and quality of care.
6. ACKNOWLEDGMENTS We are very grateful to the participating hospitals for supporting this work and to all interviewees who kindly gave their time. We also acknowledge the work by our colleagues in the evaluation team. KC was supported by an MRC studentship. We have had helpful support from colleagues at the NHS Connecting for Health Evaluation Programme and our Independent Project Steering Committee. We acknowledge the support of the National Institute for Health Research, through the Comprehensive Clinical Research Network. We are also grateful to the helpful feedback of the five expert reviewers.
7. REFERENCES
688
(1)
Catwell L, Sheikh A. Information technology system users must be allowed to decide on the future direction of major national IT initiatives. Inform Prim Care 2009; 17(1):1-4.
(2)
Protti D. Comparison of information technology in general practice in 10 countries. Healthcare Quarterly 2007; 10(2):107-115.
(3)
Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006; 144:742-752.
Cresswell K, Sheikh A. The NHS Care Record Service: recommendations from the literature on successful implementation and adoption. Inform Prim Care 2009; 17(3):153-160.
(21) Patton M. Qualitative research & evaluation methods. London: Sage, 2002.
(5)
Friedman C, Abbas U. Is medical informatics a mature science? A review of measurement practice in outcome studies of clinical systems. Int J Med Inform 2003; 69(23):261-272.
(23) Peters T, Waterman R. In search of excellence: lessons from America's best run companies. New York: Harper and Row, 1982.
(6)
Greenhalgh T, Potts H, Wong G, Bark P, Swinglehurst D. Tensions and paradoxes in electronic patient record research: a systematic literature review using the metanarrative method. Milbank Q 2009; 87(4):729-788.
(25) Miles M, Huberman A. Qualitative data analysis. London: Sage, 1984.
(4)
(7)
(22) Silverman D. Doing qualitative research, a practical handbook. London: Sage, 2000.
(24) Becker H. Sociological work. London: Allen Lane, 1971.
(26) Corbin J, Strauss A. Strategies for qualitative data analysis. CA: Sage, 2008.
Black A, Car J, Pagliari C, Anandan C, Cresswell K. The Impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011; 8(1).
(27) Mays N, Pope C. Quality in qualitative health research. London: BMJ Publication Group, 1999.
(8)
Shekelle P, Goldzweig C. Costs and benefits of health information technology: an updated systematic review. 2009. London, RAND Corporation.
(28) Berg M. Implementing information systems in health care organizations: myths and challenges. Int J Med Inform 2001; 64:143-156.
(9)
Lorenzi N, Novak L, Weiss J, Gadd C, Unertl K. Crossing the implementation chasm: a proposal for bold action. J Am Med Inform Assoc 2008; 15(3):290-296.
(29) Beynon-Davies P, Lloyd-Williams P. When health information systems fail. Top Health Inf Manage 1999; 19(4):66-79.
(10) Coiera E. Building a National Health IT system from the middle out. J Am Med Inform Assoc 2009; 16(3):271-273.
(30) Bingi P, Sharma M, Godla JK. Critical issues affecting an ERP implementation. ISM 2001; 16(3):7-14.
(11) Brennan S. The biggest computer programme in the world ever! How's it going? JITE 2007; 22:201-211.
(31) Brender J, Ammenwerth E, Nykänen P, Talmon J. Factors influencing success and failure of health informatics systems - a pilot Delphi study. Methods Inf Med 2006; 45(1):125-136.
(12) Greenhalgh T, Stramer K, Bratan T, Byrne E, Russell J, Hinder S et al. The devil's in the detail: final report of the independent evaluation of the Summary Care Record and Health Space programmes. 2010. University College London.
(32) Glaser J. More on management's role in IT project failure. Healthc Financ Manage 2005; 59:82-89. (33) Goldfinch S. Pessimism, computer failure, and information systems development in the public sector. Pub Adm Rev 2007; 67:917-929.
(13) Hendy J, Fulop N, Reeves B, Hutchings A, Collin S. Implementing the NHS information technology programme: qualitative study of progress in acute trusts. BMJ 2007; 334(7608):1360.
(34) Heeks R. Health information systems: Failure, success and improvisation. Int J Med Inf 2006; 75:125-137.
(14) Public Accounts Committee. The National Programme for IT in the NHS: progress since 2006. 2009.
(35) McManus J, Wood-Harper T. Understanding the sources of information systems project failure. Manag Serv 2007;3843.
(15) Cornford T, Doukidis G, Forster D. Experience with a structure, process and outcome framework for evaluating and information system. Omega 1994; 22(5):491-504.
(36) Pare G, Sicotte C, Jaana M, Girouard D. Prioritizing the risk factors influencing the success of clinical information system projects. A Delphi study in Canada. Meth Inf Med 2008; 47(3):251-259.
(16) Cresswell K, Worth A, Sheikh A. Actor-Network Theory and its role in understanding the implementation of information technology developments in healthcare. BMC Med Inf Dec Mak 2010; 10(1):67.
(37) Wears R, Cook R, Perry S. Automation, interaction, complexity, and failure: A case study. Reliability Engineering & System Safety 2006; 91.
(17) Cresswell K, Ali M, Avery A, Barber N, Cornford T, Crowe S et al. The long and winding road…an independent evaluation of the implementation and adoption of the NHS Care Records Service in secondary care in England. http://www.haps.bham.ac.uk/publichealth/cfhep/005.shtml. 2011.
(38) Wing L. Investigating success factors in enterprise application integration: a case-driven analysis. EJIS 2005; 14(2):175-187. (39) Department of Health. Equity and excellence: Liberating the NHS. 2010.
(18) George A, Bennett A. Case studies and theory development in the social sciences. London, England: MIT Press Cambridge, 2005.
(40) Cabinet Office. Government ICT Strategy. http://www.cabinetoffice.gov.uk/content/government-ictstrategy. 2011.
(19) Stake R. The art of case study research. London: Sage, 1995.
(41) Craig D. Plundering the Public Sector. London: Constable, 200
(20) Yin R. Case study research, design and methods. London: Sage, 2009.
689
