Failure of Electronic Medical Records in. Canada: A Failure of Policy or a Failure of. Technology? Karim KESHAVJEEa, Anil MANJIb, Baljit SINGHb, Nick ...
Failure of Electronic Medical Records in Canada: A Failure of Policy or a Failure of Technology? Karim KESHAVJEEa, Anil MANJIb, Baljit SINGHb, Nick PAIRAUDEAUc a InfoClin Inc, Toronto, Ontario, Canada b York University, Toronto, Ontario, Canada c DMLink Inc, Toronto, Ontario, Canada
Abstract. This paper utilizes 3 frameworks described in the literature to better understand the reasons why Canada lags other OECD countries in EMR implementation. First, we use an EMR policy framework to evaluate 3 provincial EMR implementation programs across Canada. Second, we use an EMR implementation multi-theoretical framework that is predictive of EMR success to evaluate those same programs. Finally, we use a cost-benefit framework developed in the US to compare the cost-benefits of EMR implementations in Canada compared to the US. We draw conclusions and make recommendations based on our findings. Keywords: EMR implementation, Policy, Implementation Frameworks,
Introduction Canada’s EMR implementation programs have not had the success enjoyed by other OECD countries. A recent international comparison showed that less than 30% of the Canadian physician population use any of e-prescribing, electronic lab review or several other ehealth technologies in comparison to rates of greater than 90% in the Netherlands, the UK and Denmark [1]. A more recent survey of physicians in Canada indicates that only 9.8% of physicians use an electronic form of record keeping in their practice [2]. This is not due to a lack of programs and policies to encourage EMR uptake. Some provinces, including Ontario, Alberta, Manitoba and Nova Scotia have had such programs for several years. British Columbia and Newfoundland have active programs in the start-up phases. However, in spite of active programs and tens of millions of dollars in expenditures, EMR uptake has grown exceedingly slowly in Canada. Ontario has had a series of EMR implementation programs over the last decade and has budgeted over $150 million for EMRs, yet adoption has increased by less than 10% of physicians across the province. We explore the underlying reasons for poor EMR adoption in Canada using multiple theoretical lenses described in the EMR literature.
1. Methodology 3 frameworks were utilized to help elucidate reasons for poor EMR uptake in Canada. Each framework is described in greater detail below. For each component in a particular framework, the provincial program was given a score of Yes (program has implemented substantial aspects of that component of the framework), Partial (some aspects of the component have been implemented, but that important aspects of that component are not implemented) or No (this component has not been implemented). Although several provinces in Canada have EMR implementation programs, 3 provinces were selected that have relatively mature and/or well-documented programs. The 3 provincial programs were compared against the 3 frameworks to determine the type and extent of short-falls in these provincial EMR roll out programs.
1.1. Frameworks for Analysis 1.1.1. EMR Implementation Policy Framework The first policy framework used was developed by one of the authors (KK) for a white paper on EMR implementation in Ontario [3]. This policy framework concentrates on larger, macro level drivers of EMR implementation, including such factors as subsidy models for encouraging uptake, engagement of key stakeholders, creating an enabling environment for EMR uptake, creating and leveraging network effects and engaging patients and patient advocacy groups. For each element, if an EMR program incorporates that feature (Yes), we gave it a score of 1. If the program does not have that feature (No), we gave it a score of 0 and if there was a partial match (Part), we gave it a score of 0.5. 1.1.2. EMR Cost-Benefit Evaluation Framework The second framework was developed by Wang et al [6] to better understand the macro level economic drivers of EMR uptake. On the cost side, their framework includes the usual elements: hardware, software, support, implementation and a one-time productivity loss. We include scanning and government subsidies into the framework, as these elements are common in Canadian EMR implementations. On the benefit side, they include decreased chart pulls, billing errors and transcription; prevention of adverse events; decreased utilization of drugs, lab tests and radiology tests; and increased revenues from improved charge capture. Their findings indicate that for a large physician-based health care organization in the US there is a net benefit to implementing EMRs based on savings in drug utilization, laboratory utilization, radiology utilization and increased revenues from improved charge capture. This framework is applied to the Canadian context, utilizing Canadian experience, costs and benefits to see how well the Canadian environment allows for these economic benefits to be realized. 1.1.3. Multi-theoretical, Best Practices in EMR Implementation Meta-framework The third framework is a multi-theoretical best practices EMR implementation framework that is predictive of success in EMR implementations [4, 5]. This meta-framework, a composite of multiple frameworks on EMR implementation described in the literature, was developed using a systematic review of the literature on EMR implementation and includes 17 factors that are discussed in the literature as being important for EMR success. This framework is relevant to individual EMR implementations and their success at a micro, practice level. We examine each of the three provincial programs to see how well they help physicians make their implementations a success.
2. Results 2.1. Policy Framework Analysis All provincial programs fare poorly in the EMR Policy Framework analysis. Table 1 shows how each province is performing on some key policy areas. Ontario has not yet engaged specialists in their EMR implementation programs, leaving out an important 50% of physicians. Both Ontario and BC do not fund practice management services, electing to provide these themselves. However, government programs are rarely responsive to client needs, which continue to evolve. In most cases, policy lags client needs significantly; for example, in Ontario, physicians are asked to take time away from clinical work to select software, yet could purchase EMR selection services easily and save a significant amount of effort and angst.
Table 1. EMR Policy Framework Analysis EMR Policy Framework Element
Ontario
Alberta
BC
Engage all physicians (specialists and GPs)
Part
Yes
Yes
Fund Practice Management Services
No
Yes
No
Fund Information Management Services
No
Yes
No
Provide CDPM Incentives
Yes
No
Part
Self-Help and Peer Sharing
Part
Yes
Yes
Engage Key Medical Players
No
No
No
Provide Key ICT Infrastructure (secure e-mail)
No
Part
Part
Set and Implement Interoperability Standards
No
Part
Part
Engage Patients and Patient Advocacy Groups
No
No
No
2
5
3.5
Score
LEGEND Maximum Score = 9
No = 0
Yes = 1
Part = 0.5
None of the provinces provides any information management services or funding for them. The Ontario Ministry of Health has embarked on a program of providing prevention and chronic disease management incentives, which is likely to be one of the drivers of EMR uptake. Peer support and self-help programs are important to allow physicians to learn from each other. Yet, most provinces do not have an active program. Canada Health Infoway has implemented a peer support program, but it is ad hoc and lacks a proper structure and outcomes measurement process. None of the provinces has engaged key medical associations that drive clinical change, such as the College of Physicians and Surgeons, which licenses physicians nor the College of Family Physicians and the Royal College of Physicians and Surgeons, which certify physicians and provide accreditation of continuing medical education programs. Although most provinces work with their local Medical Association, these bodies tend to be political organizations that generally don’t have the legitimacy to drive clinical change that the other associations have. Other areas of poor performance include a lack of success in the provision of ICT infrastructure, such as secure high speed Internet connections, secure e-mails, electronic transmission of laboratory data, radiology reports, hospital reports and specialist reports; poor implementation of interoperability standards and little meaningful engagement of patients and patient advocacy groups [6]. Overall, from a maximum possible score of 9, all 3 provinces score less than 4 on the Policy Framework analysis. 2.2. Cost-Benefit Analysis Applying Wang et al’s [7] cost-benefit framework to the Canadian context, we get the results which are presented in . There are slight differences in the cost structure between the two countries, which are explained in the footnotes. We include government subsidies in the Canadian cost structure, as these are not a feature of the US framework. The dollar figure comes from Ontario’s EMR implementation program. Essentially, we find that the cost of EMR implementation, in spite of government subsidy, is much higher in Canada than in the typical large US clinic to which this framework applies. The major cost difference comes from the need to scan documents into the EMR in Canada. This is a feature of most clinics which have gone paperless in Canada, but imposes significant overhead costs which can make EMR implementations cost-prohibitive. Most US clinics absorb the cost of integrations into their operational budgets, as the cost of integrations are one-time costs and tend to be offset by decreased cost of filing paper reports and results.
Table 2. Cost-Benefit Analysis US
Canada Costs
Initial Cost
Annual
Over 5 years
Cost
Annual
Over 5 years
Hardware
$6,600
(q 3 yrs)
$13,200
$10,000
(q 3 yrs)
$20,000
Implementation2
$3,400
$3,400
$5,000 $4,000
1
Software
3
Support4 Scanning5
$1600
$1600
$9600
$1500
$1500
$9000
-
Productivity Loss
6
$11,200
$11,200
Gov’t Subsidy TOTAL Cost/MD
$5,000 $4,000 $2,400
$12,000
$12,000
$60,000
$5,000
$5,000
-$28,000
-$28,000
$46,400
$78,000
Benefits (Savings) MD Benefits Chart Pulls7 8
$5
600
$15,000
$4,800
24%
$5,760
Transcription
$9,600
28%
$13,440
0
0
$0
Charge Capture9
$383,100
2%
$15,324
$188,000
0%
$0
Billing Errors
$9,700
78%
$15,132
$9,400
$0
$0
Total MD Benefit
$58,896
$5,760
System Benefits Drug Reactions10
$6,500
34%
$8,840
$6,500
34%
$8,840
Drug Utilization
$109,000
15%
$65,400
$109,000
15%
$65,400
Lab Utilization
$27,600
8.80%
$4,858
$27,600
8.80%
$4,858
Radiol. Utilization
$59,100
14%
$16,548
$59,100
14%
$16,548
TOTAL System Benefit
$95,646
$95,646
$154, 542
$101,406
$108,142
$23,406
$12,496
-$72,240
Total Benefits (MD+System)
Net Benefit from EMR (Savings-Cost) Net MD Benefit (Savings-Cost)
Canadian physicians do not accrue as many benefits as their US counterparts –chart pulls are cheaper in smaller clinics and most Canadian general practitioners do not dictate their notes [8]. In addition, opportunities for increased revenues and decreased billing errors are also typically 1 2 3
Hardware cost differences exist because the US has larger clinics which can achieve economies of scale. Implementation costs in Canada are higher because clinics are smaller and geographically dispersed. Software license costs in Canada tend to be one-time, but are recurring costs in the US.
4
Support costs include software maintenance costs in Canada. In the US, large clinics integrate their EMRs with lab, radiology and hospital systems. In Canada, physicians need to scan documents into their EMRs. Based on ½ FTE per physician per year [7]. 6 In the US, HMOs typically decrease physician workloads by 20% for about 4 weeks during EMR implementation. 7 Chart pulls in Canada are less costly than in the US, where large clinics incur huge filing costs. 5
8
Most primary care physicians in Canada do not dictate their notes, however specialists will dictate letters. There is no evidence that EMR implementations increase billings or decrease billing errors in Canada. In any case, the situations in the two countries are sufficiently different that this does not apply in Canada. 10 We assume that reductions in adverse drug reactions and drug, lab and radiology costs would be the same in both countries. 9
smaller and are not realized with EMR implementations. In the US, physician reimbursement is dependent on clear documentation which is facilitated with EMR. We assume that cost avoidance for drug, lab and radiology will be the same in Canada as in the US, although this is not a foregone conclusion. Overall, we can see that there is indeed a net benefit to EMR implementation in Canada, as there is in the US. However, while US physicians do see a small direct economic benefit to themselves from EMR implementation, Canadian physicians are likely to see a significant net cost to themselves, in spite of government subsidies. 2.3. EMR Implementation Meta-Framework Analysis The final analysis revolves around an EMR implementation meta-framework that has recently been developed [4, 5]. Table 3 provides a listing of the factors and the odds ratio that this particular factor has in determining success of an implementation; i.e., how often a factor was seen in successful implementations compared to less successful implementations. This table also lists whether the 3 provincial programs have implemented, partially implemented or not implemented the various factors. Governance is about setting the mission and vision for a program, allocating resources to meet those goals and engaging the right stakeholders in providing oversight. None of the provinces has fully implemented appropriate governance for their EMR programs yet. Ontario has a board which lacks transparency; there is no public listing of board members for their EMR program nor does the board have true directive oversight of the program. Ontario also has not made a longterm commitment to implementing EMRs for all physicians. Alberta and BC have made longterm commitments to financing EMRs, but still do not have good policies for helping physicians who fail in EMR implementation. Given that the EMR implementation failure rate is somewhere between 20 and 35% [10, 11] and the costs of implementation are so high, a policy that does not explicitly take this into account is likely lead to poor uptake. In the literature, projects which were successful were 6 times more likely to have good governance than those that failed. Project Leadership speaks to project management and project championship. EMR implementations are complex, yet many implementations in all 3 provinces lack proper project management expertise. Some vendors have good project managers, but by and large, project management is left to chance and the vagaries of vendor capabilities in most provinces. BC and Alberta do train local project champions who will get stakeholders involved, sell the benefits of the program and spearhead the software selection process. Projects which are successful were 26 times more likely to have good project leadership than those that didn’t. The process of choosing software is quite important and was seen 17 times more frequently in successful projects. The software selection process is a central activity which bundles additional factors, including involving local stakeholders and selling the benefits. All of the provinces do a poor job of helping physicians select software and its associated activities. Most physicians get mired in the mechanics of the selection process and have little or no time to devote to getting their partners on board, to sell the benefits of the EMR and get the buy-in necessary for a successful start to the implementation. Technology Usability, Pre-load of Data and Integration with outside systems, such as lab, eprescribing, radiology and hospital systems, are important factors in EMR implementation success. In fact, these factors are the largest determinant of success seen in the literature. Yet, none of the provinces pays much attention to these factors. BC has some integration, but Alberta and Ontario are still struggling with these. None of the provinces assists physicians with pre-load of data, even though this is identified as a major barrier to EMR implementation by physicians [3]. None of the provinces addresses the issue of EMR usability, even though successful projects were 96 times more likely to have usable technology than did failed projects. Provinces are letting the market decide on usability. This may be a reasonable approach; however, none of the provinces has an explicit strategy for dealing with the inevitable failures when you leave such an important component for the marketplace to decide. The next cluster of important activities includes Early Planning (10-fold preponderance in successful projects) and its associated activities of Workflow Redesign (36-fold), Implementation Assistance (5-fold), Training (9-fold), Privacy & Confidentiality (not significant) and Feedback & Dialogue (9-fold). All 3 provinces do a poor job of providing this cluster of services. All provide training, but this is outsourced to vendors –most of whom know little about clinical practice and the information management needs of physicians. Only Alberta provides on-site implementation
assistance and privacy and confidentiality services. The biggest determinant of success, Workflow Redesign, is ignored by all provinces. The last important factor in the Meta-framework is Incentives. The literature is quite clear that Incentives are mostly information incentives, not just monetary ones. Information incentives come from the ability to know more about the patient when using the EMR than when using paper. This includes knowing the patient’s medication list when prescribing, knowing the latest lab result when ordering lab tests and being able to find information easily and quickly while seeing patients. Few physicians find drug-drug interactions checkers to be information incentives, as there are many false positives and the noise level generally is much higher than the useful information. Information incentives properly executed are 70-times more likely to be seen in successful implementations. Table 3. EMR Implementation Meta-Framework Analysis No.
Factor
SuccessFailure Odds Ratio
Ontario
Alberta
BC
5.83
No
Part
Part
26
No
Yes
Part
4.47 NS
No
Part
Part
1.
Governance
2.
Project Leadership
3.
Involve Stakeholders
4.
Choose Software
17.25
Part
Yes
Part
5.
Sell Benefits
13.5
No
Part
Part
6.
Pre-load/Integration
999
No
Part
Part
7.
Tech Usability
96
Part
Part
Part
8.
Early Planning
10
Part
Part
Part
9.
Workflow Redesign
36
No
Part
Part
10.
Implementation Assistance
5.09
Part
Yes
Part
11.
Training
8.73
Yes
Yes
Yes
12.
Privacy & Confidentiality
4.7 NS
Part
Yes
Part
13.
Feedback & Dialogue
8.73
Part
Part
Part
14.
Support
- NS
Yes
Yes
Yes
15.
User Groups
4.7 NS
Part
Part
Yes
16.
Incentives
69.75
No
No
No
17.
Business Continuity
2.67 NS
Part
Yes
Part
6 (3.5)
11.5 (7.5)
9.5 (6)
TOTAL SCORE
LEGEND Maximum Score = 17 NS= Not Significant
No = 0
Yes = 1
Part = 0.5
(x) = score for items which reached statistical significance. Max score =12
3. Discussion Canada’s continued difficulty in implementing EMRs on a national level is difficult to understand given the significant investments made over the last decade. This analysis points to why Canada lags other OECD countries using EMRs. Clearly, Canada’s policies lack the desired effects, when it comes to the speed of rolling out EMRs and successful usage outcomes. The provinces studied in this paper have either not implemented or only partially implemented key policies that are required for EMR uptake. Provinces need to provide funding for change management, project management, practice management and information management services so that physicians implementing EMR’s can benefit from their use. Key medical stakeholders who license physicians and provide continuing medical education also need to be engaged. Patients are another group that need to be better engaged. Interoperability standards need to be aggressively set and implemented. Provinces also need to carefully consider the cost-benefit ratios of EMR use in Canada and ensure that funding and policies lead to positive benefits for all stakeholders, especially physicians. If not, physicians are unlikely to invest time, energy and money into ventures that have a high risk of failure and lead to certain losses. Provinces need to learn the lessons from other jurisdictions which have been successful in EMR implementation. Good governance needs to be instituted that provides transparency and oversight over implementation of EMR programs. Funding for EMR implementations need to include project management and appropriate training for project champions. Finally, provinces need to think about information incentives and integrate them with policies more carefully to leverage the huge benefits available from implementing EMR’s successfully.
References [1]
Schoen C, Osborn R, Huynh PT, Doty M, Peugh J, Zapert K. On the front lines of care: primary care doctors' office systems, experiences, and views in seven countries. Health Aff (Millwood) 2006; 25(6): 555-571. [2] National Physician Survey (2007). College of Family Physicians of Canada. http://www.nationalphysiciansurvey.ca/nps/2007_Survey/Results/ENG/National/pdf/Final%20Posted%20Natio nal%20Level%20Table%20Set/NPS.2007.National.Results.Binder.Final.pdf. Accessed Nov 14, 2008. [3] Keshavjee K. EMR Implementation in Ontario: A position paper to increase EMR implementation in Ontario. Intel of Canada White Paper. July 2007. The paper is available at: http://www.infoclin.ca/assets/intel%20emr%20white%20paper.pdf Accessed Nov 14, 2008. [4] Keshavjee, K., Bosomworth, J., Copen, J. & et al. (2006). Best Practices in EMR Implementation: A Systematic Review. Proceedings of iSHIMR June 2006, 233-246. [5] Kucukyazici B, Keshavjee K, Bosomworth J, Copen J, Lai J. Best practices for Implementing Electronic Health Records and Information Systems. In Human, Social and Organizational Aspects of Health Information Systems [Eds: Kushniruk A and Borycki E]. IGI Global 2008. [6] http://govdocs.ourontario.ca/results?source=Ontario.+Smart+Systems+for+Health+Agency. Accessed Nov 14, 2008. [7] Wang SJ, Middleton B, Prosser LA, Bardon CG, Spurr CD, Carchidi PJ, Kittler AF, Goldszer RC, Fairchild DG, Sussman AJ, Kuperman GJ, Bates DW. A cost-benefit analysis of electronic medical records in primary care. Am J Med. 2003 Apr 1;114(5):397-403. [8] Personal communications with Hamilton area physicians who use EMR. [9] Keshavjee K, Troyan S, Langton KB, Holbrook AM, Naghi A, Topps D, Nazaerali N and VanderMolen D. Successful Computerization in Small Primary Care Practices: A Report on Three Years of Implementation Experience. Proc eHealth 2001 64-72. [10] Conn J. Failure, de-installation of EHRs abound: study. Modern Healthcare: Oct 30, 2007. Accessed Nov 14, 2008, http://modernhealthcare.com/apps/pbcs.dll/article?AID=/20071030/FREE/310300002. [11] http://www.infoclin.ca/assets/7yearsafteremrimplementation_poster_final.pdf Accessed Nov 14, 2008.
