Journal of Medical Systems (2018) 42:28 https://doi.org/10.1007/s10916-017-0878-1
SYSTEMS-LEVEL QUALITY IMPROVEMENT
Key Performance Indicators to Measure Improvement After Implementation of Total Laboratory Automation Abbott Accelerator a3600 Marijana Miler 1
&
Nora Nikolac Gabaj 1 & Lora Dukic 2 & Ana-Maria Simundic 2
Received: 7 June 2017 / Accepted: 13 December 2017 # Springer Science+Business Media, LLC, part of Springer Nature 2017
Abstract The aim of the study was to estimate improvement of work efficiency in the laboratory after implementation of total laboratory automation (TLA) by Abbott Accelerator a3600 in the laboratory with measuring different key performance indicators (KPIs) before and after TLA implementation. The objective was also to recommend steps for defining KPIs in other laboratories. For evaluation of improvement 10 organizational and/or technical KPIs were defined for all phases of laboratory work and measured before (November 2013) and after (from 2015 to 2017) TLA implementation. Out of 10 defined KPIs, 9 were successfully measured and significantly improved. Waiting time for registration of samples in the LIS was significantly reduced from 16 (9– 28) to 9 (6–16) minutes after TLA (P < 0.001). After TLA all tests were performed at core biochemistry analyzers which significantly reduced walking distance for sample management (for more than 800 m per worker) and number of tube touches (for almost 50%). Analyzers downtime and engagement time for analyzers maintenance was reduced for 50 h and 28 h per month, respectively. TLA eliminated manual dilution of samples with extreme results with sigma values increment from 3.4 to >6 after TLA. Although median turnaround time TAT for potassium and troponin was higher (for approximately 20 min), number of outliers with TAT >60 min expressed as sigma values were satisfying (>3). Implementation of the TLA improved the most of the processes in our laboratory with 9 out of 10 properly defined and measured KPIs. With proper planning and defining of KPIs, every laboratory could measure changes in daily workflow. Keywords Key performance indicators (KPI) . Productivity . Total laboratory automation (TLA) . Quality
Introduction Total laboratory automation (TLA) consolidates preanalytical and postanalytical procedures in clinical laboratory and also includes analytical systems for complete sample processing. Although various TLA solutions are present in laboratories for almost 30 years, in recent 10 years TLA is recognized as an This article is part of the Topical Collection on Systems-Level Quality Improvement * Marijana Miler
[email protected] 1
Department of Clinical Chemistry, Sestre Milosrdnice University Hospital Center, Zagreb, Croatia
2
Department of Medical Laboratory Diagnostics, University Hospital BSveti Duh^, Zagreb, Croatia
inevitable component of every modern laboratory [1]. TLA provides standardization of the sample management, increases the workload efficiency with the optimization of laboratory processes, and the most importantly, reduces errors in all phases of laboratory work (preanalytical, analytical and postanalytical) [2]. Key performance indicators (KPIs) are objective measures for assessment of system efficiency and monitoring changes in routine laboratory work [3, 4]. KPIs should be specific, measurable, achievable, realistic and timely (SMART) [5]. KPIs are usually defined as indicators of technical or organizational changes (or combination of both). Purely technical KPIs measure, for example, performances of the new analytical system while organizational KPIs include e.g. changes of the workflow procedures. KPIs should be defined before any changes are made in everyday processes and measured before and after implementation of the changes. KPIs are needed in order to objectively assess efficiency of the change processes [4].
28
Page 2 of 12
Before the TLA implementation, in the core laboratory for the medical biochemistry and analytical toxicology most of the sample management processes were manually performed, e.g. assessment of the sample quality, aliquoting and sample distribution. Besides that, laboratory personnel had to walk a lot between rooms in the laboratory to distribute samples on different analyzers. Manual steps in the procedures were not standardized, were subjective, time consuming and prone to errors [6]. In order to standardize processes, optimize workflow, increase the productivity of personnel and to reduce the possibility for errors, Abbott Accelerator a3600 (Abbott, Abbott Park, IL, USA) TLA with complete solution for preanalytical, analytical and postanalytical sample handling was implemented. The main goal of this study was to evaluate accomplishment of the TLA implementation by measuring changes of 10 preanalytical, analytical and postanalytical KPIs, before and after reorganization of the laboratory work. Additional objective was to provide a flowchart of steps required for evaluating implementation of TLA.
J Med Syst (2018) 42:28
automation. All KPIs were measured in November 2013 and afterwards in different periods from 2015 to 2017.
Key performance indicators 1. Workspace area (WSA) Workspace area was defined as a space directly associated with preanalytical, analytical and postanalytical management of the samples in the core biochemistry laboratory. Width and length of workspace area were measured with the laser rangefinder (Bosch DLE 40, Bosch, UK) and area of each room was expressed in square meters. Area where samples were processed, with all analytical systems, centrifuges and space for sample quality assessment was designated as (pre)analytical area, while area for lab reports releasing was designated as postanalytical area (computer rooms). Considering constant increase of laboratory requests, our goal was to be able to process as many samples by occupying lower amount of space. Remaining (saved) space can be allocated for laboratory staff (reports releasing, teaching, communication with clinicians). 2. LIS registration waiting time
Materials and methods Study design and setting This study was conducted in the Department of Clinical Chemistry (DCC) of Sestre Milosrdnice University Hospital Center, accredited according ISO 15189 since 2007. In the laboratory, more than 350 tests are performed with daily workload of approximately 1500 samples. DCC had emergency and routine sample reception, with different sample management. Emergency samples were admitted, then centrifuged on the stand-alone centrifuges (Rotofix 32A, Hettich, Tuttlingen, Germany), registered to laboratory information system (LIS) and put on the core biochemistry analyzers (Beckman Coulter AU2700 or AU680). If necessary, after centrifugation samples were manually aliquoted for other analyzers: Cobas e411 (Roche Diagnostics, Indianapolis, USA), Axsym (Abbott Laboratories, Abbott Park, Illinois, USA) and Vitros 250 (Ortho Clinical Diagnostics, Buckinghamshire, UK). In December 2014, total laboratory automation (TLA) by Abbott Accelerator a3600 (Abbott, Abbott Park, IL, USA) was introduced in the laboratory routine work. TLA included automated track system with input-output module, two integrated centrifuges, decapper, two Architect c8000, one Architect i2000, Aliquoter and Sealer. In order to estimate and compare work efficiency before and after implementation of TLA, 10 key performance indicators (KPI) were defined prior the introduction of
Before TLA, processes of blood sample registration to the LIS were different for emergency and routine samples. Emergency samples were registered into LIS directly from electronic requests from hospital information system (HIS), immediately after arrival in the laboratory. On the other hand, routine sample tests were requested on paper and registered in batch, after registration of all received emergency samples. Together with TLA, HIS was introduced in the entire hospital and all tests requests were made electronically. Measurement of waiting time was done during two working days before (11th and 12th December 2013) and after (23rd and 24th August 2016) TLA implementation. For all received samples, time of arrival was recorded. Afterwards, time of registration to LIS was exported from information system. Waiting time from arrival to the registration of routine samples to the LIS was calculated and expressed in minutes. Results were presented as median and interquartile range. Our goal was to decrease LIS registration waiting time. 3. Tests panel distribution Tests panel distribution represented the number of tests measured outside of core biochemistry analyzers. Core biochemistry analyzers were defined as analyzers on which the most of the clinical chemistry tests were measured: AU2700 and AU680 (Beckman Coulter, Brea, USA) before TLA and analyzers within TLA after implementation (two Architect c8000 and one
J Med Syst (2018) 42:28
i2000; Abbott Laboratories, Illinois, USA). Cobas e411 (Roche Diagnostics, Indianapolis, USA), Axsym (Abbott Laboratories, Abbott Park, Illinois, USA) and Vitros 250 (Ortho Clinical Diagnostics, Buckinghamshire, UK) were available as analyzers outside core analyzers. Since samples with tests for other analyzers needed to be manually aliquoted and distributed, those tests were identified as having potential for errors. In November 2013 and February 2017 (before and after TLA, respectively) all biochemistry tests requests were counted from LIS. Afterwards, proportions of tests per analyzers outside the core biochemistry analyzers were calculated. As a measure of high quality performance of processes, sigma value >4 was set. Our goal was to decrease proportion of tests measured outside of core biochemistry analyzers. 4. Walking distance for sample management Minimum number of laboratory staff required for daily management of biochemistry samples was assessed. For daily
Page 3 of 12 28
management of samples, several stages were recognized: admission of samples, registration to LIS, distribution to preanalytical and analytical systems and to working stations distant from the core biochemistry analyzers. Number of steps for management of emergency and routine biochemistry samples was measured by pedomet e r ( O m r o n p e d om e t e r H J - 7 2 0 I T C , H o o f d d o r p , The Netherlands) fixed at the belt of personnel during the weekly working hours (7:30–15:30). Steps were turned into meters. Firstly, 7 chosen representative staff members crossed 20 steps. Afterwards, distance of those 20 steps were measured with laser rangefinder (Bosch DLE 40, Bosch, UK). All measured distances were summarized and divided with 7 to obtain average distance of 20 steps. Walking distance was measured in two periods: in December 2013 (before TLA) and June 2016 (after TLA implementation). Distance walked per test was calculated for 30 the most frequently requested tests in LIS according to the equation:
Meters walked per test ¼ ðsum of steps walked by all included workers during the day average distance o f 20 stepsÞ=average number of 30 most common test requested per day
For comparison of daily distance walked per worker, average number of tests requests after TLA was used: Daily distance walked per worker ðmÞ ¼ ðtotal number of walked steps in day=number of workersÞ average number of tests after TLA average distance of 20 steps
Our goal was to decrease number of steps required for sample management in order to reduce unnecessary waste of energy and increase productivity of laboratory staff.
distribution) of emergency samples was done manually. After TLA implementation all samples (routine and emergency) were managed automatically, on TLA. Proportions of samples with different combinations of tests (A, B or C) were exported from LIS for two days in November 2013 and February 2017. Proportions of A, B and C combinations of tests were multiplied with number of tube touches according to the following equation: TTM ¼ ðA number of tube touches A proportionÞ þðB number of tube touches B proportionÞ þðCnumber of tube touches C proportionÞ
5. Tube touch moment (TTM) Tube touch moment (TTM) was defined as an average number of touches necessary to process the sample (tube) from the sample registration to LIS to the sample archiving. TTM was counted for three most common combinations of routine laboratory tests: samples with tests for the core biochemistry analyzers only (A), samples with tests for the core biochemistry analyzers and aliquots for other analyzers (B) and samples that only needed an aliquots for other analyzers (C) (Table 1). Before TLA, preanalytical system aliquoted only routine samples, while management (aliquoting and
A B C TTM
samples with tests only for the core biochemistry analyzers samples with tests for the core biochemistry analyzers and aliquots for other analyzers samples that only needed aliquots for other analyzers (C) tube touch moment
Our goal was to reduce number of tube touches. 6. Manipulation tube time (MTT)
28
J Med Syst (2018) 42:28
Page 4 of 12
Table 1 Touches of tubes necessary to process the sample from registration to LIS to the sample archiving before the TLA implementation
Number
Tube touch moment (TTM)
Combination of laboratory tests
1. 2. 3. 4. 5.
Registration to LIS Barcode print Labeling primary tube Labeling aliquot tube Transfer to centrifuge
A, B, C A, B, C A, B, C B, C A, B, C
6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20.
Balancing centrifuge Putting on centrifuge Removing from centrifuge Decapping Aliquoting Putting on preanalytical system* Removing from preanalytical system* Assessment of interferences Transfer to other analyzers Putting on other analyzers Removing from other analyzers Putting on core analyzer Removing from core analyzer Putting on storage racks Archiving storage racks
A, B, C A, B, C A, B, C A, B, C B, C A, B, C A, B, C A, B, C B, C B, C B, C A, B A, B A, B, C A, B, C
Legend: *only for routine laboratory samples, A – samples with tests for the core biochemistry analyzers only, B – samples with tests for the core biochemistry analyzers and aliquots for other analyzers, C – samples that only needed an aliquots for other analyzers
Manipulation tube time (MTT) per sample was defined as duration (in seconds) of every stage counted as tube touch moment (TTM). Duration for all procedures was measured with stop-watch. To reduce variability of measurement (due to different conditions) duration of every stage was measured 20 times and at the end, average duration was calculated. Three different MTTs were calculated in emergency and routine biochemistry laboratory according to the following equations: 1. MTT per sample = tube touch moment (TTM) x duration of all procedures x proportion of samples with the most common combinations (A, B, C) 2. MTT per 500 samples (average daily number of samples) = MTT per sample × 500 3. MTT per day (in hours) = MTT per 500 samples / 3600
before TLA were: core biochemistry analyzers (AU2700/ AU680 (Beckman Coulter, Brea, USA), Cobas e411 (Roche Diagnostics, Indianapolis, USA), Vitros 250 (Ortho Clinical Diagnostics, Buckinghamshire, UK), preanalytical system Beckman OLA 2500 (Beckman Coulter, Brea, USA), Axsym (Abbott Laboratories, Abbott Park, Illinois, USA) and centrifuges. After TLA, measurement was done for 3 Architect analyzers: two c8000 and i2000, as well as for preanalytical system Accelerator 3600 (Abbott Laboratories, Abbott Park, Illinois, USA). DT was calculated for one month period by multiplying number of maintenance operations (for daily, weekly and monthly maintenance, reagents filling, calibration and quality controls) with time in minutes. Our goal was to reduce total downtime of analyzers in one month period. 8. Engagement time (ET) per month
Our goal was to reduce duration of all tube touch moments. 7. Analyzers downtime (DT) per month Analyzers downtime (DT) was defined as time while analyzers were out of function due to the maintenance procedures. DTwas measured with stop-watch from the moment of analyzers set in off-line (or shut down) until they were in the fully functioning mode (on-line). Analyzers included in the measurement
Engagement time (ET) was defined as time needed for active engagement of personnel for all maintenance procedures with analyzers (filling with reagents, calibration, quality control, daily, weekly and monthly maintenance). ET was measured for analyzers from 7th KPI (DT) using stop-watch in one month period in 2013 and 2017 and expressed in hours. Our goal was to reduce the total time of personnel spent for maintenance procedures.
J Med Syst (2018) 42:28
Page 5 of 12 28
9. Measurement ranges (MR) and dilution factors (DF)
Implementation of TLA
Measurement ranges (MR) and proportion of samples with values outside of measurement range were recorded during one year period for enzymes: amylase (AMY) in serum and urine, alanine aminotransferase (ALT), aspartate aminotransferase (AST), creatine kinase (CK) and lactate dehydrogenase (LD) in serum at core biochemistry analyzers used before (AU2700 and AU640) and after (Architect c8000) TLA implementation. All analyzers used in 2013 and 2015 had automatic dilution protocols, with different type of sample management. On AU2700/AU680, only one automatic dilution for samples with enzyme activities above upper measurement range was available. Samples had to be manually handled and put on the analyzer for rerun in dilution. Also, automated secondary dilution protocol was not available and all samples with values higher than primary dilution needed to be manually diluted and afterwards put on analyzers. With TLA, samples were automatically rerun (for primary and secondary dilution) from track-system and sent to analyzers. Automatic dilution of samples with extreme values increases turnaround-time (TAT), while manual dilution increases TAT and is more vulnerable to errors. Therefore, manual management of samples with extremely high enzyme activities was defined as possible error. The rate of errors due to manual management was evaluated and sigma values were calculated. For calculation of sigma value, total number of tests requested for each enzyme in one year period was defined as the opportunities, and number of results above MR for defined enzymes was defined as defects. Sigma values >3 were defined as acceptable. Our goal was to reduce number of samples that requited automatic or manual dilution.
When all 10 KPI’s were measured and recorded in the old setting, implementation of TLA began. Laboratory space predefined for new system was cleared out and reconstructed. Firstly, new analyzers were delivered in the laboratory. Since Department of Clinical Chemistry is accredited to ISO 15189 standard, all new tests or systems should be verified before introduction in the routine work. Therefore, verification of analytical performances (precision, linearity, accuracy, interferences) for all tests was done. Also, patient’s results were compared with results from previously used analyzers. If any bias in reported results were noticed, clinicians were notified about the possible bias regarding previous results. When all construction work was finished, analyzers were connected to the track system. About two months later, when system was stable and all personnel properly educated, all defined KPIs were measured again.
10. Turnaround time (TAT) Management of samples before and after TLA was different. Before TLA emergency samples were centrifuged immediately upon arrival and afterward registered to LIS. Routine samples were firstly registered to the LIS and afterwards put on preanalytical system for aliquoting and distribution to other working places. After TLA implementation, all samples are registered into the information system immediately after arrival and then put on the track system. Turnaround time (TAT) was calculated for emergency tests potassium and troponin in one-month period (November 2013 and March 2016). TAT was measured from registration to the LIS to releasing results and expressed as median, interquartile range and 90th percentile of all recorded TAT values (time needed for 90% of samples to be finished). Outliers (number of samples exceeded 60 and 120 min) were also counted from LIS. Our goal was to reduce TAT for potassium and troponin in emergency laboratory.
Statistical analysis Data were presented as count and percentages and median with interquartile range (IQR). Normality of data distribution was checked with Kolomogorov-Smirnov test. MannWhitney test was used for testing the differences for waiting time median and turnaround time. Comparison of proportions was used for percentages of measurement ranges and TAT outliers before and after implementation of TLA. Sigma values were calculated using web calculator [7] according to the following eqs. [8]: Defects per million opportunities ðDPMOÞ ¼ ðNumber of defined errors=Total number of opportunities ðor samplesÞÞ 1000000
Process sigma = 0.8406 + √(29.37)-2.221 × (log(DPMO)). Sigma values >3 were considered as acceptable values and P values 6.
This indicator was evaluated as successful. 4. Walking distance for sample management Laboratory personnel members (N = 7) walked approximately 14.3 m per 20 steps (average length of step 0.7 m). Sum of 30 the most common tests performed in laboratory per day increased for 16% after TLA implementation (4008 before vs. 4662 tests after, respectively). In spite of greater number of tests performed, number of laboratory workers needed for daily sample management was lower (9 vs. 8 before and after TLA, respectively). Moreover, all workers included in the sample management walked 2 m less per test (5.1 vs. 3.1 m before and after TLA, respectively), while number of steps walked by each worker decreased for 31% per day. When numbers of steps walked per person were calculated for average of 4662 tests performed daily after TLA, walking distance decreased for more than 800 m per person per one day (Fig. 3). This indicator was evaluated as successful. 5. Tube touch moment (TTM) 6. Manipulation tube time (MTT) In 2013, before TLA implementation, 20 stages for sample management from registration to LIS to storage of samples to refrigerator were possible. The most of the emergency and routine samples had only tests performed on core biochemistry analyzers (91% vs. 98% for emergency and 75% vs 74 for routine samples before and after TLA, respectively). Emergency samples before TLA had combination (B) of tests for aliquoting and core analyzers in 8%. On the other hand, proportion of routine samples with B combination remained the same (24%). The smallest proportion (1–2%) of samples
Fig. 1 Ground plan of the laboratory before and after implementation of total laboratory automation (TLA)
J Med Syst (2018) 42:28 Fig. 2 KPI Test panel distribution before and after TLA implementation. Total number of test requests increased for 26% after TLA with all tests done only core biochemistry analyzer. Abbreviations: KPI, key performance indicator; TLA, total laboratory automation
Page 7 of 12 28
KPI 3. Test panel distribuon 140% 120%
1301
100% 80% 60% 40% 20%
0
0% Total number of tests per month Number of tests per other analyzers Before TLA
Aer TLA
9. Measurement ranges (MR) and dilution factors (DF)
had C combination (only aliquoting tests) before TLA. After TLA none of emergency sample needed aliquoting for other analyzers (none with B and C combination). Total tube touch moment decreased for almost 50% after TLA for emergency and routine samples. Duration of every step in TTM fell for approximately 35–40% for routine and emergency samples. Duration expressed in hours per approximately 500 samples was lower for more than 4four h per day after TLA introduction (Fig. 4). Those indicators were evaluated as successful.
Values for upper measurement ranges with number of samples with values above MR during one year are summarized in the Table 2. Sigma values for samples with possible errors due to manual management (manual loading of samples to the analyzers) was calculated for period before and after TLA implementation. TLA eliminated manual manipulation of samples and real sigma values go to infinity. This indicator was evaluated as successful.
7. Analyzers downtime (DT) per month 8. Engagement time (ET) per month
10. Turnaround time (TAT) Turnaround time for potassium and troponin was longer after implementation of TLA for approximately 20 min. Number of samples that exceeded TAT of 1 h was larger 7% for potassium, and almost 4 times larger for troponin (Table 3). This indicator was evaluated as unsuccessful.
DT before TLA was 8875 min (148 h) and after implementation of TLA 5880 min (98 h). Before TLA analyzers were out of function even during setting the calibrators and quality controls on the analyzers. ET decreased after TLA for approximately 28 h per month (118.8 vs. 90.8 h, respectively). These indicators were evaluated as successful.
Fig. 3 KPI Walking distance for sample management
KPI 4. Walking distance for sample management 140% 4662
120% 9 100%
4008
2639
8
80%
1824
60% 40% 20% 0% Number of laboratory Number of tests staff needed Before TLA Aer TLA
Daily walking distance per worker (m)
28
J Med Syst (2018) 42:28
Page 8 of 12
Fig. 4 Tube touch moment (TTM) and manipulation tube time (MTT) calculated before and after implementation of TLA in emergency and routine laboratory
KPI 5. Tube touch moment (TTM) and KPI 6. Manipulaon tube me (MTT) 17
18 16 14 12 10 8 6 4 2 0
13 8
7
7.5
6.0
4.9
3.8
TTM per sample MTT per day (hours) TTM per sample MTT per day (hours) Emergency laboratory
Roune laboratory
Before TLA
Due to the different management of the samples before and after TLA, new KPI for TAT was defined as sigma values in which were included number of emergency samples that exceeded TAT of 1 h per all received samples. As acceptable criteria sigma value >3 was defined. Sigma values were lower in the two months period while TLA was introduced. After that period, sigma values raised to values reached before implementation of the new system (Fig. 5). Overall, 9 of 10 defined KPIs were successfully measured and improved after TLA. Only one indicator (median TAT for potassium and troponin) was not properly defined and could not be evaluated after implementation of TLA, therefore new KPIs was established for monitoring TAT.
Discussion This study showed that total laboratory automation can optimize and diminish many processes in the biochemistry Table 2 Enzyme (U/L)
Aer TLA
laboratory, reduce the waste of personnel time and energy for manual handling of samples and improve entire workflow. All increment in sample processes efficiency is possible in even smaller workspace area. Possibilities for errors are significantly reduced and personnel are less exposed to possibly contagious biological material. If any of the following changes are introduced in the laboratory: new analytical system, implementation of informational system, reorganization of the workflow, TLA or accreditation process, it is crucial to asses if those changes improved efficiency of the processes. KPIs should be objective and measureable indicators of efficiency in all stages of total testing process (preanalytical, analytical and postanalytical) [9, 10]. There are several published studies on using KPIs as a measure of success in clinical setting, laboratories and clinics. In emergency departments in Ireland KPIs were defined to monitor clinical outcomes and measure time needed to therapy and to measure economical cost of collecting data [11]. To
Measurement ranges and calculated sigma values for proportion of samples above MR Measurement ranges
Percentages of samples above MR with automatic dilution
Percentages of samples above MR with manual dilution
Sigma values for samples with automatic dilution
Sigma values for samples with manual dilution
Before TLA
Before TLA
Before TLA
Before TLA
Before TLA
After TLA
After TLA
P*
After TLA
P*
After TLA
After TLA
Amy (S)
1500
3010
0.33%
0.06%
