Clinical evidence in brachytherapy

Radiotherapy and Oncology 125 (2017) 94–100

Contents lists available at ScienceDirect

Radiotherapy and Oncology journal homepage: www.thegreenjournal.com

Patient safety

Patient safety is improved with an incident learning system—Clinical evidence in brachytherapy Christopher L. Deufel ⇑, Luke B. McLemore, Luis E. Fong de los Santos, Kelly L. Classic, Sean S. Park, Keith M. Furutani Departments of Radiation Oncology and Radiation Safety, Mayo Clinic, Rochester, United States

a r t i c l e

i n f o

Article history: Received 6 March 2017 Received in revised form 26 July 2017 Accepted 28 July 2017 Available online 17 August 2017 Keywords: Safety Incident learning system Brachytherapy Radiation Medical error

a b s t r a c t Background and purpose: Health leaders have advocated for incident learning systems (ILSs) to prevent errors, but there is limited evidence demonstrating that ILSs improve cancer patient safety. Herein, we report a long-term retrospective review of ILS reports for the brachytherapy practice at a large academic institution. Material and methods: Over a nine-year period, the brachytherapy practice was encouraged to report all standard operating procedure deviations, including low risk deviations. A multidisciplinary committee assigned root causes and risk scores to all incidents. Evidence based practice changes were made using ILS data, and relevant incidents were communicated to all staff in order to reduce recurrence rates. Results: 5258 brachytherapy procedures were performed and 2238 incidents were reported from 2007 to 2015. A ramp-up period was observed in ILS participation between 2007 (0.12 submissions/procedures) and 2011 (1.55 submissions/procedures). Participation remained stable between 2011 and 2015, and we achieved a 60% (p < 0.001) decrease in the risk of dose error or violation of radiation safety policy and a 70% (p < 0.001) decrease in frequency of high composite-risk scores. Significant decreases were also observed in incidents with root causes of poor communication (60% decrease, p < 0.001) and poor quality of written procedures (59% decrease, p < 0.001). Conclusions: Implementation of an ILS in brachytherapy significantly reduced risk during cancer patient care. Safety improvements have been sustained over several years. Ó 2017 The Authors. Published by Elsevier Ireland Ltd. Radiotherapy and Oncology 125 (2017) 94–100 This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-ncnd/4.0/).

Seventeen years have elapsed since the Institute of Medicine (IOM) concluded that ‘‘tens of thousands of Americans die each year from errors. . ., and hundreds of thousands suffer or barely escape from nonfatal injuries” [1]. The IOM concluded that patient safety should be a priority in the healthcare system and greater attention paid to systems that reduce risk and prevent errors. That report generated strong reactions—supporting and opposing—and many centers instituted mandatory reporting systems for serious events [2,3]. Progress in improving patient safety has been made since the IOM report, however medical errors are still thought to be the third leading cause of death in the United States and procedural deviations in radiation oncology have been found to adversely impact tumor control and patient overall survival [4–6]. Some reluctance remains on the part of hospital leaders to report moderate and minor incidents, and widespread adoption ⇑ Corresponding author at: Mayo Clinic, 200 1st St. SW, Rochester, MN 55905, United States. E-mail address: [email protected] (C.L. Deufel).

of ILSs would benefit from studies that demonstrate successful interventions [7–13]. Unfortunately, few studies have reported on the effectiveness of an ILS in radiation oncology, and none isolate the brachytherapy practice [14,15]. This is despite the fact that numerous radiation centers have adopted ILSs for reporting procedural deviations and sharing lessons with all staff [14,16–20]. In this review of the brachytherapy practice at a large academic institution, we found that implementation of an ILS that captured all deviations from standard operating procedure, including low risk incidents, was associated with a reduction in risk to patients and improvements in communication among staff members and quality of written procedures. This supports the broader campaign for use of ILS in radiation oncology. Methods and materials Setting This retrospective review, at a large academic medical center, evaluates the impact that implementing an incident learning

http://dx.doi.org/10.1016/j.radonc.2017.07.032 0167-8140/Ó 2017 The Authors. Published by Elsevier Ireland Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

C.L. Deufel et al. / Radiotherapy and Oncology 125 (2017) 94–100

system had on patient safety and safety culture. Patient safety was measured using scores that were generated for each incident by a multidisciplinary committee. All brachytherapy staff participated in incident reporting, including physicians, physicists, radiation therapy technologists, nurse practitioners, physician assistants, nurses, medical residents, and medical physics fellows. Between January 1, 2007 and December 31, 2015, 5258 brachytherapy procedures were performed. The low-dose-rate (LDR) brachytherapy practice included treatments for ocular melanoma with eye plaques, biliary cancer with the intraluminal-endoscopic-retrogradecholangiopancreatography (ERCP) technique, prostate cancer using permanent radioactive seeds, and the spinal dura with betaemitting planar sources. The high-dose-rate (HDR) brachytherapy practice, with two remote after loaders and two treatment suites equipped with in-room computed tomography (CT), performed treatments for cancers of the cervix with the tandem and ovoid, tandem and ring, multi-tandem, and interstitial techniques; prostate with template and needles; esophagus with a Bougie applicator; bile duct with percutaneous and ERCP techniques; breast with a strut-based applicator; vagina and vaginal cuff with single and multi-channel cylinders; rectum with a shielded cylinder; soft tissue sarcomas with interstitial needles; and surgical tumor beds for various types of cancer with an intraoperative HDR technique. Intervention An ILS was created in 2007 for the brachytherapy practice in the department of radiation oncology. The ILS extended reporting beyond the institutional pathway for medical events and nearmisses, and encouraged the brachytherapy practice to report standard operating procedure deviations, including low risk deviations that did not reach the patient, but could indirectly impact patient care (e.g. missing initials on check-forms or documents). We anticipated the intervention would generate a large number of reports, the vast majority with very low risk scores, with the intention of obtaining a more comprehensive assessment of practice health and identifying practice areas that might be made more flexible, efficient, and safe. The intervention was based on the ideology that many problems are systems-based and not primarily due to individual recklessness or poor performers [21]. The learning system aimed to identify and fix the ‘as is’ processes and conditions that enable minor and major deviations. The system included: 1) reporting mechanisms for identifying and communicating potential failure modes, 2) analysis mechanisms for quantifying incident recurrence and risk, and 3) modification mechanisms for changing the ‘as is’ process and measuring whether modifications reduced risk. The intervention applies a retrospective methodology for practice improvement, as compared with a prospective approach for evaluating safety such as failure mode and effects analysis (FMEA). The reporting mechanism was a web-based form accessible from any clinic or hospital computer by any staff in the department. Evidence-based safety practices were considered during the design of the reporting workflow [22]. The reporting form contained the submitter’s name (an optional entry), submission date, procedure type, process step in which the incident occurred, process step where the incident was discovered, and a brief description of the incident. A checkbox was used to indicate whether the physician and physics staff had already been notified of the incident. The reporting form included a header clarifying that incident discussions are for internal analysis to improve the quality of our practice, and that the ILS does not replace the reporting of sentinel events, medical events, or reportable state violations. An example of the reporting form may be found in the online supplement. The analysis mechanism was performed by a multidisciplinary committee that is comprised of physicians, physicists, and

95

radiation therapy technicians who reviewed and scored each incident within 1–2 weeks. The scoring committee indicated the root cause and assigned a score from 1 (low) to 5 (high) to each of the following five categories: likelihood of recurrence, likelihood of quality assurance failure, likelihood of non-dose related severity, likelihood of dose related severity or radiation safety policy violation (including overdoses and under doses), and staff or patient time wasted. Although radiation safety policy violations are not the same as potential radiation dose deviations, they were grouped with dose risk because the intention of radiation safety policies is to prevent unnecessary exposures. A detailed description of the scoring scales for each category and definitions of risk are provided with the scoring form in the online supplement. All risk scores evaluate the potential risk of the incident as measured by what would have happened to the patient if the incident had not been detected. A composite-risk score was calculated from the multiplicative product of the aforementioned scores (minimum score = 15, maximum score = 55). Reports were automatically highlighted with a red color in the online viewer if the composite-risk score exceeded 50. This threshold captured approximately the 5% highest scoring incidents. The scoring committee also could choose to manually highlight entries with scores lower than 50 if they were perceived as relevant. The modification mechanism was conducted for all highlighted reports, as well as reports with recurring causes. The scoring committee shared results with all brachytherapy staff at monthly meetings. Staff discussed incident causes, proposed solutions to the recurring or high scoring events, and enacted practice changes to prevent recurrence (e.g. changes to procedure, documentation, equipment, etc.)

Data and statistical analysis A retrospective analysis was performed of incident reports that were collected and scored at the time of the incident between 2007 and 2015. Reports were binned in one year increments in order to assess general trends in safety (e.g. 2007, 2008, etc.). Decreases in the frequency of dose risk scores, composite-risk scores, and root causes are desirable outcomes. Baseline dose risk, composite-risk score, and root cause levels were collected from 2011, which was the first year that staff participation in the ILS, as measured by reporting frequency, achieved stable levels. A comparison of dose risk score, composite-risk score, and root cause frequency was performed for 2015 versus 2011 using chisquared methods. Chi-squared tests were performed with a = 0.05. The relationships between reporting frequency and the number of procedures performed each year were measured with Spearman correlations. We define a safety metric which we call the Risk Frequency Histogram (RFH). The RFH is a complementary cumulative distribution function (CCDF) and was used to evaluate the distribution of dose risk and composite-risk score for the years 2011–2015.

RFHðSÞ ¼

Number of Incidents with Score P S Total Number of Incidents

ð1Þ

Readers who are unfamiliar with statistical metrics such as CCDF may be familiar with the dose-volume-histogram (DVH) used in radiation oncology. The RFH is a metric that is similar to the DVH, using cumulative risk score instead of dose, and frequency of incidents instead of contour volume. A feature of the CCDF is that the integral, or area below the curve, represents the expectation value. Hence, the area below the RFH curve can be interpreted as the total practice risk (in arbitrary units).

96

Incident learning system improves patient safety

that existing safety barriers were not effective. The majority of the recurring incidents were caused by the presence of two locations on the form where the number of implanted sources was recorded. It was common for staff to enter the number of sources in only one of the locations instead of both. The intervention included changing the document from a multipurpose HDR and LDR document into a prostate-specific LDR document so all entries on the form were to be completed. In addition, obsolete information on the form was removed and logic was embedded to warn the user of common typos before printing. Fig. 2 (Left) provides a comparison of incident frequency before and after intervention. After intervention, the frequency of incidents (9% incidents/procedure from November 2014 through December 2015) was 5 times lower (p < 0.001) than the average rate in preceding years (45% incidents/procedure from January 2013 through October 2014). This change in the written directive did not compromise the state regulatory requirements or patient safety. The second example is an intervention made to a checklist used by the HDR treatment machine operator to verify that all safety steps are completed before initiating treatment. The ILS identified recurring un-checked items after treatment completion. The intervention included breaking the list up into three categories, re-ordering items according to procedure flow, and changing checkbox style from a large matrix to distinct boxes. These changes are consistent with published checklist guidelines [22]. Fig. 2 (Right) provides a comparison of incident frequency before and after intervention. The frequency of incidents after revision (0.4% incidents/procedures from November 2014 through December 2015) was over 3 times lower (p = 0.02) than the average rate in preceding years (1.3% incidents/procedure from January 2013 through October 2014).

Results ILS participation In 5258 procedures, a total of 2238 incidents were reported between January 1, 2007 and December 31, 2015. The yearly breakdown of patients and procedures administered is listed in Table 1. Large numbers of reports were anticipated because the intervention expanded reporting beyond events and near misses to include low risk process deviations that would not reach the patient. The rate of incident submission per patient, for LDR and HDR combined, increased during the first four years of the intervention from 2007 (0.09 incidents/patient) to 2011 (1.61 incidents/patient). After this initial ramp-up period, the rate of incident submission per patient was relatively stable between 2011 and 2015 (range: 1.56–1.89 incidents/patient) and was similar among HDR (1.74 incidents/patient) and LDR (1.60 incidents/ patient) treatments. HDR courses of treatment are frequently fractionated for the best therapeutic effect; hence, the number of procedures may exceed the number of patients. The number of HDR incidents per procedure averaged 0.43 (range: 0.38–0.56) between 2011 and 2015. We evaluated whether frequent performance of procedures was related to the frequency of incidents per procedure from 2011 to 2015. Spearman analysis was used to test for significance. Correlation was observed when comparing year-to-year variations for all anatomic sites (Fig. 1) for HDR (0.05 > p > 0.025) and LDR (0.15 > p > 0.10). Correlation was also observed between incident frequency and procedure frequency when binning procedures by anatomic site (HDR, p = 0.037; LDR, p = 0.67; Table 2).

ILS-Implemented changes and improvements in clinical practice

Dosimetric risk reduction The absolute frequency of a dose or violation of radiation safety policy risk (score > 1) steadily decreased from 2011 to 2015 and was 60% lower (p < 0.001) (Table 3) in 2015 than 2011. A risk score of 3 was more frequently observed in Table 3 than a risk score of 2 because potential violations of radiation safety policy (e.g. missing regulatory information on a form) were scored a value of 3. The relative distribution of dose risk scores was described using the RFH (Fig. 3). The dose risk RFH decreased monotonically at all dose risk thresholds from 2011 to 2015.

Practice improvements Our brachytherapy practice has enacted numerous practice changes between 2007 and 2015 in response to high score, high frequency, and/or red-highlighted ILS reports. The two examples provided in subsequent paragraphs illustrate the methodology employed toward continuous improvement and risk reduction. The first example is an intervention made to regulatory documentation, specifically the written directive for prostate seed implants. The ILS identified recurring incidents, which suggested

Table 1 Incident reporting frequency. The brachytherapy practice was encouraged to report all standard operating procedures deviations, including low risk deviations that do not reach the patient but could indirectly impact patient care (e.g. missing initials on a checklist). High Dose Rate (HDR) courses of treatment are frequently fractionated for the best therapeutic effect; hence, the number of procedures may exceed the number of patients. Low Dose Rate (LDR) patients received only one procedure per course of treatment. Year 2007 Low Dose Rate Brachytherapy (LDR) Incidents 21 Patients 176 Procedures 176 Incidents/Patients 0.12 Incidents/Procedures 0.12 High Dose Rate Brachytherapy (HDR) Incidents 0 Patients 65 Procedures 271 Incidents/Patients 0 Incidents/Procedures 0 LDR and HDR Brachytherapy Incidents 21 Patients 241 Procedures 447 Incidents/Patients 0.09 Incidents/Procedures 0.05

2008

2009

2010

2011

2012

2013

2014

2015

33 106 106 0.31 0.31

35 130 130 0.27 0.27

90 118 118 0.76 0.76

201 130 130 1.55 1.55

174 99 99 1.76 1.76

126 103 103 1.22 1.22

148 92 92 1.61 1.61

140 69 69 2.03 2.03

0 94 335 0 0

0 114 365 0 0

12 100 325 0.12 0.04

157 93 279 1.69 0.56

238 119 448 2.00 0.53

249 137 633 1.82 0.39

351 193 927 1.82 0.38

263 180 652 1.46 0.40

33 200 441 0.17 0.07

35 244 495 0.14 0.07

102 218 443 0.47 0.23

358 223 409 1.61 0.88

412 218 547 1.89 0.75

375 240 736 1.56 0.51

499 285 1019 1.75 0.49

403 249 721 1.62 0.56

97

C.L. Deufel et al. / Radiotherapy and Oncology 125 (2017) 94–100

Fig. 1. Relationship between incident frequency and procedure frequency between 2011 and 2015. The number of incidents per procedure is presented for High Dose Rate (HDR) (left) and Low Dose Rate (LDR) (right) treatment methods. Incidents were binned in one year increments. Spearman methods were used to test for a significant relationship.

Table 2 Relationship between incident frequency and number of procedures performed for various anatomic sites, between 2011 and 2015. LDR and HDR procedures were categorized according to anatomic treatment site. The number of incidents per procedure and average procedures per year is listed for each anatomic site between 2011 and 2015. Spearman methods were used to test for a significant relationship between incident frequency and average number of procedures per year. Anatomic Site

Average No. Procedures Per Year

Low Dose Rate Brachytherapy (LDR) Spine 1 Eye 45 Prostate 52 High Dose Rate Brachytherapy (HDR) Bile Duct 25 Prostate 18 Other** 125 Vaginal Cuff 209 Breast 265* * **

Incident Frequency: Incidents/Procedures (%)

P-value

6/5 (120) 363/226 (161) 419/262 (160)

0.67

84/123 (68) 58/89 (65) 356/623 (57) 467/1045 (45) 293/1059 (28)

0.037

Breast treatments were performed 2012–2015 only. Includes Vagina, Cervix, Extremity, and Intraoperative HDR.

Composite-risk score reduction Composite-risk scores also declined significantly between 2011 and 2015. The improvements were greater for the higher risk scoring thresholds (Table 3). For example, the absolute frequency of incidents exceeding the immediate action threshold (i.e. 50) decreased by 70% (p < 0.001) between 2011 and 2015 (Table 3). The relative distribution of composite-risk scores was described using the RFH (Fig. 3). Incident submissions migrated from higher risk toward lower risk between 2011 and 2015, and the area below the RFH curve was markedly reduced over time.

patient intervention (15 incidents), and other causes (24 incidents) (Table 3). The greatest improvements in the practice since 2011 have been made in improving staff communication and the quality of written procedures. The likelihood of incident due to poor communication during a procedure decreased by 60% (p < 0.001), and the likelihood of incident due to poor quality of written procedure decreased by 59% (p < 0.001) between 2011 and 2015.

Incident root causes Root causes were assigned to each incident. There often are multiple contributors to an incident, and therefore an incident may be identified as having more than one cause, and the total number of contributing causes in any one year divided by the number of reports may exceed 100%. Between 2011 and 2015, the failure modes attributed to the individual, which include distraction, fatigue, and transcription, were the largest contributor to incidents (1354 incidents), followed by equipment (614 incidents), new staff/trainee involvement (509 incidents), poor quality of written procedures (291 incidents), poor communication (134 incidents),

We have demonstrated successful implementation of an ILS system in the brachytherapy practice at an academic radiation oncology department. The analysis of 9 years of data demonstrates that, after a ramp-up period, the learning system was successful in identifying potential direct and indirect risks to the patients, and as result, practice changes were implemented to improve patient care. The intervention changed our practice’s approach to incident learning. Instead of capturing only events, near-misses, and incidents that reach the patient, the intervention asked staff to also submit low risk items that would not reach the patient. The submission of low risk reports helped the team to systematically

Discussion

98

Incident learning system improves patient safety

Fig. 2. Examples of how the ILS was used to improve clinical practice. (Left) An intervention was made to regulatory documentation, specifically the written directive for prostate seed implants. The majority of pre-intervention incidents were caused by the presence of two locations on the form where the number of implanted sources was recorded. The intervention included changing the document from a multipurpose HDR and LDR document into a prostate-specific LDR document so all entries on the form were to be completed. In addition, obsolete information on the form was removed and logic was embedded to warn the user of common typos before printing. (Right) An intervention was made to a checklist used by the HDR treatment machine operator to verify that all safety steps are completed before initiating treatment. The ILS identified recurring incidents of un-checked boxes after treatment completion. The intervention included dividing the checklist into three categories, re-ordering items according to procedure flow, and changing checkbox style from a large matrix to distinct boxes.

Table 3 Absolute frequency of risk scores and causes between 2011 and 2015. All incidents were assigned contributing causes, Dose Risk score, and a Composite-Risk score. An incident may be identified as having more than one contributing cause. Dose Risk scores range from 1 (low) to 5 (high). The Composite-Risk score can range from 1 (low) to 3125 (high) and was calculated as the multiplicative product of five categories: likelihood of recurrence, likelihood of quality assurance failure, likelihood of non-dose related severity, likelihood of dose related severity or radiation safety policy violation, and staff or patient time wasted. Chi-squared tests were performed with a = 0.05. Year 2011 Dose Risk Score 1 2 3 4 5 Score > 1 Composite-Risk Score 0–4 5 10 25 50 100 Incident Cause Human Elements** Equipment New Staff or Trainee Poor Quality of Written Procedure Poor Communication Patient Intervention Other

2015 vs. 2011 2012

2013

2014

2015

Absolute frequency of dose score, No. incidents/No. procedures(%) 226/409 (58) 265/547 (49) 251/736 (34) 358/1019 (35) 306/721 (43) 24/409 (6) 27/547 (5) 22/736 (3) 22/1019 (2) 23/721 (3) 84/409 (22) 103/547 (19) 95/736 (13) 105/1019 (10) 64/721 (9) 23/409 (6) 14/547 (3) 7/736 (1) 11/1019 (1) 9/721 (1) 1/409 (0) 2/547 (0) 0/736 (0) 3/1019 (0) 1/721 (0) 139/409 (34) 148/547 (27) 124/736 (17) 141/1019 (14) 98/721 (14) Absolute Frequency of composite-risk score, No. incidents/No. procedures(%) 93/409 (23) 103/547 (19) 100/736 (14) 175/1019 (17) 159/721 (22) 265/409 (65) 309/547 (56) 275/736 (37) 324/1019 (32) 244/721 (34) 186/409 (45) 197/547 (36) 176/736 (24) 171/1019 (17) 139/721 (19) 96/409 (23) 74/547 (14) 85/736 (12) 71/1019 (7) 62/721 (9) 48/409 (12) 33/547 (6) 32/736 (4) 31/1019 (3) 25/721 (3) 23/409 (6) 12/547 (2) 4/736 (1) 8/1019 (1) 10/721 (1) Absolute frequency of cause type, No. incidents/No. procedures(%)* 220/409 (54) 287/547 (52) 225/736 (31) 347/1019 (34) 275/721 (38) 101/409 (25) 132/547 (24) 127/736 (17) 149/1019 (15) 105/721 (15) 95/409 (23) 114/547 (21) 122/736 (17) 72/1019 (7) 106/721 (15) 65/409 (16) 68/547 (12) 50/736 (7) 61/1019 (6) 47/721 (7) 31/409 (8) 33/547 (6) 21/736 (3) 27/1019 (3) 22/721 (3) 6/409 (1) 2/547 (0) 1/736 (0) 1/1019 (0) 5/721 (1) 5/409 (1) 11/547 (2) 3/736 (0) 0/1019 (0) 5/721 (1)

%Change (2015–2011)/2011

P-value

27 48 59 79 46 60