International Journal for Quality in Health Care 2002; Volume 14, Number 4: pp. 321–327
Impact of end user involvement in implementing guidelines on routine preoperative tests FABRIZIO BARAZZONI1, ROBERTO GRILLI2, ANNA MARIA VINCENZA AMICOSANTE3, SONIA BRESCIANINI3, MICHELE A. MARCA1, MARCO BAGGI4, PAUL BIEGGER5 AND REZIO RENELLA6 1
Ente Ospedaliero Cantonale, Bellinzona, 4Ospedale Regionale Beata Vergine, Mendrisio, 5Ospedale Regionale La Carita`, Locarno, Ospedale Regionale, Sede Ospedale Civico, Lugano, Switzerland, 2Agenzia Sanitaria Regionale, Bologna, 3Agenzia per i Servizi Sanitari Regionali, Roma, Italy 6
Abstract Objectives. To assess the impact of health professionals’ involvement in the implementation of practice guidelines aimed at reducing the use of pre-operative tests in patients at low anaesthetic risk undergoing elective surgery. Intervention. A 6 month (September 1997 to February 1998) strategy based upon organization of local meetings involving health professionals from six hospitals of Canton Ticino (Switzerland). Design. Observational study (pre/post) of pre-operative test utilization between March 1996 and December 1998. Subjects and methods. A total of 17 978 patients admitted for elective surgery over the study period. The latter was modelled in six intervals, three before (baseline), one during, and two after (adoption) guidelines implementation, respectively. For each time interval the proportion of patients undergoing pre-operative tests was estimated. Multilevel logistic regression analysis was used to assess patient likelihood [expressed as the odds ratio (OR)] of undergoing a diagnostic test in each period, using the implementation interval as the reference category. Main outcome measure. Change in patient probability of undergoing pre-operative tests in the adoption interval. Results. Adoption of the recommendations was associated with 81% [OR=0.19; 95% confidence interval (CI) 0.15–0.23] reduction of patient probability of undergoing coagulation test, 73% (OR=0.27; 95% CI 0.23–0.33) for glycaemia, 62% (OR=0.38; 95% CI 0.33–0.44) for azotaemia, 57% (OR=0.43; 95% CI 0.36–0.51) for chest X-ray, 49% (OR=0.51; 95% CI 0.44–0.60) for creatinaemia, and 43% (OR=0.57; 95% CI 0.48–0.69) for ECG. Overall, these findings corresponded to a cost saving of 67 890 Swiss francs (US$42 000) for the last quarter under study. Conclusions. This study indicates that an implementation strategy based upon direct involvement of end users in the identification of possible barriers to change can be successful in promoting the use of practice guidelines. Keywords: implementation, practice guidelines, pre-operative tests
A great deal of research has been devoted to studying the effectiveness of different strategies aimed at changing health professionals’ performance, and their findings have been summarized by a number of systematic reviews [1]. Overall, the conclusions indicate that there is no single effective intervention, rather a plurality of approaches can be effective as long as they are tailored to the specific characteristics of the clinical environment representing a barrier to the desired change. In this paper we report the results of the implementation
of recommendations aimed at reducing the use of preoperative tests in the hospitals of Canton Ticino, Switzerland. This issue has been the object of systematic reviews and practice guidelines developed over the last 15 years by specialist societies and technology assessment agencies [2–6], in an attempt to reduce a practice that seems to be as much widespread as it is unsupported by empirical evidence [7,8]. Current research on practice guidelines indicates the relevance of the identification of barriers to the adoption of the recommended behaviours [9]. In principle, end-users (that
Address reprint requests to Fabrizio Barazzoni, Ente Ospedaliero Cantonale, Viale Officina 3, 6501 Bellinzona, Switzerland. E-mail:
[email protected]
2002 International Society for Quality in Health Care and Oxford University Press
321
F. Barazzoni et al.
is, those by whom practice guidelines are intended to be used) are in the best position to identify contextual factors that may hinder or favour their adherence to guidelines in current clinical practice. In our study we used an implementation strategy, relying as much as possible on the involvement of guideline end-users and on the identification of the obstacles to the adoption of the recommendations.
Materials and methods Study design This study is based upon the analysis of the patterns of use of pre-operative tests before, during, and after the implementation of guidelines aimed at reducing the use of unnecessary routine tests. Study setting Six (later reduced to five, as one was closed during the study period) acute care urban community hospitals with surgery facilities exist in Canton Ticino, serving approximately 310 000 residents, and accounting for 1049 acute beds in the public sector and 316 doctors. Facilities for general surgery, orthopaedics, neurosurgery, gynaecology, urology, and ophthalmology are available, and all of the hospitals host training activities. Development of the recommendations A number of guidelines on the use of pre-operative testing have been developed, including those issued by an Italian national consensus conference on this topic [10], held under the auspices of several specialist societies. In the framework of that conference, a multidisciplinary panel of experts, also including patient and consumer representatives, developed recommendations for adult patients undergoing elective surgery on the basis of a systematic review on the performance of different diagnostic tests in the pre-operative setting, and on the critical appraisal of pre-existing guidelines. The final recommendations were adopted by the medical chief of the Canton Ticino public hospital organization. The recommendations were not intended to be mandatory, rather they were presented to health professionals as ‘advised clinical behaviours’, and were not linked to any kind of financial incentive.
groups were volunteers, although every effort was made to assure the attendance of department/ward leaders. The discussion focused on the identification of possible barriers to the adoption of the guidelines by clinicians, as well as on the development of feasible strategies to overcome them. In these meetings, two issues consistently emerged: (i) concerns about the legal implications of a policy that would restrict the use of pre-operative tests; and (ii) the need for organizational changes in the anaesthesiology services. As far as the first is concerned, clinicians claimed that routine use of pre-operative tests had a ‘protective’ effect on health care providers in case of adverse events that could lead to legal action by patients or their relatives. As for the organizational barriers, the content of the guidelines called for the design of a pre-operative process of care structured in such a way as to allow an anaesthetic workup based upon a rigorous clinical examination. The results of the tests ordered had then to be seen by the anaesthesiologists before surgery. This policy, requiring a selective ordering of specific tests according to patient clinical history and findings at the clinical examination, appeared to be more complex than the usual one of simply ordering unselectively ‘everything for everybody’ when the decision of referring the patient for elective surgery was made. The legal issue was addressed through the involvement of the Swiss Institute of Health Rights, who were asked to produce a report on the legal implications of practice guidelines [11]. The content of the report made explicit that guidelines were not to be considered per se either a protection or a danger for the practising clinician, the key issue being the ability of the latter to justify his decision through clear and complete clinical documentation. The organizational aspects were addressed by changing the procedures at the time of hospital admission or when a surgical intervention was scheduled. In particular, a manual paper reminder was designed in the form of a patient profile checklist that had to be used by a physician or a nurse in patients admitted for elective surgery to order the required pre-operative tests. According to the presence of specific patients’ characteristics (recorded ticking a box), the reminder suggested the type of diagnostic tests recommended by the guidelines. The development of such a tool was suggested by some clinicians in the discussions held at the local meetings. Adoption of the guidelines
The implementation strategy The implementation of the recommendations was based upon the organization of local meetings. Multidisciplinary working groups were convened by the medical chief of the Canton Ticino public hospital organization in each participating hospital, inviting representatives from the different categories of health professionals involved (clinicians, nurses, managers). The main task of the working groups was to examine the clinical content of the recommendations and anticipate their organizational implications, taking into account the settings in which they had to be applied. Participants in the working
322
The clinical recommendations were officially adopted on March 1998, and after that were considered as the suggested clinical policy that surgical departments were encouraged to apply. Evaluation of the impact of the guidelines The evaluation of the impact of the guidelines was based on the analysis of the patterns of use of pre-operative tests from March 1996 to December 1998. For each patient, administrative data on clinical characteristics [including the
Implementation of guidelines on routine pre-operative tests
Table 1 Patient age and ASA class distribution over time Age class (years) (%) ASA class (%) ................................................................. ................................................................. Period n 15–35 36–65 [65 I II III ............................................................................................................................................................................................................................. Baseline Mar. 1996–Aug. 1996 4352 25.4 43.1 31.5 49.4 33.6 17.0 Sept. 1996–Feb. 1997 4363 24.9 43.8 31.3 41.8 38.2 20.0 Mar. 1997–Aug. 1997 2838 22.9 42.1 35.0 34.0 41.7 24.4 Implementation Sept. 1997–Feb. 1998 1193 20.0 40.5 39.6 33.1 48.9 18.0 Adoption Mar. 1998–Aug. 1998 2337 22.8 45.1 32.1 31.2 44.0 24.8 Sept. 1998–Dec. 1998 2190 23.7 44.7 31.6 36.6 39.6 23.8 Total Mar. 1996–Dec. 1998 17 273 23.9 43.5 32.7 39.7 39.3 21.0 ASA, American Society of Anesthesiologists.
severity of the anaesthetic risk, expressed through the American Society of Anesthesiology (ASA) scale] and the type of pre-operative diagnostic tests undertaken were available. Changes in the number of patients undergoing each individual pre-operative test were assessed, dividing the overall period of observation into six time intervals: three semesters (baseline, from March 1996 to August 1997) before the inception of the implementation, a fourth representing the implementation period (September 1997 to February 1998), and two following intervals representing the adoption phase (March to August 1998 and September to December 1998). For each interval the proportion of patients receiving preoperative tests was then estimated. A logistic regression model was employed in order to assess individual patients’ probability of receiving a test within a specific time interval [12]. In order to account for lack of independence among observations (i.e. individual patients cared for at the same hospital), a multilevel hierarchical model was adopted [13,14]. With this approach, the violation of the independence of observations assumption is compensated by analysing the effect at interest simultaneously within different hierarchical levels of aggregation (in this case represented by hospitals and patients, respectively). In this study we analysed the variation within the hospital level (second level). Variables included in the best fitting model were age (categorized in classes), anaesthetic risk according to the ASA classification (dummy variables, with ASA class III as the reference category), and time (dummy variables, with the semester of implementation, September 1997 to February 1998, representing the reference category). The effect of the implementation strategy is expressed in terms of patient probability of undergoing a pre-operative test represented by an odds ratio (OR) with 95% confidence interval (CI). Therefore, if the intervention is effective, the OR is 65 years old) of the whole study period, whereas the other two classes had the lowest proportions (20% for age 16–35 years and 40% for 36–65 years). As for the anaesthetic risk, in that semester, about half the patients were ASA class II, and a third were ASA class I, while the remaining (about 18%) were ASA class III. The proportion of patients undergoing each pre-operative
323
F. Barazzoni et al.
Table 2 Proportion of patients undergoing pre-operative tests by study period Proportion of patients undergoing pre-operative tests (%) ........................................................................................................................................................ Period n Azotaemia Coagulation test Creatinaemia Glycaemia ECG Chest X-ray ............................................................................................................................................................................................................................. Mar. 1996–Aug. 1996 4352 48.1 93.0 82.4 86.7 61.7 57.4 Sept. 1996–Feb. 1997 4363 50.5 91.2 80.0 84.3 62.8 56.6 Mar. 1997–Aug. 1997 2838 55.4 92.0 78.4 84.0 63.4 58.9 Sept. 1997–Feb. 1998 1193 49.0 90.8 67.9 83.3 62.0 51.1 Mar. 1998–Aug. 1998 2337 38.3 79.3 62.1 67.7 55.1 41.5 Sept. 1998–Dec. 1998 2190 29.3 64.8 54.5 58.9 48.3 32.0 Baseline 11 553 50.8 92.1 80.5 85.1 62.6 57.5 Implementation 1193 49.0 90.8 67.9 83.3 62.0 51.1 Adoption 4527 34.0 72.3 58.4 63.4 51.8 36.9 ECG, electrocardiogram.
test is shown in Table 2. Compared with the baseline period, there has been a decrease in the adoption phase of >20% for all the tests (22% for creatinaemia and glycaemia, 21% for chest X-rays, and 20% for coagulation test) apart from azotaemia and ECG, for which the reduction was, respectively, 17% and 11%. These findings were confirmed when data were analysed at the individual patient level through the multilevel logistic regression model. Table 3 shows the parameter estimates for what we believe to be the best model for our data (statistically significant G test, P < 0.0001, when compared with the model with fixed intercept). In this model we allow the intercept to vary randomly among hospitals. Only the systematic component coefficients are reported in Table 3. For all the tests the OR shows a significant decrease in the probability of undergoing such tests after the guideline adoption phase. For some of these, like coagulation test and glycaemia, the effect was evident from the beginning of the adoption phase, while for the remaining it appeared later in the study period. At the end of the period under study there was a reduction in the probability of undergoing each preoperative test: 81% (OR=0.19; 95% CI 0.15–0.23) reduction of patient probability of undergoing coagulation test; 73% (OR=0.27; 95% CI 0.23–0.33) for glycaemia; 62% (OR= 0.38; 95% CI 0.33–0.44) for azotaemia, 57% (OR=0.43; 95% CI 0.36–0.51) for chest X-ray; 49% (OR=0.51; 95% CI 0.44–0.60) for creatinaemia; and 43% (OR=0.57; 95% CI 0.48–0.69) for ECG. As expected, the patient’s age and ASA class emerged as being associated with the use of pre-operative tests, which were used less frequently in patients younger and with lower anaesthetic risk. In addition, the effect of the implementation was more evident in small hospitals (i.e. with
